I guess it matters on what type of audience you're trying to attract (children,adults etc.), but i find using more descriptive words make it flow better. For example: “You're hardly worth talking about out of your own company,” he stated with composure then (retreating slightly, retreating). [link][link]
Huge edit? Pretentious? Far less pretentious than assuming your novel is perfect just the way it is.
In general--though I hate to suggest bending the rules as some sort of standard thing--I think commas can be used a little more loosely than most other punctuation. Rather than being something with a rigid use like an apostrophe or a huge effect like a full stop, they offer just a little pause. For character speech, that can be invaluable. For regular narration, it can sometimes put just a slightly different spin on things.
I don't like the look of that sentence without the comma either, but I'd also be inclined just to make the change =vglory suggested. "Stated coolly" doesn't really pull its weight, and the sentence is neater without it. No point fretting over small problems when you can just avoid having to deal with them!
I dealt with similar issues many years ago. Part of it was being a little rusty on formal usage, the other part was thinking that the commas served the function of adding pauses. So I was often adding commas where it felt right, denoting a breath. Thing is, while commas do sort of denote pauses, this isn't really the correct usage and often led to cluttered sentences. The reader will, in many cases, read those breaths without your having to denote them. In general, I would say it's best to stick with proper usage unless bending the rules actually benefits the expression. In most cases I've found it doesn't.
In the case of your example, I don't think dropping the comma would hurt much.