You really shouldn't have any problems. In fact, shadow priests are a strong 5/10 man class given their competitive dps, ability to cc a common mob type that many other classes have an issue with, their ability to resurrect in the inevitable case of random deaths and the ability to off-heal (although diminished severely if gearing for max dps in +shadow gear, since off-healing is often done on fumes with low rank heals).
However, you
might struggle anyway since many players will be unable to see that suboptimal in a raid setting =/= suboptimal in 5-mans.
varth wrote:mages and locks [...] easily [do] 50% more dps
Nah, not in 5-mans. In a raid setting sure. I've topped meters in 5-mans smiting and that is without raid gear since I don't get any smite gear.

*waves fist at mumin*
The problem with shadow is not output, it's sustainability without giving up output. And ofc debuff slots (even with 16), but that is really secondary tbh.