1. if they're so powerful, how were they captured?
2. why not just kill them? that's the question with just about every prisoner on belsavis
1) The Dread Masters are non-combatants and, being sith, don't have much of a connection to each other. The codex states that they meditate and/or accept gifts from supplicants as well as updating the emperor what the latest thing they saw in regards to future is. It isn't stated that the fight wars, just being the generals of armies, this doesn't have to be a combative position either. Due to the fact that they favour wielding fear, it is very likely that they felt that they wouldn't have had to even enter melee combat as everyone just lost their will to fight/fainted/peed their pants . . .among other things and therefore can't attack.
How you separate them is another can of worms as you would need at least a few master jedi and a lot of folk that are used to working through primal fear (that could be developed by facing manny, many sith and surviving the encounters). But, once they are separated and alone, I'll wager that it'll be an easy victory.
2) It is as Temeluchus said,
2. If you played KOTOR at all you'd know the Jedi do not kill their prisoners. Instead they imprison them or screw with their minds a la Revan. What they should have done at the very least was separate them in Belsavis, as they are more powerful together. The Republic had to learn the hard way to at least put them in stasis after they killed every living thing within a mile radius in their prison until the Republic smartened up enough and put them to sleep.