He could never have expected it. Ulic's return to the Light was completely unexpected. You can't really compare Exar Kun to Thrawn or Revan, as he had a different command strategy. Exar Kun employed a plan to devastate the galaxy, not conquer it.
I think Exar Kuns mistake was that he didn't expect Ulic to turn back to abandon the dark side. His mistake was similar to Tarkin's mistake in the end of Episode IV. He was not as good a tactician as the three first and he wasn't as good as Revan in planning long-time strategies, but he could still have conquered the galaxy, I think.
Wiping out the Jedi is impossible, though. Just as wiping out the Sith. They always return somehow...
Hmm, Tython. Wasn't it lost at that time?
Knowledge of Tython still remained, but the Jedi had abandoned it for Ossus. Exar Kun unleashed the terentateks on the world to devastate. Tython was cleansed of the Terentateks during the Great Hunt.