Didn't a massive Sith army battle a Massive Jedi army on the planet Russan and that was the last encounter any Jedi had with a Sith until Qui-Gon and Obi-wan?
If so then how did Yoda know only two there always will be?
You're referring to the Bane Trilogy. Also in the Trilogy, Bane makes a point to allow a few unscrupulous types know, or at least think they know, of living Sith's existence.
Those rumors eventually made it back to the Jedi.