From what i can tell a sith could only have been slave to another sith or another very important figure of the empire.
Or the Hutt Cartel or the various other crime syndicates, since underworld dealings usually center around spice/slaves/weapons/tech (in no particular order). If you go that route then the planet you're looking for is Nar Shaddaa. It's entirely plausible that someone enslaved in the underworld could be plucked out by a Sith if said slave showed unusual potential...but it's your backstory, and good luck with it
I still haven't constructed more than the most basic backstories for about half of my legacy of ten characters.