I always thought the Sith could come and go, but the Imperial bureaucracy (Navy, Intelligence) kept the show going on. Without them, the Empire would have fallen long ago.
In Sith Society, the Imperials are the peasants to working class. Some may even hold high enough rank to equate to Middle-Class. The Sith are the top 1%. They are the Ruling Class. The Imperials are the infrastructure. While it may be true the Imperials monitor themselves, the Sith do not just "Come and Go" they are an ever present, work-at-home parent. While the lower ranking Sith are off fighting wars and such, the upper echelons of the Sith ensure the laws of the Emperor are conveyed to the Imperials.