All they have is a crappy small city on a jungle planet, which would blow up if they don't keep a 24/7 watch on their lightning spires. Plus they can't go anywhere since major portions of their own planet are now controlled by rebelling slaves, a rogue Sith lord, an invading mercenary army and a bunch of angry Mandalorians.
I wonder how they can ever get something done, when they can't even control their homeworld ?!
Republic is just the same, with squabbling planetary governments threatening to leave the Republic, under-supplied army which has to contract mercenaries with rich criminal records and doubtful loyalties, and the capital city underworld controlled by gangster clans.
To make real life parallels. Every state that has nuclear bombs is taken seriously, independently of their economical and political situation and stability. And actually, the less stable they are, the more seriously they should be taken. As a threat to the stability and prosperity of the rest of the world. So, be the Empire stable and prosperous, why would they need to threaten nearby worlds? Those worlds would seek membership in the Empire themselves...
I think that the storylines on the Empire side make perfect sense to describe the conflict and the atmosphere.