Stormtroopers are an elite independent force in the same vein as the German Waffen SS.
The Imperial Army wears black or grey uniforms and sloped helmets (You know the guys you see acting as security guards and attendants on the Death Star and Star Destroyers? Those guys are Navy Troopers who wear roughly the same getup).
The reason Stormtroopers are most commonly used in the place of the Imperial Army in the EU is because the Stormtroopers are far more distinct and evocative of Star Wars.
It's sort of like why the Republic Trooper in TOR looks like a Stormtrooper/Clone Commando instead of looking more like the regular uniformed troopers shown in earlier media; The white armor and facemask are iconic for Star Wars.
^ This really...
Stormtroopers are elite, you only see them in the movies because pretty much the entire time, the movies show things that are of high priority, hence why you see the Stormtroopers because they are an elite force.
The actual soldiers though, are the guys out of Stormtrooper armor.
Example on Endor, with the AT-STs the guys inside were apart of the Imperial Army.