It's not the character that is the problem, it's the class story she is part of. Kaliyo stands out like a sore thumb as a companion of the Imperial Agent unless you let her "taint" you.
This is essentially the case with all class stories, almost all of the classes have a companion character in them that sticks out a little too much that you might want to swap out for another one. Kaliyo, for instance, I gather would make more sense for a Bounty Hunter (better her than Skadge >_>) than the Agent. She might even fit better with the Sith Warrior.
Likewise, I'd switch out Rusk from the Jedi Knight for Yuun of the Trooper. Makes more sense to me.
Which is why I don't understand the Skadge hate for BH. Both Skadge and the Bounty HUnter are pretty much killers and not particularly loyal to either Empire or Republic. I still feel Skadge is the only character that jives with the BH I have. The other companions seem way too soft and not hard enough. The Bounty Hunters that Maul and Savage fought recently for example in Clone Wars all seemed cooler and badder than any of the BH companions except Skadge.