My server is the same way. PVP doesn't happen during the week unless its those few prim time hour, even on weekends earlier in the day finding pvp matches can be.....
I hate going up against the same handful of empire people that are decked out in war hero junk (I just started pvping when i hit 50, 3 weeks or so ago) while I am in partial BM, i have troubles "bursting" down the war hero "clothies" spec'd as gunnery.
IDC about that at the moment, just because PVP in this game is gear dependent, i just hate the fact i face the same handful of people, and how long it takes me to get my wins... must be my lack of skill