You don't need cross server que. The only reason the the pool for rated wz is small is because the larger experienced PvP guilds have rofl stomped the casual teams. If they introduce brackets or something to that effect more people will play and be able to gain experience.
And many, many PVP players have already left the game as well. Rated WZs are so lame. Yeah I wan't to showcase my PVP skill by playing the same 3 WZs five hundred times and beat up on teams of newbies. Sounds pretty impressive.