Honestly, I don't recommend playing this game without at least 6 gbs of ram. There are some memory leak issues and even with 8 gbs of ram I can't play for more than a few hours with having to restart the client.
This shouldn't be happening on a 64bit system, I know it doesn't happen on mine ( I gather this is what you have, as you can put as much ram as you like in 32bit systems and it still wont use over 4Gb, actual usage will always be less then 4Gb due to driver remapping for devices like video cards )
Edit - for those truely interested in memory and how it is handled between Windows OS releases including Win 8 the below link cover's it. (not wanting to start a war over PAE either with 32bit OSs like XP, MS have limited the kernal even with PAE enabled)