It's not something NVidia/AMD side. Even though i play at 40-60FPS most of the time, it will spike occasionally especially on the fleets or any place having a huge population. The game does seem smoother with SWTOR Unleashed but loading and unloading it is too much of a bother. Maybe you should try it if you have 8GB+ RAM.
Load just DiscCasheArena file and leave it there ... it works without messing updates because this file is like Windows swap file - recreated when needed/missing automatically.
On old PC with 6 GB RAM - same file on Unleashed reduce measurable spikes and loading delays.
Check your HDD load from bloatware, torrents, browsers ... power option also.