The issues run much deeper than that. Rarely is the answer to performance issues "throw some stronger hardware at it". Usually it involves much more time consuming game engine optimizations etc.
Just my own personal observation here but I noticed more slowness happening after the Collections feature was added than I had noticed before. Things like the quest boxes taking a long time to load on fleet never happened to me prior to 2.1 which was when Collections hit.
The last go round of double XP certainly didn't have all these issues for me and I was playing on The Harbinger then too. However at that time we did not have the collections feature nor did we have to bear the load of extra players from the APAC server shutdown so one or both of those may be playing a factor.
I'd imagine Collections eats up a lot of processing power checking every item on every player. I believe that was the reason given why it wasn't feasible to have every single piece of armor add to collections separately rather than the sets, because all the checks for all those items would just slow things down too much. Still, even as it stands I suspect it's contributing to some of the slowness.