Sounds like either your connection or your server are being flaky.
If it's not happening to everyone else on your server, it's most likely your internet connection or your ISP.
Well I only have team speak running in the back ground. Router drivers updated. Just flushed DNS. Ermm ran K-Tera just fine recently with no issues... flaky ISP doesn't seem reasonable if it's only root caused to this client. No issues with Dungeon Defenders either with disconnects /shrug.
EDIT: BTW I'm only posting here when I get error'd so you can see how often it's happening since my original post :P