Maybe I havent done enough group finder queues(guessing I've done around 30-40) but I have yet to see votekick failing to do its job or seen it abused.
A quick list of things I have initiated or approved votekick for:
* A dps dies randomly in FE, has to run back. 5 minutes later the rest of the group have cleared to the next boss and he is still in fleet. When asked why he wasn't running he said "Im omw".
* Some dude pressing need on every single item(including bossdrops for other classes)
* A guy says "we are doing the optional boss", the other 3 members say "no". Guy goes ahead and pulls a bunch of trash we intended to skip, dies and starts cursing.
Of course, in these scenarios getting three people on board with a vote kick doesnt seem unreasonable so perhaps OP is right. I just have yet to see it abused, though.