I would say Force Sensitive use the force to operate the switches / buttons / triggers on the lightsabers.
So the Lightsabers do not need necessary
have outside visible buttons.
When you disarm them, they trained and learned to purposely let's go / switch /"press" the activation mechanism of a Lightsaber. (Maybe the switch needs constant focusing to operate and will disable with the break of focus)
That concept makes way more sense. Non force users can still operate Lightsabers if they got visible outside switches, and force users do not need to "touch" to change states of the Lightsaber.
I'd personally would make my lightsaber constat float next to my hand, instead of attaching it to my belt and let it swing, but then again i prolly wouldn't be a jedi.