I was always under the impression midichlorians could be looked at as "Gateways" to the force. The more you have in you, the more easily it flows through it's user. I'm assuming Jedi had higher midichlorians than the average Joe, hence qui gon's count.
A way to look at it is like an empty pool with a small drain, a garden hose and water. The pool being the person, while the garden hose is a midichlorian and water is the force. So the more hoses you have, the faster it is to fill. While an average joe may have 1-3. A Jedi could have a 1000 etc etc.