They didn't have to arrest him right away. It's just that they know that the senate is under the influence of a sith lord, they know that there is darkness surrounding palpatine and they know he's gaining too much power too quickly and they know that he is acting suspiciously enough for them to send anakin to spy on him......
"Hey, Chancellor Palpatine is a sith lord!"
All Jedi jaws drop to the floor and they're like "WHHHHHHAAAAAAAAAAT????"
Just because all that was going on around him, doesn't make him the sith lord they were looking for. If Palpatine wasn't the sith lord, he could have been somewhere influencing the minds of the senators to give Palpatine more power and corrupting Palpatine to do his bidding.