Upto 3 can group the fourth can enter group finder as a solo and then dont accept the group unless your friend is listed. This works.
sometimes doing this will knock you out of the queue or prevent you from declining after three tries. i have had this happen but am not sure if it was a bug or what ever. also you must be careful who you select as the fourth person as there will be a butt load of dps queued ahead of your fourth.