Ah, but your white list is working? Where did you put the whitelist? From within freepbx or in jail.conf? It wasn't working for me to even use ignoreip=, but since I've reinstalled fail2ban from this script: http://www.voip-info.org/wiki/view/Fail2Ban+(with+iptables)+And+Asterisk
So far so good!