fail2ban is as effective as the regexes you build to use against the log files that expose the IP addresses of the bad guys.
I will disagree with "asterisk will take a random port" , it won't. If you are talking about rtp connections, don't worry, apart from a vague possibility that a local host on your network might listen to a phone call without using srtp then there is no risk at all..
Yes 99.99% of all attacks originate on ports 5000-5999, 99.786% on 5060/5061, just don't unnecessarily expose your self .