I think this is mainly just how it works. In Asterisk Queues, the calls pick agents, rather than the agents picking calls. There is a little bit of logic to stop low priority queues overriding high priority ones, but, as many calls as there are free agents will be competing for the available agents, and if one is playing comfort messages, it will not be actively looking for an agent.
If by IVR, you really mean just playing “your call is important to us, please wait”, you can record a special musiconhold track that embeds those messages.
(I can’t remember what it does with two equal priority queues that share the same agents, but I suspect that it may not reduce the number of calls from the top of the queues that are actually looking for agents, so calls further down the queue may end up getting an agent. Normally this isn’t too much of an issue, unless multiple agents come free in the same second, as there will rarely be more than one agent free when queues are heavily loaded.)