Welp, I was frantically digging and found a solution. Actually on one of my old posts on this forum!
There must be something with the Asus RT-N66U that causes this. Both times the PBX has been behind one and both times switching "IP Configuration" to "Public" has solved the issue.