G722 Local - G711 External but long delay

Hi there,
I have a running FreePBX Distro on a Vsphere Cluster. 2x Xeon 2620v3 16GB DDR3.

I have some problems with audio delay.
My SIP Provider uses G.711
I use G.722 for local calls.

When i call out with G.711 Codec everything is working find.
When i call out with G.722, Asterisk is Transcoding to G.711. But i have a delay from 0.5 to 1 Second. This is much too long.

My Audio Layer Latency with Transcoding from G722 -> G711 is arround 580ms

Any Tips, solutions? I dont know how to solve this