[Freeswitch-users] Slow to open the inbound RTP port - wait for 183 response

I am trying to figure out why Freeswitch is sometimes slow to open the inbound RTP port. Have you run into anything that might affect that aspect of FS's behavior? FS seems to wait for the 183 response following an Invite. But if the remote side starts sending RTP data to the port FS said to use (in the SDP), then the RTP bounces back and the sender might switch to a different source port. And then we can end up with asymmetrical port relationships, and possibly one-way audio. So I'm looking for a mechanism to tell FS to get that RTP port open as soon as possible, and not to wait for the 183 response. Might have something to do with the "ignore-early-media" setting, but not sure yet?