I've opened a BUG.
http://jira.freeswitch.org/browse/FS-3739

I don't know what's you FS configuration.
What I know is:
- SIP "media proxy mode" is OK.
- re-INVITE for codec modification to image T.38 (t38-passthrough) is OK in all SIP "media mode".

The "late_negotiation" + "inherit_codec" trick to avoid transcoding in a Back2Back
(that is no media proxy) configuration works only for the very first call setup SDP offer/answer.
http://wiki.freeswitch.org/wiki/Codec_negotiation

Once the call is established, further codec modification via re-INVITE are usually closed on
each leg (except for t38) and this can led to transcoding (and obvious errors if you can't or don't want
to transcode).

Hope this helps you find out your problem.

Giovanni