Set up net.ipv4.ip_nonlocal_bind = 1 in /etc/sysctl.conf so FreeSWITCH can still set up a binding even if the IP address isn't on the box
I've previously used OVH with the failover IP addresses. There are a few nice things you can do.
Using pacemaker / corosync, you can create the floating IP as a custom resource with scripts to automatically move the IP address using API calls. Couple this with the standard IP address resource to make the IP active on one server at a time.
Configure session storage in a shared database such, then use a pacemaker resource to perform a "sofia recover" loading in all active sessions from the failed server, maintaining any active calls.
Kind regards,