we need to get nonblocking reads to the rtp working so the read_video_frame(); read_audio_frame(); don't block when there is no data. 2010/7/13 Paulo Rogério Panhoto <paulo@voicetechnology.com.br>
Hi,
This is my first post to this list and I hope this is the right one to ask. For the last couple of weeks, I've been working on a mod to play MP4 Video files (recording will be the next step). For simplicity, I chose to use the same kind of MP4 files as the Asterisk app: H26x/MPEG Video track + PCMU audio track. I used as base the mod_fsv code and added code based on the mpeg4ip library. I've actually got a video to work but I'm having some problem to synchronize the audio and video. I have an idea of how the algorithm should be but I don't know how to use the switch_core_timer to implement it.
the current loop is much like mod_fsv:
while (switch_channel_ready(channel)) { vid_frame.packetlen = vid_frame.buflen; write_frame.datalen = write_frame.buflen; bool vOk = vc.getVideoPacket(vid_frame.packet, vid_frame.packetlen); bool aOk = vc.getAudioPacket(write_frame.data, write_frame.datalen);
if (!vOk && !aOk) break;
if (vOk) { switch_rtp_hdr_t *hdr = reinterpret_cast<switch_rtp_hdr_t *>(vid_frame.packet);
// Adjusts timestamp to standard 90KHz clock. ts = ntohl(hdr->ts) * 90000 / vc.videoTrack().track.clock; hdr->ts = htonl(ts); if (pt) hdr->pt = pt;
if (switch_channel_test_flag(channel, CF_VIDEO)) { switch_byte_t *data = (switch_byte_t *) vid_frame.packet;
vid_frame.data = data + 12; vid_frame.datalen = vid_frame.packetlen - 12; switch_core_session_write_video_frame(session, &vid_frame, SWITCH_IO_FLAG_NONE, 0); } if (ts && last && last != ts) { switch_cond_next(); } last = ts; }
if(aOk) { if (write_frame.datalen > (int) write_frame.buflen) write_frame.datalen = write_frame.buflen;
switch_core_session_write_frame(session, &write_frame, SWITCH_IO_FLAG_NONE, 0);
} switch_core_timer_next(&timer); }
I think it should be like this (pseudocode)
nextAudio = nextVideo = ts = 0
while(switch_channel_ready()) { read_video_frame(); read_audio_frame();
nextVideo = ntohl(hdr->ts) * 90000 / video.clock;
waitTime = min(nextAudio, nextVideo) - ts; if(waitTime > 0) { wait(waitTime); ts += waitTime; }
if(ts >= nextAudio) { send_audio_frame(); nextAudio += 20 * 90/8; // use the same clock source } if(ts >= nextVideo) { send_video_frame(); } }
Any suggestion is much appreciated.
Regards,
Paulo R. Panhoto
_______________________________________________ FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:+19193869900