without threads for both it would require doing non blocking reads on the rtp streams. We have some provisions in the code for being able to do this but I am not sure it's been proven to work. We can always try to dust it off. I think its done by passing the io_flag SWITCH_IO_FLAG_NOBLOCK to the read_frame call. Another idea would be to open the audio and video tracks separately each in it's own thread and play them independently since it's 2 different IP streams. 2010/7/13 Paulo Rogério Panhoto <paulo@voicetechnology.com.br>
I've found a way to synchronize the A/V streams with usleep() but I don't think that's the best way to do this. Is there a switch_core.... function to perform a similar task (and maybe allow other tasks to run)?
Here's the current version of the playback loop (that takes into consideration multiple RTP packets per video frame) :
u_int64_t videoNext = 0, audioNext = 0; u_int64_t ts = 0;
bool videoSent = true, audioSent = true;
while (switch_channel_ready(channel)) { bool vOk, aOk; if(videoSent)
{ vid_frame.packetlen = vid_frame.buflen; vOk = vc.getVideoPacket(vid_frame.packet, vid_frame.packetlen); if (vOk) { switch_rtp_hdr_t *hdr = reinterpret_cast<switch_rtp_hdr_t *>(vid_frame.packet);
videoNext = vc.videoTrack().track.get90KTimestamp(ntohl(hdr->ts)); hdr->ts = htonl(videoNext);
if (pt) hdr->pt = pt; } videoSent = false; } if(audioSent)
{ write_frame.datalen = write_frame.buflen; aOk = vc.getAudioPacket(write_frame.data, write_frame.datalen); audioSent = false; }
if (!vOk && !aOk) break;
int64_t wait = min(audioNext, videoNext) - ts; if(wait > 0) { /* wait the time for the next A/V frame */ usleep(wait * 1000 / 90); //switch_core_timer_next(&timer); ts += wait; }
if (switch_channel_test_flag(channel, CF_VIDEO) && ts >= videoNext)
{ switch_byte_t *data = (switch_byte_t *) vid_frame.packet;
vid_frame.data = data + 12; vid_frame.datalen = vid_frame.packetlen - 12; switch_core_session_write_video_frame(session, &vid_frame, SWITCH_IO_FLAG_NONE, 0); videoSent = true; }
if(aOk && ts >= audioNext)
{ if (write_frame.datalen > (int) write_frame.buflen) write_frame.datalen = write_frame.buflen;
switch_core_session_write_frame(session, &write_frame, SWITCH_IO_FLAG_NONE, 0); audioSent = true; audioNext += 90000 / 50; // 20ms } }
Btw, thanks for the tip. I'll keep the separate thread to store video for the record function.
Regards,
Paulo
On 13/07/10 15:39, Anthony Minessale wrote:
we need to get nonblocking reads to the rtp working so the
read_video_frame(); read_audio_frame();
don't block when there is no data.
_______________________________________________ FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:+19193869900