[Freeswitch-dev] Video synchronization

[Paulo Rogério Panhoto]

Perhaps threading is the best to do. Does FS handle calls in separate
threads?

2010/7/13 Anthony Minessale <anthony.minessale at gmail.com>

> without threads for both it would require doing non blocking reads on the
> rtp streams.
> We have some provisions in the code for being able to do this but I am not
> sure it's been proven to work.
> We can always try to dust it off.
>
> I think its done by passing the io_flag SWITCH_IO_FLAG_NOBLOCK to the
> read_frame call.
>
> Another idea would be to open the audio and video tracks separately each in
> it's own thread and play them independently since it's 2 different IP
> streams.
>
>
>
> 2010/7/13 Paulo Rogério Panhoto <paulo at voicetechnology.com.br>
>
>>      I've found a way to synchronize the A/V streams with usleep() but I
>> don't think that's the best way to do this. Is there a switch_core....
>> function to perform a similar task (and maybe allow other tasks to run)?
>>
>>
>>     Here's the current version of the playback loop (that takes into
>> consideration multiple RTP packets per video frame) :
>>
>>         u_int64_t videoNext = 0, audioNext = 0;
>>         u_int64_t ts = 0;
>>
>>         bool videoSent = true, audioSent = true;
>>
>>         while (switch_channel_ready(channel))
>>         {
>>             bool vOk, aOk;
>>             if(videoSent)
>>
>>             {
>>                 vid_frame.packetlen = vid_frame.buflen;
>>                 vOk = vc.getVideoPacket(vid_frame.packet,
>> vid_frame.packetlen);
>>                 if (vOk)
>>                 {
>>                     switch_rtp_hdr_t *hdr =
>> reinterpret_cast<switch_rtp_hdr_t *>(vid_frame.packet);
>>
>>                     videoNext =
>> vc.videoTrack().track.get90KTimestamp(ntohl(hdr->ts));
>>                     hdr->ts = htonl(videoNext);
>>
>>                     if (pt)
>>                         hdr->pt = pt;
>>                 }
>>                 videoSent = false;
>>             }
>>             if(audioSent)
>>
>>             {
>>                 write_frame.datalen = write_frame.buflen;
>>                 aOk = vc.getAudioPacket(write_frame.data,
>> write_frame.datalen);
>>                 audioSent = false;
>>             }
>>
>>             if (!vOk && !aOk)
>>                 break;
>>
>>
>>             int64_t wait = min(audioNext, videoNext) - ts;
>>             if(wait > 0)
>>             {
>>                 /* wait the time for the next A/V frame */
>>                 usleep(wait * 1000 / 90);
>>                 //switch_core_timer_next(&timer);
>>                 ts += wait;
>>             }
>>
>>             if (switch_channel_test_flag(channel, CF_VIDEO) && ts >=
>> videoNext)
>>
>>             {
>>                 switch_byte_t *data = (switch_byte_t *) vid_frame.packet;
>>
>>                 vid_frame.data = data + 12;
>>                 vid_frame.datalen = vid_frame.packetlen - 12;
>>                 switch_core_session_write_video_frame(session, &vid_frame,
>> SWITCH_IO_FLAG_NONE, 0);
>>                 videoSent = true;
>>             }
>>
>>             if(aOk && ts >= audioNext)
>>
>>             {
>>                 if (write_frame.datalen > (int) write_frame.buflen)
>>                     write_frame.datalen = write_frame.buflen;
>>
>>                 switch_core_session_write_frame(session, &write_frame,
>> SWITCH_IO_FLAG_NONE, 0);
>>                 audioSent = true;
>>                 audioNext += 90000 / 50; // 20ms
>>             }
>>         }
>>
>>     Btw, thanks for the tip. I'll keep the separate thread to store video
>> for the record function.
>>
>>     Regards,
>>
>>     Paulo
>>
>>
>> On 13/07/10 15:39, Anthony Minessale wrote:
>>
>> we need to get nonblocking reads to the rtp working so the
>>
>>   read_video_frame();
>>    read_audio_frame();
>>
>>  don't block when there is no data.
>>
>>
>>
>> _______________________________________________
>> FreeSWITCH-dev mailing list
>> FreeSWITCH-dev at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
>> http://www.freeswitch.org
>>
>>
>
>
> --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
> googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
> pstn:+19193869900
>
> _______________________________________________
> FreeSWITCH-dev mailing list
> FreeSWITCH-dev at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-dev/attachments/20100713/d507bfbf/attachment-0001.html