[Freeswitch-dev] Video synchronization
Anthony Minessale
anthony.minessale at gmail.com
Tue Jul 13 14:30:36 PDT 2010
each channel runs in a separate thread. so in playback you could use the
current thread for audio and launch another for the video. Since they are 2
different rtp streams you could read and write from them at the same time.
2010/7/13 Paulo Rogério Panhoto <paulo at voicetechnology.com.br>
> Perhaps threading is the best to do. Does FS handle calls in separate
> threads?
>
> 2010/7/13 Anthony Minessale <anthony.minessale at gmail.com>
>
> without threads for both it would require doing non blocking reads on the
>> rtp streams.
>> We have some provisions in the code for being able to do this but I am not
>> sure it's been proven to work.
>> We can always try to dust it off.
>>
>> I think its done by passing the io_flag SWITCH_IO_FLAG_NOBLOCK to the
>> read_frame call.
>>
>> Another idea would be to open the audio and video tracks separately each
>> in it's own thread and play them independently since it's 2 different IP
>> streams.
>>
>>
>>
>> 2010/7/13 Paulo Rogério Panhoto <paulo at voicetechnology.com.br>
>>
>>> I've found a way to synchronize the A/V streams with usleep() but I
>>> don't think that's the best way to do this. Is there a switch_core....
>>> function to perform a similar task (and maybe allow other tasks to run)?
>>>
>>>
>>> Here's the current version of the playback loop (that takes into
>>> consideration multiple RTP packets per video frame) :
>>>
>>> u_int64_t videoNext = 0, audioNext = 0;
>>> u_int64_t ts = 0;
>>>
>>> bool videoSent = true, audioSent = true;
>>>
>>> while (switch_channel_ready(channel))
>>> {
>>> bool vOk, aOk;
>>> if(videoSent)
>>>
>>> {
>>> vid_frame.packetlen = vid_frame.buflen;
>>> vOk = vc.getVideoPacket(vid_frame.packet,
>>> vid_frame.packetlen);
>>> if (vOk)
>>> {
>>> switch_rtp_hdr_t *hdr =
>>> reinterpret_cast<switch_rtp_hdr_t *>(vid_frame.packet);
>>>
>>> videoNext =
>>> vc.videoTrack().track.get90KTimestamp(ntohl(hdr->ts));
>>> hdr->ts = htonl(videoNext);
>>>
>>> if (pt)
>>> hdr->pt = pt;
>>> }
>>> videoSent = false;
>>> }
>>> if(audioSent)
>>>
>>> {
>>> write_frame.datalen = write_frame.buflen;
>>> aOk = vc.getAudioPacket(write_frame.data,
>>> write_frame.datalen);
>>> audioSent = false;
>>> }
>>>
>>> if (!vOk && !aOk)
>>> break;
>>>
>>>
>>> int64_t wait = min(audioNext, videoNext) - ts;
>>> if(wait > 0)
>>> {
>>> /* wait the time for the next A/V frame */
>>> usleep(wait * 1000 / 90);
>>> //switch_core_timer_next(&timer);
>>> ts += wait;
>>> }
>>>
>>> if (switch_channel_test_flag(channel, CF_VIDEO) && ts >=
>>> videoNext)
>>>
>>> {
>>> switch_byte_t *data = (switch_byte_t *) vid_frame.packet;
>>>
>>> vid_frame.data = data + 12;
>>> vid_frame.datalen = vid_frame.packetlen - 12;
>>> switch_core_session_write_video_frame(session,
>>> &vid_frame, SWITCH_IO_FLAG_NONE, 0);
>>> videoSent = true;
>>> }
>>>
>>> if(aOk && ts >= audioNext)
>>>
>>> {
>>> if (write_frame.datalen > (int) write_frame.buflen)
>>> write_frame.datalen = write_frame.buflen;
>>>
>>> switch_core_session_write_frame(session, &write_frame,
>>> SWITCH_IO_FLAG_NONE, 0);
>>> audioSent = true;
>>> audioNext += 90000 / 50; // 20ms
>>> }
>>> }
>>>
>>> Btw, thanks for the tip. I'll keep the separate thread to store video
>>> for the record function.
>>>
>>> Regards,
>>>
>>> Paulo
>>>
>>>
>>> On 13/07/10 15:39, Anthony Minessale wrote:
>>>
>>> we need to get nonblocking reads to the rtp working so the
>>>
>>> read_video_frame();
>>> read_audio_frame();
>>>
>>> don't block when there is no data.
>>>
>>>
>>>
>>> _______________________________________________
>>> FreeSWITCH-dev mailing list
>>> FreeSWITCH-dev at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
>>> http://www.freeswitch.org
>>>
>>>
>>
>>
>> --
>> Anthony Minessale II
>>
>> FreeSWITCH http://www.freeswitch.org/
>> ClueCon http://www.cluecon.com/
>> Twitter: http://twitter.com/FreeSWITCH_wire
>>
>> AIM: anthm
>> MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
>> IRC: irc.freenode.net #freeswitch
>>
>> FreeSWITCH Developer Conference
>> sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
>> googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
>> pstn:+19193869900
>>
>> _______________________________________________
>> FreeSWITCH-dev mailing list
>> FreeSWITCH-dev at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
>> http://www.freeswitch.org
>>
>>
>
> _______________________________________________
> FreeSWITCH-dev mailing list
> FreeSWITCH-dev at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
> http://www.freeswitch.org
>
>
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:+19193869900
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-dev/attachments/20100713/a848cef4/attachment.html
More information about the FreeSWITCH-dev
mailing list