each channel runs in a separate thread. so in playback you could use the current thread for audio and launch another for the video.  Since they are 2 different rtp streams you could read and write from them at the same time.



2010/7/13 Paulo Rogério Panhoto <paulo@voicetechnology.com.br>
Perhaps threading is the best to do. Does FS handle calls in separate threads?

2010/7/13 Anthony Minessale <anthony.minessale@gmail.com>

without threads for both it would require doing non blocking reads on the rtp streams.
We have some provisions in the code for being able to do this but I am not sure it's been proven to work.
We can always try to dust it off.

I think its done by passing the io_flag SWITCH_IO_FLAG_NOBLOCK to the read_frame call.

Another idea would be to open the audio and video tracks separately each in it's own thread and play them independently since it's 2 different IP streams.



2010/7/13 Paulo Rogério Panhoto <paulo@voicetechnology.com.br>
    I've found a way to synchronize the A/V streams with usleep() but I don't think that's the best way to do this. Is there a switch_core.... function to perform a similar task (and maybe allow other tasks to run)?


    Here's the current version of the playback loop (that takes into consideration multiple RTP packets per video frame) :

        u_int64_t videoNext = 0, audioNext = 0;
        u_int64_t ts = 0;
       
        bool videoSent = true, audioSent = true;

        while (switch_channel_ready(channel))
        {
            bool vOk, aOk;
            if(videoSent)

            {
                vid_frame.packetlen = vid_frame.buflen;
                vOk = vc.getVideoPacket(vid_frame.packet, vid_frame.packetlen);
                if (vOk)
                {
                    switch_rtp_hdr_t *hdr = reinterpret_cast<switch_rtp_hdr_t *>(vid_frame.packet);

                    videoNext = vc.videoTrack().track.get90KTimestamp(ntohl(hdr->ts));
                    hdr->ts = htonl(videoNext);

                    if (pt)
                        hdr->pt = pt;
                }
                videoSent = false;
            }
            if(audioSent)

            {
                write_frame.datalen = write_frame.buflen;
                aOk = vc.getAudioPacket(write_frame.data, write_frame.datalen);
                audioSent = false;
            }

            if (!vOk && !aOk)
                break;


            int64_t wait = min(audioNext, videoNext) - ts;
            if(wait > 0)
            {
                /* wait the time for the next A/V frame */
                usleep(wait * 1000 / 90);
                //switch_core_timer_next(&timer);
                ts += wait;
            }

            if (switch_channel_test_flag(channel, CF_VIDEO) && ts >= videoNext)

            {
                switch_byte_t *data = (switch_byte_t *) vid_frame.packet;

                vid_frame.data = data + 12;
                vid_frame.datalen = vid_frame.packetlen - 12;
                switch_core_session_write_video_frame(session, &vid_frame, SWITCH_IO_FLAG_NONE, 0);
                videoSent = true;
            }

            if(aOk && ts >= audioNext)

            {
                if (write_frame.datalen > (int) write_frame.buflen)
                    write_frame.datalen = write_frame.buflen;

                switch_core_session_write_frame(session, &write_frame, SWITCH_IO_FLAG_NONE, 0);
                audioSent = true;
                audioNext += 90000 / 50; // 20ms
            }
        }

    Btw, thanks for the tip. I'll keep the separate thread to store video for the record function.

    Regards,

    Paulo


On 13/07/10 15:39, Anthony Minessale wrote:
we need to get nonblocking reads to the rtp working so the 

  read_video_frame();
   read_audio_frame();

don't block when there is no data.



_______________________________________________
FreeSWITCH-dev mailing list
FreeSWITCH-dev@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
http://www.freeswitch.org



_______________________________________________
FreeSWITCH-dev mailing list
FreeSWITCH-dev@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
http://www.freeswitch.org




--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
googletalk:conf+888@conference.freeswitch.org
pstn:+19193869900