we need to get nonblocking reads to the rtp working so the 

  read_video_frame();
   read_audio_frame();

don't block when there is no data.


2010/7/13 Paulo Rogério Panhoto <paulo@voicetechnology.com.br>
   Hi,

   This is my first post to this list and I hope this is the right one
to ask. For the last couple of weeks, I've been working on a mod to play
MP4 Video files (recording will be the next step). For simplicity, I
chose to use the same kind of MP4 files as the Asterisk app: H26x/MPEG
Video track + PCMU audio track. I used as base the mod_fsv code and
added code based on the mpeg4ip library. I've actually got a video to
work but I'm having some problem to synchronize the audio and video. I
have an idea of how the algorithm should be but I don't know how to use
the switch_core_timer to implement it.

the current loop is much like mod_fsv:

   while (switch_channel_ready(channel))
   {
       vid_frame.packetlen = vid_frame.buflen;
       write_frame.datalen = write_frame.buflen;
       bool vOk = vc.getVideoPacket(vid_frame.packet, vid_frame.packetlen);
       bool aOk = vc.getAudioPacket(write_frame.data, write_frame.datalen);

       if (!vOk && !aOk)
           break;

       if (vOk)
       {
           switch_rtp_hdr_t *hdr = reinterpret_cast<switch_rtp_hdr_t
*>(vid_frame.packet);

           // Adjusts timestamp to standard 90KHz clock.
           ts = ntohl(hdr->ts) * 90000 / vc.videoTrack().track.clock;
           hdr->ts = htonl(ts);
           if (pt)
               hdr->pt = pt;

           if (switch_channel_test_flag(channel, CF_VIDEO))
           {
               switch_byte_t *data = (switch_byte_t *) vid_frame.packet;

               vid_frame.data = data + 12;
               vid_frame.datalen = vid_frame.packetlen - 12;
               switch_core_session_write_video_frame(session,
&vid_frame, SWITCH_IO_FLAG_NONE, 0);
           }
           if (ts && last && last != ts) {
               switch_cond_next();
           }
           last = ts;
       }

       if(aOk)
       {
           if (write_frame.datalen > (int) write_frame.buflen)
               write_frame.datalen = write_frame.buflen;

           switch_core_session_write_frame(session, &write_frame,
SWITCH_IO_FLAG_NONE, 0);

       }
       switch_core_timer_next(&timer);
   }

I think it should be like this (pseudocode)

nextAudio = nextVideo = ts = 0

while(switch_channel_ready())
{
   read_video_frame();
   read_audio_frame();

   nextVideo = ntohl(hdr->ts) * 90000 / video.clock;

   waitTime = min(nextAudio, nextVideo) - ts;
   if(waitTime > 0)
    {
       wait(waitTime);
       ts += waitTime;
   }

   if(ts >= nextAudio)
   {
      send_audio_frame();
      nextAudio += 20 * 90/8; // use the same clock source
   }
   if(ts >= nextVideo)
   {
       send_video_frame();
   }
}

   Any suggestion is much appreciated.

   Regards,

   Paulo R. Panhoto

_______________________________________________
FreeSWITCH-dev mailing list
FreeSWITCH-dev@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
http://www.freeswitch.org



--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
googletalk:conf+888@conference.freeswitch.org
pstn:+19193869900