<div dir="ltr"><div>(Anthony, response to you is below Michael's response to consolidate messages). Apologies in advance for the long reply - but love the discussion.</div><div><br></div>Hey Michael,<div><br></div><div>It's not so much the number of clients that is problematic but rather the difficulty of capacity planning when operations become increasingly more expensive. It presses towards building a smaller number of very large FS instances, over a smaller number of medium sized ones that you could get away with for audio + network traffic. The system I'm building is multi-tenant, and there's application code written around distributing conferences across FS hosts. But say an FS host has 80% CPU, it's hard to know in advance if a conference I'm about to place there will turn into a 10 person video chat or a 2 person audio chat. With an MCU approach, there's a very real possibility that if 10 people join I exhaust CPU and run into issues. Whereas with an SFU, there's much less risk of any one conference pushing over that edge, because that 20% can still handle a considerable number of clients. It just makes the capacity planning and distribution process easier when a typical video conference consumes a very small number of CPU resources.</div><div><br></div><div>As far as the second comment - I agree. Certain devices may be limited in their processing power. But mobile devices are <i>also</i> limited in screen real state, so there may not be an MCU layout that works well for a combination of both mobile and desktop devices. With an SFU approach, you could choose to receive all streams on the mobile device but only decode 2 or 3 of them (at the application's discretion), and lay them out in a way that is optimal for that device. It wastes network bandwidth, but preserves experience. You could do similar with a canvas per caller, but then you're even further increasing the cost on the server.</div><div><br></div><div>In any case I'm not intending to argue that SFU is a better approach generally (or argue at all for that matter), just trying to better understand the required additions to FS to make it happen. With the addition of mod_verto and full native WebRTC support, having an SFU capability would allow FS to cater to the FaceTime/Hangouts/Skype types of applications with highly customized (and hopefully optimized) experiences.</div><div><br></div><div>Anthony,</div><div><br></div><div>I agree that what I'm referring to would be little more than a replication of what Jitsi does, but the "value add" would be having it baked directly into mod_conference with no need to interface with another product. I'm finding it hard to rationalize how I would get Jitsi Video Bridge working with Freeswitch in a way that doesn't cause more problems than it solves - and FS caters to far more use cases independently than Jitsi can. To clarify, I'm not asking anyone here to contribute the code - rather just looking for any pointers on what it might take. If SFU were to be added as an exclusive <i>OR</i> with the current canvas muxing, is this a PR that the FS would accept? Or is it a requirement that it be able to operate in a hybrid MCU + SFU mode? I'm still not quite sure how useful that would be, though - except in the cases of better supporting mobile (for which there are SFU workarounds), or older devices that don't do multi-stream (seems unlikely that these use cases would want SFU in the first place though).</div><div><br></div><div>Sorry again if I'm annoying anyone! I'm really just looking for the dialog/hoping to learn more.</div><div><br></div><div>Best,</div><div>Colin</div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Aug 4, 2016 at 2:59 PM Michael Jerris <<a href="mailto:mike@jerris.com">mike@jerris.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">how many clients are you talking about? You may want to look closely, particularly at mobile clients, as they wont gracefully handle multiple streams, they don't have the processor to handle that much video decode, and often don't have the bandwidth either.<span></span><br><br>On Thursday, August 4, 2016, Colin Morelli <<a href="mailto:colin.morelli@gmail.com" target="_blank">colin.morelli@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Anthony,<div><br></div><div>Thanks for the quick reply.</div><div><br></div><div>Could you elaborate more on the need for FS to support a hybrid mux/multi stream model (for a single conference?) I'd love to understand more about this so I can wrap my head around the efforts involved. I wonder if there's a phase 1 that could be reached where you'd be forced to choose between SFU/MCU modes? Understanding that it might sacrifice compatibility with clients that can't accept multiple video streams, or the particular codecs that are used on a stream, I would think this allows FS to <i>at least</i> play the role that Jitsi would play today - and allow for future enhancements.</div><div><br></div><div>I think MCU makes a lot of sense for many deployments - and is very sensible to support. The two concerns I personally have with MCU are a) the obvious CPU constraints imposed on the hosts, but more concerning is 2) the UX constraints it imposes on the resulting interfaces. #2 becomes increasingly relevant as WebRTC/mobile/desktop clients become more and more prevalent.</div><div><br></div><div>Best,</div><div>Colin</div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Aug 4, 2016 at 2:18 PM Anthony Minessale <<a>anthony.minessale@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">It will be a massive project to support SFU. It would be a hybrid of the 2 not an either/or. Its all been considered but there is less to gain from SFU so its not a priority.<div><div>Unlike Jitsi, FS would have to actually implement the media side of the equation and dual support muxing and multi streaming where Jitsi enjoys only needing to provide signaling between browsers.<br></div><div><div><div>FreeSWITCH is better suited to an MCU and we made a specific decision to focus on that but there might be a slow march towards these other things as time progresses and interest does or doesn't happen.</div></div></div></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"></div></div><div class="gmail_extra"><div class="gmail_quote">On Thu, Aug 4, 2016 at 1:04 PM, Colin Morelli <span dir="ltr"><<a>colin.morelli@gmail.com</a>></span> wrote:<br></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">For what it's worth I also understand FS would need to be able to push multiple video streams to each connected client, but given that FS can already handle audio + video streams, I assumed multi stream handling exists (though it may be limited to just 1 audio + 1 video - I haven't dug into this at all).</div><div><div><br><div class="gmail_quote"><div dir="ltr">On Thu, Aug 4, 2016 at 1:52 PM Colin Morelli <<a>colin.morelli@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hey there,<div><br></div><div>Has anyone had success pairing up Freeswitch + Jitsi? Specifically I'm interested in the SFU capabilities of Jitsi to replace just the video conferencing portions of Freeswitch. I'm doing a lot of other things in FS that will continue to be important on their own (DTMF, voicemail, SIP + PSTN integration, etc), but the SFU support would be an awesome addition. I saw that there's a SIP <-> Jitsi bridge which looks like it might be promising - but I feel like the complexity would fairly quickly become overwhelming.</div><div><br></div><div>Alternatively, I wonder how much work it would be to add video-mode="sfu" to mod_conference in FS. It's something that (in the future, not quite yet), I might be interested in taking a stab at. Has anyone come up with a list of things that would need to happen for SFU support? I might be a bit naive here, but it seems FS has most of the building blocks in place already, and mostly it would come down a) having a video stream selection process, and b) generating appropriate SDPs when participants are added/removed. I'm sure I must be missing something, though.</div><div><br></div><div>Thoughts?</div><div><br></div><div>Best,</div><div>Colin</div></div></blockquote></div>
</div></div><br></blockquote></div></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a>consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" rel="noreferrer" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" rel="noreferrer" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" rel="noreferrer" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a>FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" rel="noreferrer" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" rel="noreferrer" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬<div><br><div>☞ <a href="http://freeswitch.org/" target="_blank">http://freeswitch.org/</a> ☞ <a href="http://cluecon.com/" target="_blank">http://cluecon.com/</a> ☞ <a href="http://twitter.com/FreeSWITCH" target="_blank">http://twitter.com/FreeSWITCH</a></div><div><div>☞ <a href="http://irc.freenode.net" target="_blank">irc.freenode.net</a> #freeswitch ☞ <u><a href="http://freeswitch.org/g+" target="_blank">http://freeswitch.org/g+</a></u><br><br></div><div>ClueCon Weekly Development Call <br></div><div>☎ <a>sip:888@conference.freeswitch.org</a> ☎ +19193869900 </div><div><br></div></div></div><div><a href="https://www.youtube.com/watch?v=9XXgW34t40s" style="color:rgb(17,85,204);font-size:12.8000001907349px" target="_blank">https://www.youtube.com/watch?v=9XXgW34t40s</a></div><div><a href="https://www.youtube.com/watch?v=NLaDpGQuZDA" target="_blank">https://www.youtube.com/watch?v=NLaDpGQuZDA</a><br></div></div></div></div></div></div></div>
</div>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a>consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" rel="noreferrer" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" rel="noreferrer" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" rel="noreferrer" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a>FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" rel="noreferrer" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" rel="noreferrer" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a></blockquote></div>
</blockquote>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" rel="noreferrer" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" rel="noreferrer" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" rel="noreferrer" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" rel="noreferrer" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" rel="noreferrer" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a></blockquote></div>