February 17, 2022
Comments Off on Configuring FreeSWITCH as a WebRTC MCU Media Server

A Media Control Unit or MCU is the most established and time tested approach to setting up conferences via bridges. Conference bridges add centralized call and media features like mixing and quality control among other advanced controls such as a DID number, announcement, admin control, secure PIN-based access. Due to centralized media management, they are also ideal for connecting mixed streams with media pipelines for recording, broadcasting or plugging into machine learning models for transcription for NLP or sentiment analysis. This post will cover using FreeSWITCH as a WebRTC Multipart conferencing server using a video mixer and conference bridge.

Mixer for bridging calls from heterogeneous endpoints

This is heavily reliant on a media server with a mixer for audio and/or video streams. Meeting rooms often involve a bridge with dozens of participants that multiplex their incoming audio and video streams into one output and input stream per participant.

Sources of energy consumption

Sources of energy consumption in a MCU based conference:

  • Transmission: 2 * transmission_energy (kWh)  * n
  • Device (media): 2 * device_energy (kWh) * n
  • Signaling Server: signaling_energy (kWh) * n   
  • Media Server: mediaserver_energy (kWh) * n

Relatively higher energy consumption in Mesh based (peer to peer) conference:

  • Transmission: transmission_energy (kWh)  * n * (n-1)
  • Device: device_energy (kWh) * n * (n-1)
  • Signaling Server: signaling_energy (kWh) * n 
  • Media Server: 0

In this fashion of a centralized setup, the CPU consumption is highest at the centre at the media server. This is due to MCU’s real-time media processing that may include decoding, mixing and re-encoding every stream and mixing as one. It should be noted that CPU consumption could be higher where a centralized server does extensive transcoding between peers supporting different codecs.

On the other hand, the CPU utilization is lowest at receivers who only need to decode one single stream incoming stream from the media server. On the side, extra latency caused by media server decoding necessitates the use of a jitter buffer.

Figure: relative client side energy consumption of different RTP topologies 

Installation and Setup of FreeSWITCH for Video MCU

FreeSWITCH™ is an open source carrier-grade telephony platform implemented as a back-to-back user agent.

FreeSWITCH is a SIP standard specific communication platform that forms the core of many cloud telephony and communication services. The pluggable modules make FreeSWITCH suited to almost any role in a SIP platform (SBC, gateway, SIP application server, media server, etc.). Let’s set up FreeSWITCH as a WebRTC MCU using a video mixer and conference bridge.

More configuration files related to this setup can be located at https://github.com/agilityfeat/Energyefficient_WebRTC_Asterisk_systems/tree/master/Freeswitch_MCU_Webrtc

Configuration files for freeswitch. Below is a sippet of freeswitch.xml

The vars.xml  is the most important file ( according to me ) for starting freeswitch as it contains the Preprocessor Variables. These values should be consistent across the modules 

The internal.xml manages the configuration for internal clients such as SIP endpoints registered with the SIP server.

<profile name="internal">
 <settings>
   <param name="auth-calls" value="true"/>
   <param name="apply-nat-acl" value="nat.auto"/>
 
   <param name="debug" value="0"/>
   <param name="sip-trace" value="yes"/>
 
   <param name="dialplan" value="XML"/>
   <param name="context" value="default"/>
   <param name="codec-prefs" value="$${global_codec_prefs}"/>
 
   <param name="rtp-ip" value="$${local_ip_v4}"/>
   <param name="sip-ip" value="$${local_ip_v4}"/>
   <param name="ext-rtp-ip" value="auto-nat"/>
   <param name="ext-sip-ip" value="auto-nat"/>
   <param name="sip-port" value="$${internal_sip_port}"/>
 </settings>
</profile>

Conference Layout

The Conference.xml is critical to setup the bridges and allow for mixing. The Conference_layout.conf enables us to add various layouts for users. Once a conference begins this is the layout for incoming video that will be assembled and sent out to all participants.

<configuration name="conference_layouts.conf" description="Audio Conference">
   <layout-settings>                                                                                                                                            
       <layouts>                                                                                                                                                  
           <layout name="presenter-overlap-small-top-right">
               <image x="0" y="0" scale="360" floor-only="true"/>
               <image x="300" y="0" scale="60" overlap="true" reservation_id="presenter"/>
           </layout>

           <layout name="1up_top_left+5">
               <image x="0" y="0" scale="240" floor="true" audio-position="-.5:0:0"/>
               <image x="240" y="0" scale="120" audio-position=".25:0:0"/>
               <image x="240" y="120" scale="120" audio-position=".25:0:-.25"/>
               <image x="0" y="240" scale="120" audio-position="-.5:0:-1"/>
               <image x="120" y="240" scale="120" audio-position="0:0:-1"/>
               <image x="240" y="240" scale="120" audio-position=".25:0:-1"/>
           </layout>

           <layout name="1x1">
               <image x="0" y="0" scale="360" floor="true"/>
           </layout>

           <layout name="2x2">
               <image x="0" y="0" scale="180"/>
               <image x="180" y="0" scale="180"/>
               <image x="0" y="180" scale="180"/>
               <image x="180" y="180" scale="180"/>
           </layout>
       </layouts>
       <groups>
           <group name="grid">
               <layout>1x1</layout>
               <layout>2x2</layout>
           </group>
       </groups>
   </layout-settings>
<configuration>

Users  

Each extension contains a parameter “context” that matches a diaplan in conf/dialplan/.

When FreeSWITCH gets a register packet it looks for the user in the directory based on the domain in the packet depending on how sofia profile is configured. For this demo, I have set auth to false since I will only be using dial in extensions for users to join the bridge.

My user profile 

<users>
   <user id="altanai">
   <params>
     <param name="password" value="$${default_password}"/>
     <param name="vm-password" value="1000"/>
   </params>
   <variables>
     <variable name="toll_allow" value="domestic,international,local"/>
     <variable name="accountcode" value="987"/>
     <variable name="user_context" value="video-mcu-stereo"/>
     <variable name="effective_caller_id_name" value="altanai"/>
     <variable name="outbound_caller_id_name" value="altanai_outbound"/>
   </variables>
 </user>
</users>

Dialplan 

 conf/dialplan/ contain XML files for dialplan

For this demo, I am using the video-mcu-stereo conf profile (shown below) which allows the communication service provider to add sounds for user activity as join, leave, mute, locked out, wrong pin and even other advanced features such as comfort noise, video canvas layout among others. There are many other samples for the video MCU configuration and one can always design their custom configuration.

<profile name="video-mcu-stereo">
     <param name="domain" value="$${domain}"/>
     <param name="rate" value="48000"/>
     <param name="channels" value="2"/>
     <param name="interval" value="20"/>
     <param name="energy-level" value="200"/>
     <!-- <param name="tts-engine" value="flite"/> -->
     <!-- <param name="tts-voice" value="kal16"/> -->
     <param name="muted-sound" value="conference/conf-muted.wav"/>
     <param name="unmuted-sound" value="conference/conf-unmuted.wav"/>
     <param name="alone-sound" value="conference/conf-alone.wav"/>
     <param name="moh-sound" value="$${hold_music}"/>
     <param name="enter-sound" value="tone_stream://%(200,0,500,600,700)"/>
     <param name="exit-sound" value="tone_stream://%(500,0,300,200,100,50,25)"/>
     <param name="kicked-sound" value="conference/conf-kicked.wav"/>
     <param name="locked-sound" value="conference/conf-locked.wav"/>
     <param name="is-locked-sound" value="conference/conf-is-locked.wav"/>
     <param name="is-unlocked-sound" value="conference/conf-is-unlocked.wav"/>
     <param name="pin-sound" value="conference/conf-pin.wav"/>
     <param name="bad-pin-sound" value="conference/conf-bad-pin.wav"/>
     <param name="caller-id-name" value="$${outbound_caller_name}"/>
     <param name="caller-id-number" value="$${outbound_caller_id}"/>
     <param name="comfort-noise" value="false"/>
     <param name="conference-flags" value="livearray-json-status|json-events|video-floor-only|rfc-4579|livearray-sync|minimize-video-encoding|manage-inbound-video-bitrate|video-required-for-canvas|video-mute-exit-canvas|mute-detect"/>
     <param name="video-auto-floor-msec" value="1000"/>
     <param name="video-mode" value="mux"/>
     <param name="video-layout-name" value="3x3"/>
     <param name="video-layout-name" value="group:grid"/>
     <param name="video-canvas-size" value="1920x1080"/>
     <param name="video-canvas-bgcolor" value="#333333"/>
     <param name="video-layout-bgcolor" value="#000000"/>
     <param name="video-codec-bandwidth" value="3mb"/>
     <param name="video-fps" value="30"/>
     <!-- <param name="video-codec-config-profile-name" value="conference"/> -->
   </profile>

Security certs and WSS Binding

Add certs like /usr/local/freeswitch/certs/wss.pem shown below

Add wss-bind-url in internal.xml

   <!-- for sip over websocket support -->
   <param name="ws-binding"  value=":5066"/>
   <!-- for sip over secure websocket support -->
   <!-- You need wss.pem in $${certs_dir} for wss or one will be created for you -->
   <param name="wss-binding" value=":7443"/>

Check that profile is loaded 

 sofia status profile internal
screenshot for sofia status profile internal

Give permission to wss page. For this example my ip and port made this url which I gave advanced permissions to.

Screenshot for advanced permission for self signed security keys

WebRTC Clients 

Using jssip for 2 clients.

screenshot for registering client on JSSIP for Freeswitch MCU server

Monitor Successful registration on sofia by checking registrations by profile with the command

sofia status  profile <ip>  reg
Profile lookup on sofia for registrations
Two party WebRTC conference via FreeSWITCH Video MCU Server (remote video is mixed with local)

For Multiparty conference trials one can use Cyber Mega Phone 2K Ultimate Dynamic Edition browser side client application by Digium.

Multiparty WebRTC conference via Freeswitch Video MCU Server

Up Next 

The last blog post in this series will compare the SFU and MCU against Mesh using Chrome’s energy usage in ‘Analyzing compute and energy consumption of the most popular video conferencing topologies’ that indicates the carbon footprint of a conference call.

About the Author: Altanai Bisht

Here at WebRTC.ventures, we pride ourselves on offering our clients the opportunity to work with extremely bright developers, like Altanai. Don’t trust your custom video solution to just anyone. Trust the experts!

Recent Blog Posts