As most of you already know, Skype for Business Online works with two main protocols: UDP and TCP.
Microsoft and most other VOIP solution providers recommend using Skype for Business with UDP rather than TCP. The purpose of this article is to understand why and in which condition it matters.
We’ve configured multiple Robot Users, using Skype in different simulated network conditions.
So, when are UDP or TCP used? Why does Skype sometimes use TCP? Does Microsoft recommend using UDP?
UDP is fastest at delivering packets on the network for many reasons. The first is that it doesn’t check if the packet is fully received. Another is that this protocol is not sensitive to packet reorder. This means that UDP can deal with it without considering it as packet loss. The problem is that you have to open many more ports on your network for it to work. And that is why, sometimes UDP is not allowed for communication outside of the company network.
In these cases, the protocol used is TCP. This protocol is much more secure, because it needs to open significantly fewer ports. On top of that, TCP makes sure that all packets are received at the destination, and if not, it will send them again. However, all of that processing takes time, so TCP is supposedly slower.
Let’s see if this is true!
First, we conducted our tests in a normal environment, and then we played with it to stress the network in order to see how the Skype end-user experience would change, depending on the protocol used. We experimented with low available bandwidth, a congested network, packet loss and reorder. We will explore the results in the second part of this article.
First experiment: Basic environment – Stable network and bandwidth
This is an environment that has been designed to be stable from a network perspective.
Here, we are looking at data we gathered for about a week, coming from 2 Robot Users: one doing TCP (green), the other one is using UDP (grey).
Every 5 minutes they perform a 15-second call to Office 365 and gather the quality and network metrics during the call. Let’s examine at the critical statistics here.
This explains why each of these metrics is important and what their impact is on the Skype for Business end-user experience.
Let’s focus on the MOS. As you can see here, the Robot User on UDP and the one on TCP perform almost identically. Of course, there is some variance due to normal network life and traffic.
Average MOS, healed ratio, and most of the statistics that show voice degradation are almost the same.
To complete these tests in a normal / optimized environment, we did the same tests in a completely isolated lab and what we saw is that TCP and UDP were again, statistically-speaking, identical.
We had expected UDP to be much better. But on a high speed, low latency network, the two are identical. That is why we decided to tweak the network to see if the end-user experience would be more affected with UDP or with TCP protocol.
Second experiment: UDP vs TCP on 15% packet loss network
Here we can see the results of the voice quality in a network with 15% of packet loss injected.
We can see right away that the MOS dropped sometimes, even to zero, because calls were failing.
This was expected because packet loss is included in the computation of the MOS score.
The healed ratio changed a lot, that was also expected.
Just like any voice product, Skype has to do some sample repair because of network instability. The Skype team anticipated some jitter and packet loss, and therefore have algorithms that can deal with these problems. These statistics are critical to understanding voice quality, and only GSX Robot Users are able to show you these statistics.
Comparing calls using UDP and TCP, here we can see that there is a tremendous amount of healing on UDP, which does not have a built-in mechanism for making sure that the packets arrive at the destination. On TCP, it is different because a retransmission takes place to automatically resend the information.
As you can see, the Jitter is impacted too.
This is directly related to the nature of the protocol, as UDP doesn’t really mind the
order of the packets. It will send all the packets regardless, whereas TCP will.
Also, you need to consider the codec. The codec 104 is called SILK, which has built-in
repair capabilities to try to smooth things over. Skype automatically chooses SILK when
the network is in trouble because if its capabilities to repair samples.
The value of that healed ratio goes up to 10 and we know that once it passes 7, the users become unhappy.
Obviously, this test was conducted in an extreme environment, as 15% packet loss is drastic. However, it is useful to understand how Skype is repairing calls and how UDP or TCP are impacted by the quality of the network.
After having described the effect of packet loss on Voice quality for UDP and TCP, let’s see what latency brings to the table.
Third experiment: UDP vs TCP with high latency network (200ms)
The Microsoft recommendation about latency between the Skype clients and Microsoft Edge is 100ms. In order to show you the impact, we have put that figure to 200ms.
First thing you see here is the MOS, which stayed pretty stable, above 4.0.
Again, you can see the jitter, especially for the TCP Robot User and the healed ratio that is way higher for the UDP protocol.
A robotic sound or any low-quality voice is certainly disturbing, but it may also have more impact. For example, when you are having a discussion with someone that uses your language as a second or third language. In this case, it can be complex to understand an accent and the conversation, even if the MOS is above 4.
A last thing to notice here is that the latency starts to impact packet reorder. The UDP protocol doesn’t care about the packet order but the TCP protocol does. This is another reason why UDP is generally recommended.
Now, let’s focus on these packet reordering statistics.
Fourth Example: UDP vs TCP with 10% packet reorder ratio
It is interesting to see the MOS more stable on the UDP (purple) protocol than the TCP one (red) that shows many more drops.
Drops usually cause user complaints.
Once more, we see how UDP doesn’t care much about the reordering ratio and how it doesn’t impact the call too much. As explained earlier, it is one of the main reasons why UDP is recommended by Microsoft.
The MOS is in high fluctuation, which is why it is important to look at the variance. If we compute the average, it might look fine, but in reality, users would likely complain about half of these calls.
Another interesting thing is the healed ratio here. It stays somewhat below the 2% mark that we know as a good limit. It is impressing to see how the Skype client is resilient and able to cover the packet reorder.
Now that we have seen the performance impact of these critical statistics on a UDP Skype call or TCP Skype call, let’s now see how Skype for Business Online deals with a congested network.
Fifth example: UDP vs TCP with a Congested Network
At your client companies, the network is used for a variety of reasons (large transfer, numbers of emails, multiple calls and presentations at the same time, YouTube, etc. ) so that the available bandwidth is often not the maximum possible.
We tested the impact low bandwidth has on voice quality, with both protocols.
Here, you can see the UDP in red and in green, the TCP. The differences are truly critical. The MOS score for the UDP stays in the 4 range. For TCP, it stays between 0 and 3.
The down count is also impressive, meaning a lot of calls would be impossible on the TCP side.
Here is the reality of the situation: you will have a hundred or more people on the same internet in a building, surfing YouTube or streaming Spotify while working with 30 or 40 Skype for Business related calls at the same time. So low available bandwidth will affect your Skype client.
That is where you really see the UDP protocol shining.
All the benefits of TCP from a delivery standpoint won’t help in a congested environment. They don’t bring additional value for “smoothing” the sound in a nice fashion into someone else’s headset.
This is the main reason why Microsoft and other VOIP want you on UDP.
Here, TCP is losing data and suffering from the jitter.
Even some MOS scores fall all the way down to zero because the calls dropped.
We can clearly line the Microsoft recommendation with facts here.
In order to understand how UDP and TCP affect the end-user experience, you need to perform tests in a diverse range of conditions, including extremely bad environments. You also need a large sample of tests to cover any behavior and pattern in your organization. Finally, you need metrics from all your locations.
At last, is UDP versus TCP a no-brainer?
We’ve seen that in a perfect environment they are identical, but as soon as you use Skype in an unstable environment, UDP clearly has an advantage.
In most companies, if UDP is not possible because of firewall configuration, it will fall automatically back on the TCP protocol.
It’s important to be aware in order to understand end-user complaints.