We would like to share some of our expertise using the following example.

The Company:

• Large oil & gas company.
• 50 locations, most remotely located in Africa, and the Middle East.
• Migrated from Exchange on-premises to Exchange Online.
• Uses SharePoint on-premises.
• Plans to deploy Skype for Business Online across all locations.
• Sensitive to challenges that Skype for Business Online might introduce to network management and stress to current infrastructure.
• Route to Office 365 via ISP.
• Connection of the remote locations to headquarters via VPN.
• Tenant in EMEA.
• No Azure ExpressRoute.

Primary Need:

Deploying Skype for Business Online can be risky, especially if you don’t prepare well. The primary challenge is to understand the network, its current usage and capabilities in order to predict the impact of the change. With dozens of remote locations scattered around Africa and the Middle East, the IT management team knew that an extensive preventive assessment was critical to:

• Identify locations that might experience problems.
• Pinpoint what can be done to improve the level of service.
• Calculate the ROI of identified infrastructure updates / improvements.
• Select which sites will benefit from Skype for Business Online.
• Determine which sites will have to stick to satellite communications or landlines.

The network usage of each site varies over the day, week, month depending of the level of activities. IT management needed to continuously monitor performance statistics as recommended by Microsoft. However, Microsoft’s free assessment tools are not designed to conduct an evaluation over long periods of time nor in so many locations. In urgent need of a solution, they contacted GSX Solutions early 2017 for assistance.

GSX Set Up:

The challenge was to continuously test, collect and analyze performance data from 27 locations remotely across Africa and the Middle East over a period of two months. We sat down with Microsoft’s Skype for Business Online Engineering team, and decided on a step by step plan.

Since GSX Robot Users can be seamlessly rolled out remotely from a central interface, the setup was very quick. Within a week, all 27 locations were equipped with Robot Users, either installed on a workstation or a virtual machine (VM). All Robot Users were configured to monitor key metrics recommended by Microsoft.

The complete set of tests is performed every five minutes, stored in a SQL database, and displayed on a PowerBI dashboard specially designed for the Skype for Business Online readiness assessment. Data included:

• Latency time one way from the Robot User to the end-point and round trip. This is key to understand if communication will be smooth, without a ‘satellite phone-like feeling’.
• Packet Loss and packet reorder to see if interruptions or distorted voice quality doesn’t happen.
• Jitter (inter-arrival jitter): to make sure that there isn’t delay, distortion, speed up or slow down during calls.
• Network MOS: to forecast the overall score of the audio quality.
• DNS resolution time to make sure that the DNS configuration won’t impact the overall performance.
• Traceroute: to measure the consistency of the number of hops between locations and the Microsoft datacenter.
• A real Skype Voice call that collect quality information during the duration of the call
• Bandwidth estimation during the time of the call

As Microsoft recommends, the tests were performed both directly from the client Edge to the Microsoft datacenter and from the Enterprise Edge in Paris to the Microsoft datacenter.

Once everything was set up, we created a PowerBI dashboard that displayed the results. This allowed us to clearly identify usage pattern of the network across the 27 locations.

We built the dashboard to provide a clear understanding of statistics and to identify usage patterns during the next two months.

We advised analyzing the network load to identify the peak times for each day, week and month.

To determine whether a site was suitable for Skype For Business Online, it was decided that the network tests should at least succeed during the daily and weekly peak times.

A daily test failure during a monthly peak was investigated to clarify whether or not the required infrastructure improvement would provide a sufficient ROI.

Our Robot Users tests showed the daily peak time was experienced between 7 AM and 8 AM local time. The weekly high was on Thursdays (Friday is part of the weekend in most Middle Eastern countries) and the monthly workload peaked in the first week of the month.

Outcome:

Out of 27 locations tested, 5 failed to meet Microsoft performance recommendations during the peak time of the month; 11 failed during the peak time of the week, and 5 failed the daily tests. Only 6 locations performed fully within recommended guidelines during the entire test period.

What does ‘failed’ mean, exactly?

Following Microsoft recommendations, here are the success margins:

We won’t detail the entire dataset or metrics of each site here, but we will explain how we analyzed our results with IT management.

The five sites failing daily tests had MOS quality below 3.

With these results in mind, IT management was ready to look into a set of potential network tweaks to see if they could improver these sites without spending too much money.

First, we recommended they worked on their path to connect to the Microsoft datacenter. Management had to be convinced that the extra layer of network infrastructure that was put in place was actually unnecessary. So, we asked a Microsoft expert to show how Skype communication is encrypted out-of-the-box, and how using a proxy and a VPN only affects performance without adding any extra security.

All of the 16 sites that failed all the time were set up to break out to Internet directly and connect to a tenant in the Middle East (they were before configured to go through a VPN and a Proxy to access a Tenant in Europe). On top of that, the Network QoS was implemented anywhere possible in the network. Our Robot Users monitored the effects of these changes, providing a new set of performance metrics. It turns out that just adjusting the path to connect to the Microsoft datacenter considerably improved the situation for 11 of the 16 sites. New global results showed 15 sites now passing monthly, weekly and daily tests; 6 failing the monthlies, 3 failing weeklies and only 2 failing dailies.

A second round of network performance and configuration assessment was initiated to solve the remaining site issues. With GSX Robot User we were able to determine that two of the locations had firewall issues. This was spotted by the abnormal value of the round-trip time. Three other sites, including the two that were failing daily tests, had bandwidth issues. Their percentage of packet loss was clearly linked to network constrictions problems. The remaining site checked out from a network standpoint.

So, the final results after network improvements showed 2 sites failing daily tests, 1 failing weeklies, 6 failing monthlies and 15 sites succeeding. IT management went through an ROI analysis for the weekly and daily tests failing sites, considering bandwidth improvement and Azure ExpressRoute.

It was concluded that the two sites failing dailies would not benefit from Skype for Business Online because the costs of network improvements were too high compared to the number of users present on these sites. The site failing weeklies will get additional bandwidth from a new ISP and will get Skype for Business Online services. All other sites were ready for deployment.

Conclusion

As you can see, the Skype for Business Online readiness assessment was necessary.

Simple changes clearly identified by GSX Robot Users improved the audio experience and overall user satisfaction. By systematically monitoring and analyzing crucial metrics, the company avoided a lot of trouble before, during and after the deployment, while controlling expenses. Smart decisions were taken based on facts. This assessment led to broader adoption while also providing better user experience and ultimately an overall reduction of the TCO of Office 365 services.