Case Study Office 365 Exchange

Business Case
Office 365 Migration: User Satisfaction is the Key Factor to Measure Success

   Large financial company in Canada 

  First batch included 90,000 mailboxes

  • Moving from Exchange 2016 to Office 365 Exchange Online.
  • The financial institution will deploy Teams collaboration and Voice later. SharePoint will stay on-premises for now but OneDrive will be enabled.
  • Final architecture will be hybrid with Azure AD Connect, ADFS and 15 Exchange servers left on-premises.
  • The email security appliance is IronPort with multiple boxes on-premises.
  • Branch offices are spread across multiple continents and thousands of locations.
  • Route to Office 365 follows Microsoft connectivity best practices.
  • Multiple mail flow paths existed, creating a complex messaging environment.

Executive summary

In order to ensure the success of their migration to Office 365, the IT management of one of the largest financial institution in North America decided that the service quality delivered to business lines was the key indicator to measure. The objective was to deliver the same end-user experience with Office 365 that was provided by Exchange on-premises. In order to measure the results and achieve this, they implemented GSX digital experience monitoring for Office 365, enabling them to continuously ensure the best service performance for every site during and after the migration.

Primary need:

The IT management decided that one of the primary factors to determine the success of the migration was the end-user satisfaction. The issue is that most users experiencing issues do not open IT tickets. When they do, there is generally no data to back the claims, which leads to long and costly investigations.

The purposes of the project were:

  • Understand the difference in the end-user experience between Exchange on-premises and Exchange Online.
  • Anticipate any performance issues.
  • Prove that performance is stabilized and predictable at the end of the migration, maintaining an acceptable latency that does not impact end users.

To this end, the end-user experience of Exchange on-premises at several critical locations was measured and compared with the new Office 365 service delivery during the entire migration process.

How GSX was put in place

The company contacted GSX during spring of 2019.

GSX worked with the Office 365 project manager to install the solution prior to the first batch of moves.

In May, four GSX Robot Users were installed at key locations to test the end-user experience: one in Canada, one in Germany, a third one in Brazil and a fourth one in the main US office.

GSX Robot Users were all installed under an hour. They were connected to the central Robot User management & Power BI to report on historical data.

For each Robot Users, key Exchange scenarios were set up (open mailbox, create email, create meeting, search, free/busy lookup) as well as internal (between offices) and external mail routing to test the Exchange 2016 on-premises environment.

On top of these end-user actions, the Robot Users also measured and recorded key metrics on the network availability and performance between users and the Exchange 2016 data center.

The purpose was to detect potential bottlenecks that could later degrade the Office 365 quality of service.

The mail routing was set up to test the response time of the infrastructure, including IronPort, to deliver email from one office to another and from offices to external domains.

This control point was key during the migration because it is the first service that users use when they have Office 365 Exchange Online.

How GSX was used during the migration

This initial phase established the existing end-user experience baseline (using Exchange servers) that was the basis of comparison for the Office 365 services.

After setting up the initial end-user experience monitoring configuration, GSX experts worked with the IT operations staff to determine what statistics to include in monthly reports. These would be sent to the IT managers to measure the success of the migration.

After two weeks of on-premises end-user experience monitoring, the monitoring was extended to the Office 365 infrastructure that was being installed.

On-premises measurement continued, but Office 365 end-user experience monitoring was added for the Robot Users. That is a particularity of GSX, as our Robot Users are able, at the same time, to perform end-user and network latency tests both on-premises and to the Cloud.

The monitoring of Cisco IronPort was also kept. Microsoft Azure AD Connect and ADFS were added to the mix in order to detect any potential issue with end-user authentication during the process.

This second phase lasted during the whole migration, which took about six months. During this time, the operations team was alerted in case of any end-user experience issues even before the users could notice them.

At the end of each month, all the data was analyzed by GSX experts. They presented the results to the customer management team in a monthly meeting.

The outcome for the Company

The initial tests of the on-premises environments (Exchange 2016, IronPort, Gateways) already detected significant differences in end-user experience among the offices tested.

It was no surprise that the Brazilian office experienced repeated latency issues during the week. The overall performance wasn’t that different between the German and North American offices. What was striking was the number of latency peaks during the week, especially during the morning hours. We also discovered that the route to the datacenter was inconsistent because the number of hops was clearly unstable.

The first action of the central IT group was to investigate their local network to eliminate these potential bottlenecks that would certainly affect the Office 365 service.

The root cause was found thanks to the continuous DNS resolution time test that the Robot Users perform. Clearly, the DNS was not able to support the load when users first connected in the morning. Consequently, changes in DNS resolution targeted servers that were not optimum for the route to the cloud. Once this issue was solved, the migration continued until it produced interesting results.

As expected the overall end-user experience of the Office 365 Exchange Online services was good but sometime lower than the on-premises experience. This was not a surprise. Security is the main concern of the Cloud services, not performance. But even if the performance was lower than that experienced before, it was still in the acceptable range that had been defined.

For mail routing, the performance was exactly as it was before. Several other network bottlenecks (inconsistent routes, local IT that had to be updated, internet latencies) were shown in reports and fixed during the migration period.

At the end of the migration project from Exchange 2016 to Office 365, the evidence provided by GSX Solutions and its Robot Users proved that the project had been a success. The number of tickets was limited and handled proactively thanks to GSX Solutions. The company decided to continue using the solution and has since expanded the number of Robot Users to more than 100.

The end-user complaints were rare and anticipated. Every month, the Office 365 service quality delivered to business lines is justified by GSX reports given directly to management.

Want more? 
Find out what makes our Office 365 monitoring tool unique, or request a demo.

Let's get started.