Why Multi-CDN has moved out of the lab an into the mainstream
“Multi-CDN is not the goal. It’s the tool helping people realize the goal.” — Rob Malnati, Citrix
In a recent webcast from Dan Rayburn and Rob Malnati, I heard a clear and articulate explication of (a) why even smaller Broadcasters are adopting Multi-CDN strategies (in spite of some additional burden) and (b) why certain strategies that are adopted fail, while others succeed. I thought I would share these insights here as well as provide some further food for thought about why certain strategies succeed. Having been a multi-CDN practitioner myself I have some opinions (from the field so to speak) about what often leads to a failed experiment.
Let’s start what is a Multi-CDN solution? What qualifies as multi-CDN?
Multi-CDN is any configuration of infrastructure that uses more than 1 Content Delivery Network (CDN) as part of the delivery. This can be an Origin-Shield configuration, an Active-Passive configuration for failover or a true Active-Active configuration. We will delve deeper into this later in this blog.
Why are broadcasters moving to Multi-CDN solutions? What are the biggest drivers? Quite simply it is cost, performanceand availability. Which of those is most important is the key to what configuration you choose, and what strategies you use to manage and maintain your Multi-CDN solution. Let’s look at these important drivers.
Costs
While Dan Rayburn makes the valid point in the webcast that ‘CDNs have come down in price dramatically over the last 15 years’, a CDN is still an expensive part of an OTT provider’s delivery infrastructure. For Netflix, it was so expensive that they build their own CDN. Same for YouTube. Commercial CDNs for large content providers are in the top 10 items of your costs on the delivery side. Maybe the top 3. In my case it was number 1.
So, not letting your CDN become a runaway cost is important. Vendor diversity is the oldest tool to ensure the best pricing and this has driven many providers to adopt Multi-CDN. To negotiate better price, you need leverage. Multi-CDN requires more burden — but it can often be less expensive. Multi-CDN provides leverage.
Performance
Performance is everything on the internet. Millions are spent every year trimming a few milliseconds off of delivery time, so that video does not buffer, and video startup time is optimized. There are practical reasons for this. Consumers do not tolerate poor performance. Companies go out of business because of bad performance.
CDNs are networks. They have specific peering relationships with ISPs. They are stronger in certain geographies than others. For instance, some CDNs have points of presence in South America, and others do not. The ones that do not often rely on a well peered Datacenter in Miami, Florida for their POP’s — so that they can peer directly with the South American ISPs from that location who use the underwater cable to that runs through the Gulf of Mexico. This can cause a lot of problems when trying to deliver content into Brazil or Chile.
Because of these types of differences in the network configurations, layouts, Point-Of-Presence (POP) positioning and peering, no two CDNs are totally alike with regard to performance. Some might be really strong in Chile on one ISP while the other is stronger in Chile on a different ISP. This can even be different depending on what time of day since ISP’s get congested in certain peak times. And neither of those CDNs might be very good in China or Russia.
No two CDNs have the same performance from every locations and every network. This truth drives many content providers to adopt multi-CDN.
Availability
This is the big kahuna. If you are down, you are out of business. You are not streaming. Your customers hate you. And ALL CDNs eventually go down. I bet in the last month you have lost the lights in your house at least once. A thunderstorm rolled through and your lights flickered and went out for a minute. Or a car hit a pole down the road and blew a transformer. Well the electrical grid is 100X more resilient than Internet infrastructure. This is the most obvious and important reason to engage in a Multi-CDN solution. While costs and performance are great goals — availability of your service is paramount, and any good COO would gladly pay for redundancy of this sort.
So now you understand the main drivers of why content providers utilize multiple CDNs. But how many CDNs should you have? Geography? Availably and performance? In the webcast, Dan Rayburn shared with us that many of his clients didn’t see the difference between the various CDNs. In other words — from a broad performance perspective, how can you select the correct CDNs — and how many do you need?
At Dan’s prompting, Rob Malnati shared some nuggets about how content providers can evaluate a multi-CDN strategy. This is necessarily broad, but there are some great truths here. They basically break into 2 rules: know your users and your user’s relationship to the content and use data driven automation. Let’s take these one at at time.
Know the User
Measuring users is one of the toughest and most important things on the Internet. Understanding real user experiences matters. As we saw above – knowing where your customers are is important since CDNs vary wildly with regard to performance, costs and availability in various regions around the world. One of the simplest ways to measure performance is by taking Real User Measurements (RUM) from your site. This provides the most valid data you can use to understand how your users are experiencing your video start time, buffering and other key metrics.
It is also important to understand what they are doing on your site. Is it just video? Or are there other elements to their experience? Is your content VOD only? Do you have Live events?
Using a tool like Cedexis Radar tool can help you understand where your users are, what times of day they come to your site for content, what ISPs they are typically on and many other variables that can then be used in your CDN calculation.
For instance, do you have customers in China? If so there are some specific CDN’s that you must utilize to provide a buffer free experience. Are your users also in North America on the East Coast? Do they tend to be on Comcast or ATT? Once you know this you can evaluate which CDNs are winning in which regions, networks and times of day that you care about. This information is invaluable.
Finding Ways to Automate Changeover
Once you have selected the correct set of CDNs, how can you implement traffic management? The simple answer is that you must use Data to determine the best CDN, and then have the traffic flow to that CDN. Real time traffic management is the key to top performance and availability. Outages can happen in an instant and nobody has a pager anymore. Your system must know what to do in a failure. By using Real User Measurements in real time to shift traffic to the best performing CDN you can create excellent user experience while avoiding outages. This seems like a no-brainer, but you would be surprised at how many people still don’t get this story.
Many broadcasters will tell you they have implemented a multi-CDN strategy for performance. Recently I was speaking to one and the example was “I am sending my ROKU traffic to one CDN and web traffic to another, so I am doing Multi-CDN”. While I would not argue with that, I would say that this configuration is not optimal. This is not providing any real failover to note, and certainly not providing better performance.
Some Concluding Thoughts
Using Multi-CDN for OTT broadcasters has become the de facto standard although not all implementations are the same. How you select the CDNs you do business with is important. Use data to identify the best providers for your scenario and users. Then find the 3 CDNs that fit your requirements using Cedexis Radar. After that, using real time, data driven Internet Traffic Management (ITM) is the key. Pawn your pager and join the next generation of always on, always performing sites.
