A year ago, in XenApp and XenDesktop 7.12, we pioneered the first version of HTML5 Video Redirection in the industry, a solution aimed at extending the multimedia redirection capabilities of HDX MediaStream to HTML5 audio and video.
Recently, Adobe announced that Flash is scheduled for EOL by Q4 2020, and Microsoft announced that they would also phase out Flash, culminating in its removal entirely by the same date. As a result, many Citrix Customers have already started the transition from that format to HTML5.
The feature ‘HTML5 Video Redirection’ in the 7.12 release required the manual injection of a JavaScript (included on the XenApp/XenDesktop installation media) into the HTML code of the webpage, which allowed us to detect the HTML5 <video> element and its coordinates inside the webpage. An additional requirement was the type of media we supported (MP4) and the delivery mechanism (Progressive Downloads). This first phase of optimizing HTML5 video delivery was aimed at customer-managed content such as training videos and corporate communications videos. Adaptive Bitrate Streaming was not supported and that meant YouTube and similar websites could not be redirected. These external HTML5 video-based sites were handled by server-side rendering and Adaptive Display with Selective H.264 encoding, delivering a great user experience, but at the cost of server resources.
In this post, I will introduce our next generation technology for redirecting HTML5 video, designed to handle Internet web sites like YouTube. To start, I will lay out all the jargon we use in HDX, so we can all be on the same page, explain how video is delivered over the internet, the different formats and codecs, and finally close with a section on where we are heading.
1. Video in the internet today
There are three basic ways a user can watch a video on a browser: through progressive downloads, streaming, or adaptive bitrate streaming. These distribution methods are intrinsically different, and a basic understanding is necessary before we dive into HDX details.
1.1. Progressive Downloads
Progressive Downloads refer to a single file that you are playing back as you download it from an origin web server (HTTP). The file is encoded at one quality only (e.g. 720p) and there is no content protection (as generally the file is stored on the client’s hard drive). The premise is that the bitrate of the encoded file has to be lower than the available bandwidth between the origin server and the player (otherwise the file will stop playing).
A good example of a progressive download is our HTML5 video redirection test page. You can use the developer tools in your browser to inspect the video element, and find the source in the HTML5 <video> tag. You will notice a few things when playing that video.
First, it’s an MP4 file that starts playing even before it is fully downloaded. This ‘fast start’ is achieved by placing the metadata at the beginning of the media file (MP4 is the container format, the video is encoded using H.264 and the audio using AAC).
Second, even if you pause the video, the buffering is limited to a few extra seconds. This allows the multimedia content provider to save bandwidth and server resources.
Third, you can seek (or time shift) the video to any point (achieved by sending HTTP messages to the Apache server hosting the video with a range-byte offset header).
These 3 characteristics augment progressive downloads into something called pseudo-streaming.
1.2. Streaming
This delivery mechanism requires specialized servers like the Adobe Media Server and Real Time Protocols (RTP, RTMP, RTSP, etc.). The multimedia is sent in chunks at paced intervals, which are discarded by the player after rendering. ‘Just enough’ data is sent to the player, so this saves bandwidth for the content provider.
Live events and VoD thrived thanks to this technology, but it became evident that it was not efficient, especially for large-scale events (if we need to open an RTP socket to every user, we need thousands of servers). It also required proprietary software. The industry needed something better.
1.3. Adaptive Bitrate Streaming
Adaptive Bitrate Streaming, on the other hand, uses a different approach: the multimedia content is segmented into discrete files (~2 to 6 sec long) that can be served via HTTP over any standard web server (and Apache/IIS/nodeJS are built to reply to traffic all day long … also, HTTP is firewall-friendly unlike UDP).
As CDN networks are designed for caching and optimizing HTTP traffic, adaptive streaming is appealing, since it can tap into the existing infrastructure of the internet.
And this changed the game.
There are two main surviving standards today for adaptive streaming: HLS (HTTP live streaming, proprietary from Apple) and DASH (Dynamic Adaptive Streaming over HTTP, a company independent, open and international standard ratified by ISO/IEC). YouTube and Netflix are examples of websites that have adopted DASH, replacing FLV (Flash video) or smooth streaming (Silverlight) respectively … and at that point the industry started following.
All the different media segments are organized under a manifest (.mpd for DASH, or .m3u8 for HLS), which is a playlist containing metadata information about the multimedia content URI location.
The real power of adaptive streaming is that for live or on-demand streams, the origin web server can deliver the multimedia files at either high, medium, or low quality depending on the network’s condition, and switch between them on a per-segment basis without disrupting the user experience.
It’s important to point out that the adaptation to the bitrate is client-side via JavaScript, which has multiple advantages as the browser has better visibility of the capabilities and throughput of the client. JavaScript is leveraged not only for playback control and buffer health, but also for injecting the media chunks into the <video> element used in HTML5 (this is known as Media Source Extensions). JavaScript gives developers the ability to control almost every aspect of the video, from designing the player’s UI right through to firing actions based events.
The bitrate adaptation logic is not specified by any standard, and is subject to R&D done by the company playing the video on their website / video player.
YouTube and Netflix have very different logics; YouTube videos are generally short (~3mins), so they want to play them as quickly as possible, otherwise the user will divert their attention. Netflix, on the other hand, understands that their users are about to engage with their player for the long haul (~60mins), so they will take more time in building up a healthy buffer.
The so-called “spinning wheel of death” is fatal for any user-generated content website that relies on ad revenue, such as YouTube or Vimeo.
The Netflix business model is different, as their revenue comes from the viewer who is willing to wait for a high definition experience and won’t accept anything less.
But I digress (sorry … can’t help it; in HDX we love multimedia). Let’s switch gears into XenApp/XenDesktop mode.
2. Bringing virtualization and HDX into the mix
The meticulous among you might have spotted 2 intrinsic flaws when trying to redirect adaptive bitrate streams:
- The video player relies heavily on JavaScript for bitrate adaptation and media fetching, and splitting the media from webpage context (as we used to do in HDX MediaStream redirection) poses many integrity challenges
- If the video player on the Virtual Desktop’s Browser has a great internet connection back to, let’s say, YouTube, then content will be downloaded at High Quality. Even if it was possible to rip the media out of the website and redirect it to the client (a.k.a server fetch client render), the network between Receiver ↔ VDA is probably not as good as VDA ↔ YouTube. This means that the ICA multimedia virtual channel will not be able to sustain the necessary throughput for Receiver to play the content without glitches!
Last but not least, there was the problem of the websites themselves. Let’s take YouTube as an example. While you were watching that cat video that went viral, YouTube engineers are running dozens of experimental features on the background; eventually rolling them out into production using a canary release method.
Any modification to their website/player could break something in our redirection mechanism (Flash flashbacks anyone?). To provide a robust solution, a different approach was required.
2.1. Browser Content Redirection
With HTML5 in mind then, we devised a new redirection mechanics, based on the redirection of the entire webpage.
The viewport is the rectangular area in your browser where content is displayed. It does not include things like the address bar, favorites toolbar, status bar, etc. All that belongs to the user interface.
The admin would first configure a studio policy where an access control list is specified with the URLs whitelisted for redirection (www.youtube.com/*, etc.).
For the browser on the VDA to detect that the URL the user is navigating to matches the whitelist, we rely on a browser extension (included with the XenApp/XenDesktop installation media) that performs the comparison.
If a match is found, a virtual channel will instruct Receiver that a redirection needs to happen and relay the URL. At this stage, Citrix Receiver will instantiate a local rendering engine (already present natively in the client OS, e.g. the Trident layout engine in Windows) and display the website.
The final step is for Receiver to blend back the website into the virtual desktop’s browser viewport seamlessly (a.k.a reverse seamless).
How Receiver fetches the content is an important question, and we offer you three options:
- Server Side Rendering (SSR): No redirection happening, either because you did not whitelist the site, or the redirection failed and we fallback to rendering the webpage on the VDA and use Thinwire with Adaptive Display to remote the graphics, as usual. Fallback behavior can be controlled via Policies. High CPU, RAM and Bandwidth consumption on the VDA are to be expected.
- Server Fetch – Client Render (SFCR): Receiver is contacting and fetching content from the web server through the VDA using a virtual channel. This is especially useful when the client does not have internet access (Thin Clients, for example). Low CPU, GPU and RAM consumption are expected on the VDA, but bandwidth will be consumed on the ICA virtual channel.
- Client Side Fetching (CSF): Receiver is contacting the web server directly, therefore it requires internet access. This offloads all the network usage from your Citrix site, and of course all the CPU, GPU and RAM. All the heavy lifting is done by Receiver.
2.2. System Requirements:
On both XenApp and XenDesktop 7.16 VDA, the first browser we will support is Internet Explorer 11 (with Chrome and Edge to follow), and a browser extension will be installed. The first Citrix Receiver version to support this new technology will be Receiver for Windows 4.10, with Linux and other versions to follow.
2.3. Citrix Ready Ecosystem:
As always, you’ll never walk alone in your multimedia journey. Our Citrix Ready program works very closely with the leading Enterprise Video Platforms vendors out there: IBM, Vbrick, Qumu, Kollective, Mediaplatform, Ramp and many others are adapting their products so they can interact smoothly with your Citrix deployment and provide you with efficient media delivery at the best qualities. We suggest you visit their websites and get in touch with them to learn more.
In addition, we are working hand in hand with all our Thin Client vendors so they can package the new Receiver for Linux and additional components necessary for Browser Content Redirection. Unicon, HP, IGEL, Dell, Stratodesk and others are already developing with us to deliver this new feature.
3. Conclusion:
So, there you have it; a full review of our redirection technologies. This feature is currently available in 7.16.
In Part 2 of this series, we will go deeper into Browser Content Redirection and explain the mechanics behind it, so stay tuned. These are exciting times for HDX Multimedia!
Citrix TechBytes – created by Citrix Experts, made for Citrix Technologists! Learn from passionate Citrix experts and gain technical insights into the latest Citrix technologies.
Click here for more TechBytes and subscribe!
Want specific TechBytes? Let us know tech-content-feedback@citrix.com
