The team run an environment called the RTST: the Real-Time System Test.
The RTST is a live production XenApp/XenDesktop environment… with a twist. Within it, we regularly deploy pre-release, in-development builds of multiple Citrix products. It’s a real environment, complete with all the usual infrastructure any regular IT department would have: Active Directory, DNS, SQL, VPN, and so on. It has real-life users, too, doing real work things on it, all day, every day. This isn’t simply a clinical environment spun up and torn down to run some tests against — it’s been around for many years, just like many of our customers’ environments, so it’s as close to real as we can get.
So, why do we do this?
The benefits of this approach are numerous and multi-faceted, but the primary aim is to catch bugs before release. It’s also about improving customer experience, both from a sysadmin perspective and an end-user perspective.
Additional benefits of our work include building Engineering and Product feedback loops, building SysAdmin empathy, improving end-user experiences, and interoperability testing (that is, testing how well our own products work with each other).
Engineering and Product liaison
Engineering teams at Citrix like to get their products in the RTST because we’re able to provide a close-to-real-life test platform. We work closely with Product Managers and Engineering to implement pre-release builds of Citrix products into the RTST environment. If we — or our colleagues using our environment — encounter issues, we work with Engineering to diagnose and fix the bugs. And because the bug source is internal, it makes it a lot easier to reproduce the issue and collect logs. Through our relationships and with direct lines to the right people, we’re able to find bugs and fix them quickly.
SysAdmin empathy
Managing the underlying IT infrastructure and deploying Citrix XenApp/XenDesktop environments every week helps build a team of SysAdmins who know Citrix products inside and out. If something doesn’t work right, or is a bad experience for SysAdmins, we can use our relationships with Engineering and Product teams to make a case for improvements.
End user advocacy
Having internal Citrix staff use the published apps and desktops from the RTST environment for their daily work is invaluable. Often our colleagues notice issues while consuming our XenDesktop published resources that we wouldn’t notice ourselves in the implementation stage. We take feedback and reports from our colleagues, determine if they’re environmental or product issues, and then pass them on to Engineering, creating bug reports where necessary.
Interoperability
An important by-product of what we do is interoperability testing. Because we integrate multiple Citrix products and their features into a live IT environment, we end up testing the interoperability of many of our products. Our product and feature mix is fairly diverse: XenServer, XenApp/XenDesktop, Citrix Cloud, NetScaler Gateway, GSLB, PVS, MCS, smartcards, StoreFront, Linux VDA, and Session Recording, among others. If something isn’t working, we’ll know about it quite quickly and can use our experience and relationships to get the right Engineering teams talking to each other.
It’s all about finding and fixing bugs
Ultimately, the RTST environment exists as part of a collective effort from Citrix to find bugs that might otherwise be missed. In the end, Engineering wins, and our customers win, too.
So, what’s next for the team?
Right now the SysAdmins in the RTST team are focused on helping with Citrix’ cloud journey. We recently joined the Cloud Operations team for the Citrix Cloud XenDesktop service, and we’re looking forward to using our experience with our on-premises products to help test and improve our Cloud products.
In terms of our impact: we helped out, most recently, with user acceptance testing for the XenApp Essentials product, contributed to the new Citrix OS Optimization tool, and helped verify and create a reproducible environment for a recent issue with Seamless Apps not launching on 7.14 with Server 2008 R2.
In the long term, we want to increase the number of Citrix products we implement into the RTST environment at a pre-release stage. We’re currently investigating Citrix App Layering, Workspace Environment Manager, and NetScaler MAS
We also want to talk more publicly about what we do, why and how, so expect to hear more from us over time. In the meantime though, just know that we’ve got your back, and that we’re working with everyone here to help make Citrix products the best they can be!
