A common question I receive as Security Product Manager for XenApp and XenDesktop is “what information is left behind client side after using a Citrix XenApp or XenDesktop hosted application?” This happens most often for PCI DSS and HIPAA audits and the question is normally one of audit “scope.”
The answer is that no screen or keyboard data is ever written to persistent endpoint storage
This post describes in detail the small bits of information that are written to the endpoint and describes the technique for reproducing the study.
PCI DSS and HIPAA with XenApp and XenDesktop
Running applications on hosted XenApp and XenDesktop machines, where only those data center resident hosted machines have vision to protected networks and databases and where end users have no network connection to databases, greatly reduces “scope”. The security assessment can focus on the potential areas of “high loss” and give lower focus to endpoint computers.
The NetScaler Gateway Module ICA Proxy provides separation of networks and only ICA/HDX traffic makes it across the Gateway. The end user machine has no IP network connectivity to the host XenApp or XenDesktop computer and more importantly, the end user machine has no IP connectivity to the database server holding credit card or patient data. The screen and keyboard data is wrapped inside TLS on its path to and from the gateway so there is little risk of exposure on the main corporate network. This is huge win for security.
The auditor though has to quantify and study the potential risk of endpoint data loss and this includes a study of potential data lost via “leaks” from to the persistent client side storage of the endpoint computer. What data “leaks”? Could an attacker study the endpoint computer later and resurrect PCI or HIPAA data?
The rest of this post is dedicated to the gory details.
What “leaks” onto the endpoint?
There are several client side components in Citrix Receiver. It starts programs that get lists of published applications and make icons available to users for starting hosted applications and desktops. Receiver.exe as the native receiver, or Receiver for Web (Receiver run in web browser). Eventually all of these “launchers” kick off a program called WFCRUN32.exe which itself starts the HDX Engine or artist formally known as the ICA client (WFICA32.exe). It is this final piece that we care about for today’s discussion. This is the piece that displays server screen content as if programs were running locally and redirects client side keyboard and mouse to the host XenApp or XenDesktop systems.
A complication on answering a question of “what leaks” is that the answer to anything really specific like that has to be measured against a specific point in time or product version. Programmers change things! They don’t do it that often and most of the time everything is as it was, but occasionally the list of things changes and this makes security people apprehensive to make large statements. For example, take a look at CEIP data in the detail below. It does what it does on purpose, but for a security person, storing anonymous usage data on the endpoint before sending home is a leak. It’s not a bad leak, but from a security view, it’s a leak and this didn’t exist a few years ago.
Today I chart “what leaks” with the May 3rd 2017 Win32 HDX Engine (wfica32.exe) version 14.8.0.369. That’s probably a released version and happens to be the version running on my primary desktop computer. In the pages below, I provide the technique I used to see what “leaks”, so hopefully in the future, people other than me can make future measurements…
Once upon a time, I wrote code for a living
Security PM, yes. Today I go back to my heritage of application virtualization and about 20 years of writing device drivers. Dusting off some old skills, we can dissect the client side executable and watch what it does at runtime. Note that none of this requires access to source or even disassembly – we just run the program and watch what it does.
Step 1 – Process Monitor
Process Monitor from SysInternals is incredible software. It monitors and logs all file system and registry activity on the machine. Download here. From admin privilege, run procmon.exe tell procmon to monitor the activity of wfica32.exe (the HDX engine). At this point that process isn’t running yet, but when you launch a hosted application or desktop, that process will get executed and procmon will capture everything it does related to disk and registry.
Here’s the filter configuration. Get here with Ctrl-L. Hit “reset” to reset the filter state to default, then add one rule to say track things done by processes named wfica32.exe.
Step 2 – Launch a hosted application or desktop
In my case, launched Notepad++ as a published application. It launched, ran and looked happy and then I closed it, which terminated the session and ultimately killed wica32.exe. While the hosted session ran, process monitor captured a ton of information! A more efficient version of this would define many filters to tell procmon to monitor only things that are writes. I didn’t do that, wanted to see “everything” that the application does to the file system and registry and then manually filter that down to only things that “write”.
Step 3 – Tell Process Monitor we’re done capturing
Ctrl-E to stop capturing, or click File, Capture to uncheck the capturing mode.
Inside of process monitor, we want more information than the default display
- Process Monitor – Event, Include, Detail
- This includes Desired Access, “read” or “write” information in addition to the path
Look at the captured data
Here’s a screen shot of the start.
Step 4 – Filter this down to a set we can comprehend
Tell Process Monitor to save the data to a CSV file, “all events”!
Launch Microsoft Excel
File Open, Browse, open file CSV above. Load as a comma separated file. Get this
When studying leaks, the column for Time is uninteresting, so we delete it. For trivia, in my case it took about a minute to launch, observe the pretty application running and terminate it. Not needed today.
The process name and PID (process ID) are always same. So, we delete those columns, as well.
And finally, the first ROW has titles, which we don’t need, so delete and we get something a bit more manageable. It still has 65,579 rows describing every file system and registry access, so it isn’t really good yet, but we’re on the right track.
We care about the writes! I expect excel could be used to hide all the reads, but I’m better with GREP so we need to move the worksheet to text file. Save this version of the spreadsheet, as .xlsx. Copy that version to a backup…
Save the worksheet to .csv file and load in your favorite text editor. Do some change, replace to get the commas converted into about 30 spaces and align the columns in all 60K rows to make the data pretty and organized. This making it pretty part probably isn’t necessary, but for me makes the data easier to view. Even with remembered key patterns, this took about 20 minutes and was very tedious.
We only care about “writes”
When looking for potential information that leaks from server to client, we care only about information written to the client. Process Monitor captures all of the file system and registry operations. We need to get rid of all the items that are reads and the vast majority of items are reads.
The left most column lists all of the “operations” that are captured. If it’s a file or registry write, we care! If it’s a read, we don’t care.
Still in excel, copy the contents of the “Operations” column to a text file, e.g. unsorted.txt and then sort and get rid of all the duplicates.
- sort < unsorted.txt > sorted.txt
- uniq < sorted.txt > uniqueoperations.txt
Tools above: sort comes with Windows, uniq is from GNUTools.
Now have a list of all the operations conducted, but it’s a minimal list containing ONE entry for each operation class. For entertainment, you can also count the number of occurrences of each operation. I missed this step, but the vast majority are reads.
We now have a list of all of the operation. The question is, does it “write” and where it doesn’t write, I added a “Remove” label to the list.
| CloseFile | Remove |
| CreateFile | Study |
| CreateFileMapping | Remove |
| Load Image | Remove |
| LockFile | Remove |
| Process Exit | Remove |
| QueryAttributeInformationVolume | Remove |
| QueryAttributeTagFile | Remove |
| QueryBasicInformationFile | Remove |
| QueryDirectory | Remove |
| QueryEaInformationFile | Remove |
| QueryFullSizeInformationVolume | Remove |
| QueryInformationVolume | Remove |
| QueryNameInformationFile | Remove |
| QueryOpen | Remove |
| QueryRemoteProtocolInformation | Remove |
| QuerySecurityFile | Remove |
| QuerySizeInformationVolume | Remove |
| QueryStandardInformationFile | Remove |
| QueryStreamInformationFile | Remove |
| ReadFile | Remove |
| RegCloseKey | Remove |
| RegCreateKey | Study |
| RegDeleteKey | Study |
| RegEnumKey | Remove |
| RegEnumValue | Remove |
| RegOpenKey | Remove |
| RegQueryKey | Remove |
| RegQueryKeySecurity | Remove |
| RegQueryValue | Remove |
| RegSetInfoKey | Study |
| RegSetValue | Study |
| SetBasicInformationFile | Study |
| SetDispositionInformationFile | Study – File erase |
| SetEndOfFileInformationFile | Study – Set file size |
| TCP Connect | Remove |
| TCP Disconnect | Remove |
| TCP Receive | Remove |
| TCP Send | Remove |
| TCP TCPCopy | Remove |
| Thread Create | Remove |
| Thread Exit | Remove |
| UnlockFileSingle | Remove |
| WriteFile | Study |
Export the worksheet (all captured events) to a text file (e.g. edit.txt) and then do a repeated operation of the below to filter out all of the captured events that are reads.
- findstr /v “CloseFile” edit.txt > temp.txt
- copy /y temp.txt edit.txt
- Repeat above for each of the “Remove” items listed above
Notice that the things recorded in the capture file (edit.txt) are STILL in sequential order on how they occurred. This helps us understand what the program was doing when the given write occurred.
OKAY, we’re down to 5,336 items.
The noise from the Windows installer
5,212 of the remaining entries are RegSetInfoKey. A complete morass of crap from Windows installer. This API isn’t documented very well, but appears to set the last access time on registry items, keys? There are also a few others that can be filtered such as UnlockFileSingle.
What is written?
We’re now down to a very small set of operations that are “writes”. I review each below.
SetDispositionInformationFile %LOCALAPPDATA%\Citrix\ICA Client\zlcache\icass3.rng
“SetDispositionInformationFile” is file system device driver speak for “erase file”.
A “write” to perform an erase is “good” from a security view. But, for this file, we’re talking “font data” and that requires more analysis. The zlcache directory was once upon a time a disk stored font cache for the ICA client, the directory is still present! Font cache: If a font is used, its likely it will be used again and the program keeps a cache so the host doesn’t have to transfer font data if the client already has it.
In times before RAM, this cache was kept on disk and this also had a small performance advantage that font data would survive across executions.
The cache disk file isn’t used anymore, everything is kept in RAM and the small cross execution performance advantage of using disk is discarded in preference to the security advantage of never writing font data to disk. It has been this way for all of my memory at Citrix, so this goes back a long time.
The operations against the file though still exist and that raises questions, so I wrote a CPR! Get rid of the directory and stop writing!
Analysis by the programming team says that removing the directory will break old versions of the client that could potentially exist on the same machine. The program client solves by leaving the directory in place and … actively erasing the disk cache contents at runtime, the disk cache that it doesn’t use. That’s actually a security WIN as it will clean up from older versions of the program, but when capturing the execution in procmon, it will appear as an item to analyze in the captured trace and anything that says “font” gets attention.
To be clear, the disk data is erased and RAM is used for font cache.
A number of writes occur to HKEY_CURRENT_USER\Software\Citrix\ICA Client\CEIP\Data
CEIP is Customer Experience Improvement Program, link. This is anonymous “phone home” data to help Citrix improve the XenApp and XenDesktop products. On my machine CEIP is enabled and data is collected and stored in the registry, I expect as a caching space for historical data before transmission to home.
CEIP function can be disabled via installation option. CitrixReceiver.exe /EnableCEIP=false. The function can be more aggressively disabled via CitrixReceiver.exe /EnableCEIP=none which prevents the registry writes.
The data that is written to the CEIP data is anonymous when sent in, but when looking at a single computer before it is sent, it says things about the user’s use of the ICA client on the machine. This can be turned off, but leaving it enabled helps the world and I’m doing my part to help the world, CEIP is enabled on my computers. Wow, I feel good!
Dig in and you’ll see information on usage patterns. Number of application launches per hour.
From a PCI DSS and HIPAA perspective, the CEIP data does not contain credit card numbers or patient medical data.
%APPDATA%\ICAClient\UISTATE.INI
User Interface State dot INI. Looking inside, the file is text file that holds GUI Window X, Y screen positions for various applications have been run on this computer by the present user. This file is written at runtime and file date/time is updated when the file is written. From a security view, that’s a leak of information saying the time of day that the most recent hosted application was terminated. It also provides a history of names of applications that have been executed by this user.
Example content:
[Notepad++]
WindowXPos=952
WindowYPos=500
DeskTopWidth=3840
DeskTopHeight=1080
What’s the upside? When users set the size and position of hosted applications on the screen, the next time that hosted application is run, the application window is placed into the same position on the screen where it was when the user most recently closed it.
Users like that! It’s a good feature! But, it’s a leak. Does it matter? For PCI and HIPAA, no credit card or customer PIA is leaked.
Experiment #1. Erase file. Launch application again, works. As predicted, the file gets recreated with only the window positions of the just now launched program. If you were a paranoid sort, you could write a program to automatically delete this file every time it gets created. I’m not aware of a “disable” and for PCI DSS and HIPAA, it contains no credit card data and no patient data. It does though indicate the names of hosted applications that have been run by this user.
Client side printers mapped into host. RegSetValue (registry writes) defining printers
- HKCU\ Software\Citrix\PrinterProperties\Microsoft XPS Document Writer\Client/
- HKCU\Software\Citrix\PrinterProperties\Microsoft XPS Document Writer\…
- HKCU\Software\Citrix\PrinterProperties\Microsoft XPS Document Writer\…
If mapping of client printers to host was disabled, would this go away? Not sure.
Internet Explorer ZoneMap configuration is set!
Spoiler alert, its nothing! But it sure sounds pretty bad. Let’s review.
Each of the below are written twice in immediate succession during hosted application launch
- HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\ProxyBypass (1)
- HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\IntranetName (1)
- HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\UNCAsIntranet (1)
- HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\AutoDetect (0)
Experiment: Change every one of the 4 items to the opposite. Relaunch, do they change? Answer, yes, all 4 written back to the 1, 1, 1, 0 state. The change occurs during application start, not terminate. Also note that they are written even if they are already in the right state. #badprogrammer
For PCI DSS or HIPAA, it isn’t a leak of credit card or patient data, but there could be some security questions of why are you messing with IE zones? Here’s some reading on Internet Explorer zones. (Remember, no issue here, I’m showing how I got there ).
The solution is in process explorer
Start fresh with process explorer, this time focusing ONLY on writes to these registry items. Ctrl-e to turn off capturing. Ctrl-x to flush the capture buffer. Ctrl-l to configure filter, click default, and then capture all processes that perform RegSetValue with path containing “ZoneMap”. Close the filter configuration and then Ctrl-e to start capture.
For verification, change one of the values in regedit, and yes, process explorer observes. Launch hosted application, and yes, process explorer captures just like in the writing above.
Let it sit there a while. Move the start menu from the bottom to the right side of the screen to make more vertical room on screen and process explorer catches! What?
- SearchProtocolHost.exe making a bunch of writes to this space! Hum…
- Bamn! Where’d that come from lync.exe is hitting it as well. We’re on to a pattern!
- Internet Explorer! Bamn! More writes.
Answer: It isn’t us. Best bet is wininet.dll doing some actions to make sure the settings in the registry match those that are configured via the system settings. Learn more by pointing at any of the captured events in process explorer, right mouse button, properties and the common theme is \Windows\SysWOW64\urlmon.dll. Precisely why remains a mystery, but after a multi-hour red herring, there is nothing to see here, move along.
And to cheat ever the small bit, I reached out to programmers with the source and requested a few greps. Answer, no hits.
HKCU\Software\Citrix\XenDesktop\DesktopRestartAllowed (0)
Looks innocuous. Inquiry to wise people tells me this is used in kiosk mode to prevent the kiosk from being rebooted. For PCI DSS and HIPAA, that’s a “don’t care”.
And that’s it. There aren’t any more writes!
Summary
We describe that in server based computing, “nothing” is left behind on the endpoint when the client side display code terminates and for data loss questions, this is accurate.
No KEYBOARD data is retained; no SCREEN data is retained. This means that for purposes of a PCI DSS or HIPAA audit, no credit card data or patient data is retained and this meets the requirements of most scenarios and helps auditors define the audit “scope”. In most cases it permits increased focus on the security of the hosted applications, network and databases and this is a security win.
Application usage data though DOES leak! It is possible to conceive of cases where even information about the endpoint being used, information about the names of the hosted applications accessed and when they were accessed being a security concern. Here, that information is “leaked”, even if leaked only to the point of the MSI installer updating time stamps on its verification that the installed client side application is still installed. In such cases, thin clients with non-persistent local storage could be appropriate. For the vast majority of PCI and HIPAA cases though, vanilla Citrix Receiver running on an endpoint computer should meet the needs.
Enjoy!
Joe Nord
Citrix XenApp/XenDesktop, Security Product Manager
Citrix TechBytes – Created by Citrix Experts, made for Citrix Technologists! Learn from passionate Citrix Experts and gain technical insights into the latest Citrix Technologies.
Click here for more TechBytes and subscribe.
Want specific TechBytes? Let us know! tech-content-feedback@citrix.com
