Since the acquisition of Norskale back in September 2016, Citrix has been making further developments to the technology. Norskale was aptly renamed “Workspace Environment Manager”, or “WEM” for short, and a plethora of new, exciting features have been added.
However, although significant technological developments have been made, it is clear, from speaking with the customers I meet, that there isn’t enough information available about WEM. As a result, it remains a bit of a mystery. Some customers have never even heard of WEM, aren’t really sure if they are entitled to use it, what WEM actually does, how it does it, and more importantly, how it can help them. This leads to customers that are entitled to WEM, missing out on the many benefits of such a great product.
So, I decided, with the WEM PM Team, that an educational WEM Blog series, covering different aspects of WEM, was in order.
Today’s post is concentrating on the System Optimisation capabilities of WEM, in particular, CPU, RAM and I/O Management.
Thanks to Rod Haanappel, Senior Readiness Specialist, and Miguel Contreras, Senior Solutions Architect, both of whom have helped me extensively so far with this blog series (and hopefully will continue to do so). Big thanks Rod and Miguel!
So firstly, for those who don’t know what WEM is…
WEM is a XenApp and XenDesktop Enterprise / Platinum entitlement that uses intelligent resource and profile management technologies to deliver the best possible performance, desktop logon, and application response times for XenApp and XenDesktop deployments. It is a software-only, driver-free solution which focuses on both resource management and profile management.
Resource Management: WEM provides the best experience for users by monitoring and analysing both user and application behaviour in real time, then intelligently adjusting RAM, CPU, and I/O within the users workspace environment.
Profile Management: WEM delivers the best possible logon performance by replacing commonly used Windows Group Policy Objects, logon scripts, and preferences with a WEM agent. The WEM agent is deployed on each virtual machine or server. The WEM agent is multi-threaded and applies changes to user environments only when required, ensuring users always have access to their desktop as soon as possible.
NEW! Application Security: WEM Application Security, allows WEM Administrators to create granular “Allow or Deny” rules for individuals, or groups of users, directly from within the WEM Console. This allows WEM Administrators to easily, and effectively control what applications, scripts, installers, packaged applications and even DLLs users can run.
Hopefully, over the coming weeks and months, I can explain at a high level, how WEM does all of this.
WEM System Optimisation.
After installing the WEM Agent on a VDA, that VDA can make use of the WEM System Optimization features shown in Figure 1 below. These features are a group of user-centric, machine-based settings that are applied to all user sessions, and designed to reduce overall VDA resource usage. If used correctly, these features could potentially increase user density on XenApp deployments, saving money on infrastructure costs.
Let’s take a closer look at each one!
- CPU Management
CPU Management is the most complex section of WEM System Optimisation, and also the area of a lot of discussion. So, it makes sense to start here.
- CPU Spikes Protection:
What does it do?
When enabled, CPU Spikes Protection manages a process when it exceeds a set CPU Usage Limit (%). It does not reduce the impact that a process has on overall CPU usage, but instead, it reduces the potential negative impact that CPU intensive processes have on user experience.
How does it do it?
As you can see in Figure 2 above, once we enable CPU Spikes protection, we can set 3 values.
- CPU Usage Limit (%) – Hard Limit set. Once this limit is reached and breaches the Limit Sample Time (s) below, CPU Spikes Protection will trigger.
- Limit Sample Time (s) – The set period of time a violating process spikes for (exceeds the CPU Usage Limit (%))
- Idle Priority Time (s) – The amount of time a violating process will be demoted for. After this time elapses, the process will return to its previous priority.
In Figure 2 above, we see that if a process exceeds the CPU Usage Limit (%) value, for a set period of time or more (defined in Figure 2 by the Limit Sample Time (s) value), the process will be demoted to “Low Priority” for a set period of time (Idle Priority Time (s) value).
Important: It is extremely important to bear in mind that the CPU usage Limit (%) value is a global value across all available logical CPU Cores in the system, and not the utilisation percentage of any single Logical CPU Core.
Not all processes consume CPU in the same way.
There are processes that are able to use multiple CPU Cores (multi-threaded processes) and there are other processes that are only able to use a single CPU Core, regardless of how many are available in the system (single-threaded process). When it comes to multi-threaded processes, also note that these processes may not utilise equal amounts of each of the CPUs they use, and that some might be able to leverage all the Logical CPU Cores available, while others might be limited to only using a specific maximum number of CPUs, even if there are more CPUs available!
Why does this matter?
This is extremely important because, as mentioned earlier, the CPU usage Limit (%) value of CPU Spikes Protection is a global value across all available Logical CPU Cores in the system, and not the utilisation percentage of any single Logical CPU Core.
Confusing right?
Well, let’s take a look at a few scenarios to better understand what this means. In all scenarios, the system in question has four Logical CPU Cores available, each of which represents 25% of the total CPU available to the system (100% / 4 = 25%). The admin wants to trigger CPU Spikes Protection when a process hits 50% CPU utilisation, therefore the CPU Usage Limit (%) value is set to 50%. Also, for simplicity, let’s assume that the process in the following examples is the only thing consuming CPU.
Scenario 1: We have a multi-threaded process that consumes CPU equally across all available Logical CPU Cores in the system. When this process reaches 50% CPU utilisation, CPU Spikes Protection is triggered. From a monitoring perspective, the utilisation for the system’s overall CPU will show 50% utilisation and the utilisation of each individual Logical CPU Core will show 12.5% utilisation.
Scenario 2: We have a multi-threaded process that can only use two CPU cores. When this process reaches 50% CPU utilisation, CPU Spikes Protection is triggered. From a monitoring perspective, the utilisation for the system’s overall CPU will show 50% but this time the utilisation of two of the individual Logical CPU Cores will show 100%, while the other two Logical CPU Cores will show the utilisation as 0%. Because the CPU Usage Limit (%) is for the entire system’s Logical CPU and not for each individual Logical CPU core, CPU Spikes Protection was not triggered when the CPU utilisation reached 50% in the individual Logical CPU Cores, but instead, when the overall CPU utilisation reached 50%.
Scenario 3: We have a single-threaded process that reaches 50% utilization in a single Logical CPU Core. From a monitoring perspective, the utilisation for this single Logical CPU Core will show 50%. However, the utilisation for the system’s overall CPU will only show 12.5%. Because the CPU Usage Limit (%) is for the entire system’s CPU and not for each individual core, CPU Spikes Protection was not triggered when the CPU utilisation reached 50% in the individual Logical CPU Core. Furthermore, if this process reaches 100% CPU utilisation in that single Logical CPU Core, which would only amount to 25% utilisation of the system’s overall CPU capacity, so CPU Spikes Protection would never be triggered for that process.
In the last scenario, to prevent that single threaded process from “thrashing” a single Logical CPU Core, CPU Usage Limit (%) would need to be set to 25% (or 24% to be safe).
The considerations don’t end there, though!
All processes are treated equally by CPU Spikes Protection. That means that, if we lowered the CPU Usage Limit (%) value to 25% to accommodate the single-threaded process, all other processes running in the system would also be subjected to that 25% CPU utilisation threshold, regardless of whether they are single-threaded or not.
So, what do we have in the end?
Citrix Solutions Architect Miguel Contreras advises his customers to use a CPU Usage Limit (%) value of 25% as a starting point.
In a 4-core system this will prevent single threaded processes from thrashing a single Logical CPU Core, while providing a sensible value for those multi-threaded processes.
There is no one-size-fits-all value, and I’m sure Miguel wouldn’t be hard and fast on his 25% either. In situations where a value higher for CPU Usage Limit (%) is desired but there are some trouble processes that would not allow for this to work effectively, Miguel recommends to look at the other CPU Management features, namely CPU Priority and CPU Clamping, that can be used to manage CPU utilisation for those exceptions.
However, these CPU Management features also come with caveats, as discussed later in this article.
NB: All process violations are logged in the event viewer of the machine hosting the offending process, within Application and Services Logs, Norskale Agent Service, look for “Initializing process limitation thread for process:”
- Limit CPU / Core Usage
What does it do?
Limits a process that has violated the CPU Limit Value to use specific number of CPU Cores.
How does it do it?
Once a managed process has violated the CPU Limit Value, WEM will limit the number of CPU Cores that the offending process can use to the value entered in CPU / Core Limit shown in Figure 3, for the duration of the Idle Priority time.
NB: In order to use the Limit CPU / Core Usage feature, CPU Spikes Protection must be enabled.
- Intelligent CPU Optimisation:
What does it do?
When Intelligent CPU Optimisation is enabled, all processes that a user launches, start with a priority of “high”. Intelligent CPU Optimisation continues to monitor and manage these user launched processes based on their current, and historical behavior.
How does it do it?
The assumption is that if a user is launching the process, that process is important the user and therefore must be performant. So, when Intelligent CPU Optimization is enabled all user launched processes start with a priority of “High”.
However, if that user launched process violates CPU Spikes Protection (Section 1.1), CPU Spikes Protection will demote the offending process, as expected, from a priority of “High” to a priority of “Low” for the duration of the Idle Priority Time (s) value in CPU Spikes Protection (180s in Figure 2: CPU Management Features). WEM also continuously monitors and records any process violations.
If the user launched process violates CPU Spikes Protection a number of times, Intelligent CPU Optimization will step in and permanently demote the offending process from a priority of “High” to a priority of “Above Normal” at launch. If the user launched process still continues to violate CPU Spikes Protection for a further a number of times, Intelligent CPU Optimization will step in again and further demote the offending process from a priority of “Above Normal” to a priority of “Normal” at launch (and so on, until the offending priority reaches “Low”).
The WEM Agent records the historical behavior of every process in the WEM Agent local cache on each machine that has triggered CPU Spikes Protection. It records details such as; the number of times that the process has triggered CPU Spikes Protection, and he user for which the trigger occurred.
NB: In order to use the Intelligent CPU Optimization feature, CPU Spikes Protection must be enabled.
- Enable Intelligent I/O Optimization:
Intelligent I/O optimization adopts exactly the same principles as Enable Intelligent CPU Optimization, but for I/O instead of CPU. So we won’t go into any further detail here.
NB: In order to use the Intelligent I/O Optimization feature, CPU Spikes Protection must be enabled.
- Exclude specified processes:
What does it do?
WEM administrators can also manually add processes they want to exclude from CPU Spikes Protection to the list. As shown above in Figure 6.
How does it do it?
By simply adding the process to the exclusion list, the process will not be managed by WEM. Typically, processes such as Anti-Virus would be added to the exclusion list. (Also, bear in mind that in order to stop Anti-Virus scanning taking over disk I/O in a session, administrators would also set a static I/O Priority of ”Low” for “Anti-Virus” processes. This is of course not exclusive to Anti-Virus processes, and relates to all processes that are deemed I/O hungry (Covered in Section 3, I/O Management).
By default, WEM CPU Management excludes all of the most common Citrix and Windows core service processes. This is because these processes need to make their own decisions about how much CPU time and priority they need to run the environment.
NB: Whenever we configure a process list, the entered process name is a match to the process name’s entry in Windows Task Manager. Process names are not case-sensitive. You also do not enter the extension, so for instance, enter “iexplore” rather than “iexplore.exe”.
- CPU Priority
What does it do?
Processes on a system compete for “CPU Time”. The higher the process priority, the more “CPU time” that process is allocated on a CPU. Using CPU Priority, a WEM administrator can set the priority of a process manually.
How does it work?
You can set the CPU Priority to either:
-
-
- Realtime (Not Recommended)
- High
- Above Normal
- Normal
- Below Normal
- Low.
-
This is called the “Base CPU Priority”
Important: Setting the CPU Priority value as “Realtime” is not recommended. Ever! (Under any circumstances).
Once the base CPU Priority is set, that process will never have a lower priority than the base CPU priority setting.
NB: The process could of course have a higher priority than the base CPU priority you have set, but never lower.
For example, if we set the base CPU priority of a process to “Normal” that process would never run at “Below Normal” or “Low”. However, it could run at “Above Normal”, “High” or “Real-time”.
- CPU Affinity
What does it do?
By using CPU Affinity, WEM Administrators can define how many logical CPU a process can use.
As an example, you could restrict every instance of “iexplore” to a single logical CPU. Thus potentially improving process performance.
How does it do it?
CPU affinity can improve process performance by taking advantage of certain process remnants that were run on a processor, remaining in that processor’s “state”. A good example would be, data in the CPU cache.
Executing that process on the same processor every time it is launched can improve process performance by reducing cache misses.
NB: Underlying processor architecture needs to be taken into consideration here. I.e. CPU Affinity may not be effective in systems with hyper threading enabled, as these virtual CPU’s compete against each other for resources, but also share the same resources they are competing for, including Cache.
- CPU Clamping
What does it do?
CPU Clamping prevents a process from consuming more than a set percentage of a processor’s resources.
How does it do it?
When a clamped process reaches the set percentage, WEM throttles the process. You can therefore prevent CPU heavy processes from degrading overall system performance. This would be particularly useful in an SBC / XenApp environment, where all Users would be effected by a rogue process consuming too much CPU.
The set percentage is applied (per process instance) to the total power of any individual processor in the server, not to any individual processor core. For example, a 30% set percentage on a dual-core processor is 30% of the entire processor, not 15% of one core.
NB: CPU clamping is suitable for controlling processes which are known to be bad resource managers that cannot tolerate being dropped in priority.
However, CPU clamping is computationally expensive and generally perceived as taking a “sledgehammer to crack a walnut” approach. The recommended first approach when tackling troublesome processes would be to use a combination of CPU Spikes Protection, assigning static CPU priorities and CPU affinities.
- Memory Management
When using Memory Management, WEM Administrators are able to optimize an applications RAM usage. This is a great way to increase overall user density.
- Working Set Optimization
What does it do?
When enabled, Working Set Optimization forces idle applications to release any excess memory.
How does it do it?
WEM actively calculates the following:
-
- How much RAM a process is actually using.
- The bare minimum amount of RAM that the application needs to run in a stable manner.
The difference between these two figures is considered “Excess RAM”. When the process is in an idle state, the “Excess RAM” is released to the page file, and the process is also optimized for future usage.
An application is considered to be in an idle state, when the applications CPU usage drops below the set Idle State Limit (percent) – see below. There was a general misconception that the application had to be minimized to the task bar for Memory Management feature to work, this is incorrect.
When the application is maximised, it runs in the optimised state. However, bear in mind that the optimised state is not permanent, and the application could still go on to consume additional RAM as and when the application needs it.
As you can see in Figure 9 above, once we enable Working Set Optimization, we can set 2 values.
- Idle Sample Time (min) – This is the amount of time an application must be in an idle state before its excess RAM is released.
- Idle State Limit (percent) – This is the percentage of CPU usage under which a process is considered to be idle. Citrix do not recommend using a value above 5%: otherwise a process being actively used can be mistaken for an idle process!
NB: A WEM administrator can also exclude specific processes, as shown in Figure 9. Normal process exclusion rules apply! See Section 1.5 – Exclude specified processes.
- I/O Management
What does it do?
I/O Priority allows you to optimize the I/O priority of specific processes to reduce / eliminate performance bottlenecks caused by processes contending for disk and network I/O access.
How does it do it?
I/O Priority adopts exactly the same principles as CPU Priority, but for I/O instead of CPU. So we won’t go into any further detail here.
Please see Section 1.6 CPU Priority for more information.
NB: I/O Priority relates to Disk and Network I/O.
- Fast Logoff
What does it do?
Fast Logoff is used in SBC / XenApp environments to end a user’s session as soon as the user hits Logoff.
How does it work?
When enabled, Fast Logoff ends a user’s HDX session as soon as the user hits Logoff. However, even though the user’s session appears to end immediately, the session actually cycles through the Logoff process on the VDA. This is invisible to the User, who has already walked away from their endpoint device.
This is extremely handy in environments that share Endpoint devices, such as Libraries, Hospitals, Universities and other organisations using a Kiosk type approach.
WEM Administrators can exclude certain groups from Fast Logoff if required, as shown in Figure 11.
NB: Fast Logoff is only supported in SBC / XenApp Environments.
Well, That’s it for Part 1 of this WEM Blog series. I hope that you found some of the information useful.
Watch out for Part 2.
Citrix TechBytes – Created by Citrix Experts, made for Citrix Technologists! Learn from passionate Citrix Experts and gain technical insights into the latest Citrix Technologies.
Click here for more TechBytes and subscribe.
Want specific TechBytes? Let us know! tech-content-feedback@citrix.com
