I’ve been putting together a page of useful tools for investigating and profiling Citrix XenServer. This is designed as part of a site we’ve put together to help those designing applications for XenServer to test and optimise their environments but will also be useful for enthusiastic system administrators for identifying bottlenecks and implementing performance improvements for XenServer (see my post on performance tuning XenServer). This should be useful to those working with Citrix XenServer for server virtualisation or as a platform for XenDesktop, CloudPlatform, Netscaler SDX or similar. In addition some advice on tools for investigating specific features such as GPU passthrough or vGPU can be found here.
So here you go 10 useful tools for understanding XenServer (alphabetical):
1. Bonnie++
This is an open source project which helps measure and investigate network load and includes tools to investigate large file IO and creation/deletion of small files. There are lots of user guides and blogs on this tool.
2. CPU-Z
CPU-Z can be useful particularly if considering: How to investigate and use Turbo mode, C-States and P-States in XenServer
3. IOMeter
IOMeter is an open source tool, an I/O subsystem measurement and characterization tool for single and clustered systems. IOMeter is an easy way to generate stress on the I/O system and as such can be very useful within development test of products interacting with or generating load on the OVS.
4. iperf
iperf is an open source utility that can be a very useful for diagnosing network issues in a XenServer environment. There are a large number of how-to-guides and introductory tutorials available such as here.
5. OProfile
OProfile is an open source tool available from http://oprofile.sourceforge.net/news/. A Xen specific varient is currently shipped in versions XS6.1 and upwards. It is detailed here: http://xenoprof.sourceforge.net/xenoprof_2.0.txt
6. vhd-util
vhd-util is an unsupported tool shipped with XenServer and as such should never be used as an “API” around which to construct an application relying on its provision or stability of results. However it is very useful as a tool for working with VHDs and snapshots. It can be used to check, display and understand VHD files including snapshot chains. There is limited documentation and you will probably need to refer to the command line help by typing “vhd-util” at the XenServer command line and then requesting help for the desired option e.g.
[root@dot56 ~]# vhd-util
usage: vhd-util COMMAND [OPTIONS]
COMMAND := { create | snapshot | query | read | set | repair | resize | fill | coalesce | modify | scan | check | revert }
[root@dot56 ~]# vhd-util check -h
options: -n <file> [-i ignore missing primary footers] [-I ignore parent uuids] [-t ignore timestamps] [-p check parents] [-b check bitmaps] [-s stats] [-h help]
[root@dot56 ~]#
Some further information is available:
- on this Citrix blog
- by searching the Citrix Knowledge Base for “vhd-util”
- searching the Citrix Support Forums
7. WinDbg
WinDbg is one of a number of tools available for Debugging Windows Guests on XenServer (see link for details)
8. XenCenter
A wide range of metrics are available for XenServer including: C-State, P-State, IOPS, Latency and many more. Not all of these are turned on by default. The full range of metrics available for both Hosts and VMs are detailed in Chapter 9 of the XenServer 6.2 Administrators Guide. This chapter also details how you can explore these metrics via XenCenter.
9. xentop
xentop displays real-time information about a Xen system. It is shipped with xen tools. This Citrix Support article details its use further: http://support.citrix.com/article/CTX127896.
10. xl
The utility xl is actually part of the upstream Xen hypervisor developed as Open Source, as such the utility isn’t maintained by Citrix in XenServer but by Xen.org.
We generally would not recommend that a XenServer developer use xl routinely to configure test cases or similar as it affects only part of the toolstack and as such the XenAPI and XAPI will be unaware of changes that can lead to a very confused toolstack and some rather strange effects.
xl can however be of some use for debug and diagnosis, particularly the inquiry options such as info [-n, --numa], by which you can query hardware information such as cores_per_socket and threads_per_core and similar data that you might want to log or keep in benchmarking assessments.
I intend to update the page on which this blog post is based, so please do add any others to the comments below, or feedback on known gotchas with those I have suggested.