A few weeks ago I wrote a blog post on how to empower and extend your default Wireshark configuration titled NetScaler + Wireshark = A Perfect Combination! I’d like to follow up that post with this article which focuses on how to interpret some of the data that you may see in a NetScaler trace.
The TCP Reset [RST]
Quick Tip: If you want a general overview of what a TCP reset is, then please visit the following URL to gain some high level insight: http://en.wikipedia.org/wiki/TCP_reset_attack
There are numerous different reasons for why a TCP reset [RST] may have occurred, but understanding why the [RST] was issued by one of the TCP end-points will provide you with insight into why a particular TCP communication flow was stopped.
As a note, a TCP [FIN] is similar to a TCP [RST] in that they both conclude a TCP communication. However, [FIN]’s are the gracious means of ending the communication and a [RST] is a rather abrupt method for terminating the communication, perhaps similar to slamming the door or hanging up the phone rather abruptly.
OK, so you’ve taken a NetScaler trace and you’d like to see if there are any TCP [RST]’s in the trace. Simply put the following filter expression (tcp.flags.reset == 1) into Wireshark and click “Apply”:
If there are any TCP [RST]’s from applying the filter, you’ll see them in the presented output, coupled with also seeing the SOURCE IP address which sent the [RST]:
Highlight the packet row with the actual [RST] and look at the packet detail pane under Transmission Control Protocol. You’ll see that the TCP Reset flag is set with the (1) value:
Now that you’ve seen that a particular HOST has issued a [RST], how do you determine what the reason was for the end of communication? Well, in the previous graphic you can see additional detail included in the “Info” section, such as the Seq, Win and Len respectively.
Make note of the “Win” field. In the example provided you can see that the value is Win=9700. This field gives you the NetScaler-assigned code for the actual [RST].
When you see a TCP [RST] issued, jot down the Win value and then reference the following chart to see why the [RST] was issued:
