During some unique circumstances, there might be a need to create persistence profiles between web services bound to different virtual servers used to load balance disparate web applications. An example scenario is described below:
Scenario:
- Two VIPs for abc.company.com and xyz.company.com
- 1.1.1.11 and 1.1.1.12 load balanced for abc.company.com; 1.1.1.21 and 1.1.1.22 load balanced for xyz.company.com
- When user requests abc.company.com and request is sent to 1.1.1.11, then when the same user sends a request to xyz.company.com, the request should be sent to 1.1.1.21
- When user requests abc.company.com and request is sent to 1.1.1.12, then when the same user sends a request to xyz.company.com, the request should be sent to 1.1.1.22
Challenges:
- For this we need persistence profiles between backend services but we currently only support persistence profiles for LB VIPs.
- CS VIP gives us content switching but not load balancing
- This would also require us to take the load balancing decision at the CS vServer level as opposed to the usual LB vServer level
- This requires a custom persistence profile between actual web servers not between virtual servers
High Level approach on NetScaler:
- 4 separate LB vServers for each backend services using null IP addresses
- 2 CS vServers, one for abc.company.com and one for xyz.company.com
- Insert cookies with number of connections going to each of the four backend services using rewrite policies on the NetScaler
- Use values in those cookies to do least connections based load balancing. If vip1 has more connections than vip2, then send connection to vip2, and vice versa. Do the same for both CS vServers.
- This can be done using throughput values, source IPs, etc. by inserting different values to the cookies
- Create persistence groups on the NetScaler between vip1 (1.1.1.11) and vip3 (1.1.1.21) and another for vip2 (1.1.1.12) and vip4 (1.1.1.22).
- The persistence groups will maintain persistence between vip1 and vip3 so when connection for abc.company.com goes to vip1, connection for xyz.company.com goes to vip3 (1.1.1.21).
Configuration:
##Create 4 services for the backend web servers
add service company-11 1.1.1.1 HTTP 80 -gslb NONE -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -sp ON -cltTimeout 180 -svrTimeout 360 -CustomServerID “\”None\”” -CKA NO -TCPB NO -CMP YES
add service company-12 1.1.1.2 HTTP 80 -gslb NONE -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -sp ON -cltTimeout 180 -svrTimeout 360 -CustomServerID “\”None\”” -CKA NO -TCPB NO -CMP YES
add service company-21 1.1.1.3 HTTP 80 -gslb NONE -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -sp ON -cltTimeout 180 -svrTimeout 360 -CustomServerID “\”None\”” -CKA NO -TCPB NO -CMP YES
add service company-22 1.1.1.4 HTTP 80 -gslb NONE -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -sp ON -cltTimeout 180 -svrTimeout 360 -CustomServerID “\”None\”” -CKA NO -TCPB NO -CMP YES
##Create 4 LB vServers, one for each backend web servers, using null IP addresses
add lb vserver company-lbvip11 HTTP 0.0.0.0 0 -cltTimeout 180
add lb vserver company-lbvip12 HTTP 0.0.0.0 0 -cltTimeout 180
add lb vserver company-lbvip13 HTTP 0.0.0.0 0 -cltTimeout 180
add lb vserver company-lbvip14 HTTP 0.0.0.0 0 -cltTimeout 180
##Bind the LB vServers with the respective services
bind lb vserver company-lbvip11 company-12
bind lb vserver company-lbvip12 company-12
bind lb vserver company-lbvip13 company-21
bind lb vserver company-lbvip14 company-22
##Create 2 Content Switching vServers, one for each abc.company.com and xyz.company.com
add cs vserver company-csvip-abc.company.com HTTP 3.3.3.3 80 -cltTimeout 180
add cs vserver company-csvip-xyz.company.com HTTP 4.4.4.4 80 -cltTimeout 180
##Create 4 Content Switching policies that compare the number of connections on each of the LB vServers pointed at the respective backend web servers.
##This emulates Least Connections Load Balancing based on values written into cookies as rewrite policies below. The type of load balancing can be selected
##to be based on Source IP addresses, the cookie values, bandwidth used by each of the servers, etc. If connections on vip1 is greater than connections on vip2, then
##the CS vServer sends connection to vip2 and so on and so forth.
add cs policy company-cspol_vip1 -rule “HTTP.REQ.COOKIE.VALUE(\”vip1\”).TYPECAST_NUM_AT > http.REQ.COOKIE.VALUE(\”vip2\”).TYPECAST_NUM_AT”
add cs policy company-cspol_vip2 -rule “HTTP.REQ.COOKIE.VALUE(\”vip2\”).TYPECAST_NUM_AT > http.REQ.COOKIE.VALUE(\”vip1\”).TYPECAST_NUM_AT”
add cs policy company_cspol_xyz_vip3 -rule “HTTP.REQ.COOKIE.VALUE(\”vip3\”).TYPECAST_NUM_AT > http.REQ.COOKIE.VALUE(\”vip4\”).TYPECAST_NUM_AT”
add cs policy company_cspol_xyz_vip4 -rule “HTTP.REQ.COOKIE.VALUE(\”vip4\”).TYPECAST_NUM_AT > http.REQ.COOKIE.VALUE(\”vip3\”).TYPECAST_NUM_AT”
bind cs vserver company-csvip-abc.company.com -policyName company-cspol_vip2 -targetLBVserver company-lbvip11 -priority 90
bind cs vserver company-csvip-abc.company.com -policyName company-cspol_vip1 -targetLBVserver company-lbvip12 -priority 100
bind cs vserver company-csvip-xyz.company.com -policyName company_cspol_xyz_vip4 -targetLBVserver company-lbvip13 -priority 90
bind cs vserver company-csvip-xyz.company.com -policyName company_cspol_xyz_vip3 -targetLBVserver company-lbvip14 -priority 100
##These rewrite policies insert values for the number of connections on vip1, vip2 to all requests to Content Switching vServer for abc.company.com and vip3,vip4 for all requests going
##to Content Switching vServer for xyz.company.com. If these cookies already exist, then it replaces these values with updated number of connections on the
##backend web servers.
add rewrite action company_abc_cookie_insert1 insert_http_header vip2 “SYS.VSERVER(\”company-lbvip12\”).CONNECTIONS” -bypassSafetyCheck YES
add rewrite action company_xyz_cookie_insert insert_http_header vip3 “SYS.VSERVER(\”company-lbvip13\”).CONNECTIONS” -bypassSafetyCheck YES
add rewrite action company_xyz_cookie_insert1 insert_http_header vip4 “SYS.VSERVER(\”company-lbvip14\”).CONNECTIONS” -bypassSafetyCheck YES
add rewrite action company_abc_insert_cookie insert_http_header vip1 “SYS.VSERVER(\”company-lbvip11\”).CONNECTIONS.TYPECAST_NUM_AT” -bypassSafetyCheck YES
add rewrite action company_delete_cookie_vip1 delete “http.REQ.COOKIE.NAME_VALUE(\”vip1\”)” -bypassSafetyCheck YES
add rewrite action company_replace_cookie_vip2 replace “http.REQ.COOKIE.NAME_VALUE(\”vip1\”)” “\”vip1=\”+ SYS.VSERVER(\”company-lbvip12\”).CONNECTIONS.TYPECAST_NUM_AT” -bypassSafetyCheck YES
add rewrite action company_replace_cookie_vip1 replace “http.REQ.COOKIE.NAME_VALUE(\”vip1\”)” “\”vip1=\”+ SYS.VSERVER(\”company-lbvip11\”).CONNECTIONS.TYPECAST_NUM_AT” -bypassSafetyCheck YES
add rewrite action company_xyz_replace_cookie_vip3 replace “http.req.COOKIE.NAME_VALUE(\”vip3\”)” “\”vip3=\”+ SYS.VSERVER(\”company-lbvip13\”).CONNECTIONS.TYPECAST_NUM_AT” -bypassSafetyCheck YES
add rewrite action company_xyz_replace_cookie_vip4 replace “http.REQ.COOKIE.NAME_VALUE(\”vip4\”)” “\”vip4=\”+ SYS.VSERVER(\”company-lbvip14\”).CONNECTIONS.TYPECAST_NUM_AT” -bypassSafetyCheck YES
add rewrite policy insert_header1_pol “http.REQ.HOSTNAME.CONTAINS(\”abc.company.com\”)” insert_header1
add rewrite policy company_abc_insert_cookie_pol “http.REQ.HOSTNAME.CONTAINS(\”abc.company.com\”) && !HTTP.REQ.COOKIE.CONTAINS(\”vip1\”)” company_abc_insert_cookie
add rewrite policy company_abc_insert_cookie_pol1 “http.REQ.HOSTNAME.CONTAINS(\”abc.company.com\”)&&!http.REQ.COOKIE.CONTAINS(\”vip2\”)” company_abc_cookie_insert1
add rewrite policy company_xyz_cookie_insert_pol “http.REQ.HOSTNAME.CONTAINS(\”xyz.company.com\”)&&!http.REQ.COOKIE.CONTAINS(\”vip3\”)” company_xyz_cookie_insert
add rewrite policy company_xyz_cookie_insert1 “http.REQ.HOSTNAME.CONTAINS(\”xyz.company.com\”)&&!http.REQ.cookie.CONTAINS(\”vip4\”)” company_xyz_cookie_insert1
add rewrite policy company_abc_replace_cookie_pol1 “http.REQ.HOSTNAME.CONTAINS(\”abc.company.com\”) && HTTP.REQ.COOKIE.CONTAINS(\”vip2\”)” company_replace_cookie_vip2
add rewrite policy company_xyz_replace_cookie_pol1 “http.REQ.HOSTNAME.CONTAINS(\”xyz.company.com\”) && HTTP.REQ.COOKIE.CONTAINS(\”vip4\”)” company_xyz_replace_cookie_vip4
add rewrite policy company_xyz_replace_cookie_pol “http.REQ.HOSTNAME.CONTAINS(\”xyz.company.com\”) && HTTP.REQ.COOKIE.CONTAINS(\”vip3\”)” company_xyz_replace_cookie_vip3
add rewrite policy company_abc_replace_cookie_pol “http.REQ.HOSTNAME.CONTAINS(\”abc.company.com\”) && HTTP.REQ.COOKIE.CONTAINS(\”vip1\”)” company_replace_cookie_vip1
##Bind Content Switching vServer to the rewrite policies
bind cs vserver company-csvip-abc.company.com -policyName company_abc_insert_cookie_pol1 -priority 90 -gotoPriorityExpression NEXT -type REQUEST
bind cs vserver company-csvip-abc.company.com -policyName company_abc_insert_cookie_pol -priority 100 -gotoPriorityExpression END -type REQUEST
bind cs vserver company-csvip-abc.company.com -policyName company_abc_replace_cookie_pol -priority 105 -gotoPriorityExpression NEXT -type REQUEST
bind cs vserver company-csvip-abc.company.com -policyName company_abc_replace_cookie_pol1 -priority 110 -gotoPriorityExpression END -type REQUEST
bind cs vserver company-csvip-xyz.company.com -policyName company_xyz_cookie_insert1 -priority 90 -gotoPriorityExpression NEXT -type REQUEST
bind cs vserver company-csvip-xyz.company.com -policyName company_xyz_cookie_insert_pol -priority 100 -gotoPriorityExpression END -type REQUEST
bind cs vserver company-csvip-xyz.company.com -policyName company_xyz_replace_cookie_pol -priority 110 -gotoPriorityExpression NEXT -type REQUEST
bind cs vserver company-csvip-xyz.company.com -policyName company_xyz_replace_cookie_pol1 -priority 120 -gotoPriorityExpression END -type REQUEST
##Creates persistence group/profile between vip1 and vip3 (bound to 1.1.1.11 and 1.1.1.21) and another persistence group/profile between vip2 and vip4 ((bound to 1.1.1.12 and 1.1.1.22)
##This would ensure that when a connection goes to the .11 address when a user requests abc.company.com, persistence is
##maintained between .11 and .13 and the subsequent request to xyz.company.com goes to .21.
set lb group company_persistence_11_and_21 -persistenceType COOKIEINSERT -persistenceBackup SOURCEIP -cookieName custom_cookie_company -cookieDomain company.com
set lb group company_persistence_12_and_22 -persistenceType COOKIEINSERT -persistenceBackup SOURCEIP -cookieName custom_cookie_company -cookieDomain company.com
bind lb group company_persistence_11_and_21 company-lbvip11
bind lb group company_persistence_11_and_21 company-lbvip13
bind lb group company_persistence_12_and_22 company-lbvip12
bind lb group company_persistence_12_and_22 company-lbvip14