VuGen scripting for BMC Remedy Action Request System 7.1

I recently created some BPM scripts for the BMC Remedy Action Request System 7.1 web client. This Tech Tip contains some of the things that I learnt.

My favourite part of this exercise was proving that the person from BMC who said "we have already tried, and found that it is impossible to script ARS with VuGen" was totally wrong.

Note that the information should be equally relevant whether you are creating VuGen scripts for LoadRunner, or for use as BPMs for BAC.

Recording and Script Generation

Remedy ARS has both a Win32 client and a web-based client. The Win32 client communicates with the server using a proprietary protocol that is not supported by VuGen. Traffic can be recorded using the Windows Sockets vuser type, but the data is in an unintelligible binary format (just because you can record an application with Winsock doesn't mean you should try to create a script using it).

Note that it might be possible to configure VuGen to record ARS by using the VuGen protocol development kit, and referring to the ARS API documentation, but time limitations prevented me from experimenting with this.

The Web (Click and Script) vuser type does not successfully record ARS, but it is possible to create a working script using the Web (HTTP/HTML) vuser type.

When recording, you must either use URL mode (so that each HTTP request is a separate function call), or change your recording settings to treat the MIME type "text/plain" as a non-resource (Recording Options > HTTP Properties > Advanced > Non-Resources). This will ensure that the calls to BackChannel are not recorded as EXTRARES.

VuGen 9.5 has an odd behaviour when generating code for the BackChannel requests. These appear as a web_custom_request with "Method=GET", but still have a Body parameter with the same value as the arguments passed in the URL (obviously GET requests should not have a body). As long as the HTTP method is GET, then this body may be safely removed.

Backchannel Requests

You may have already guessed that BackChannel requests are kind of important for ARS. These requests are sent behind the scenes to fetch data to display in the GUI and to update data on the server. This means that you will be scripting "blind" as VuGen will not show you what any pages look like in either the Tree View or the Test Results window.

Here is an example of a BackChannel request:

// Lookup of Incident # INC00000126798
web_custom_request("BackChannel_15", 
    "URL=https://remedy-ars.example.com/arsys/BackChannel/?param=416%2FGetEntryList%2F22%2Fremedy-ars.example.com31%2FHPD%3AIncident%20Management%20Console22%2Fremedy-ars.example.com13%2FHPD%3AHelp%20Desk0%2F30%2F4%5C1%5C1%5C1000000161%5C99%5C302085700%5C13%2F1%2F9%2F30208570020%2F1%2F15%2FINC0000012679875%2F1%2F1%2F41%2F21%2F3212%2F17%2F9%2F24000100510%2F100000000110%2F10000057919%2F20000000510%2F100000000010%2F100000005610%2F100000016410%2F100000015110%2F100000008010%2F10000002179%2F20000000310%2F100000056010%2F10000057859%2F2000000049%2F24000100310%2F10000000999%2F240001002", 
    "Method=GET", 
    "TargetFrame=", 
    "Resource=0", 
    "RecContentType=text/plain", 
    "Referer=https://remedy-ars.example.com/arsys/forms/remedy-ars.example.com/HPD%3AIncident+Management+Console/Default+User+View+%28Support%29/?cacheid=70246b58", 
    "Snapshot=t22.inf", 
    "Mode=HTML", 
    "EncType=text/plain; charset=UTF-8", 
    LAST);
/*
URL decoded request:
https://remedy-ars.example.com/arsys/BackChannel/?param=416/GetEntryList/22/remedy-ars.example.com31/HPD:Incident Management Console22/remedy-ars.example.com13/HPD:Help Desk0/30/4\1\1\1000000161\99\302085700\13/1/9/30208570020/1/15/INC0000012679875/1/1/41/21/3212/17/9/24000100510/100000000110/10000057919/20000000510/100000000010/100000005610/100000016410/100000015110/100000008010/10000002179/20000000310/100000056010/10000057859/2000000049/24000100310/10000000999/240001002

JSON response:
this.result={n:1,f:[{t:0},{t:4,v:"JDS"},{t:4,v:"PPL000000132801"},{t:4,v:"Network Management Systems"},{t:4,v:"Test Environment: Please free swap space on following Production servers\r\n"},{t:4,v:"(03) 9663 4573"},{t:6,v:"2 Medium"},{t:4,v:"Test Environment: Please free swap space on following Production servers\r\n\r\jdsweb01.jds.net.au                             \r\jdsweb02.jds.net.au  \r\jdsapp01.jds.net.au                             \r\jdsapp02.jds.net.au                            "},{t:4,v:"PPL000000132801"},{t:4,v:"JDS Remedy Test Support"},{t:4,v:"Services"},{t:7,v:"1248074292"},{t:4,v:"(03) 9663 4573"},{t:4,v:"Systems"},{t:4,v:"JDS"},{t:6,v:"1 User Service Request"},{t:4,v:"ARS"},{ts:1249950456}]};

The data in the URL "param=" argument becomes a little clearer once you URL decode it.

• The first part of the string (416/) is the length of the string, not including this first length parameter and the delimiter.
• The second part of the string (GetEntryList) is the action that is being called on the server. Other possible values include:
  • GetURLForForm
  • GetEntry
  • GetEntryList
  • GetTableEntryList
  • GetCurrencyExchangeRates
  • SetEntry
  • SetEntryList
  • SetOverride
  • SaveSearch
  • ExpandMenu (used to populate a drop-down list)
  • CompileExternalQualification
  • ServerRunProcess
• Each value after the action has a field length argument immediately before it e.g. "22/remedy-ars.example.com31/HPD:Incident Management Console". Note that 0 is a valid length, and a length value may be specified that includes muliple "/" characters (like the "212/" value in the request above).
• Creating a BackChannel request that does not conform to this format may result in a "Network protocol/data error" response but, then again, it may not - the BackChannel requests don't always seem to be validated on the server.

Error Messages and Verification

A successful BackChannel request will always return "this.result=", so it is a good first step to put a basic web_reg_find function before each BackChannel request to verify that this string appears in the HTTP response.

web_reg_find("Text=this.result", LAST);

Other possible responses start with "CurWFC.status" or "retry=this.Override". Some examples are:

• CurWFC.status([{cId:1,t:2,m:"Network protocol/data error when performing data operation. Please contact administrator.",n:9350,a:""}]);
• CurWFC.status([{t:2,m:"",n:1291039,a:"The incident location information is invalid. Use the menus on the Region, Site Group, and Site fields or the type ahead return function on the Site field to select this information."}]);
• CurWFC.status([{cId:1,t:2,m:"Session is invalid or has timed out. Please reload page to log in again.",n:9201,a:""}]);
• CurWFC.status([{cId:1,t:2,m:"Message not found",n:-1,a:""}]);
• CurWFC.status([{cId:1,t:2,m:"User is currently connected from another machine",n:9084,a:""}]);
• retry=this.Override([{cId:1,t:2,m:"User is currently connected from another machine",n:9093,a:""}]);

The difference between the last two messages is that the last message can be bypassed by acknowledging the message and continuing, while the second-last message cannot be bypassed and will prevent a user from opening the requested screen/module inside ARS.

When BackChannel returns an empty recordset (like from GetEntryList), it will look like "this.result={n:0};"

Save and Set functions may indicate success by returning "this.result=1" or "this.result=true".

But most of the time BackChannel will return data in a format with 1 elements (n=1), with the element being an array of values (f:[{t:0},{t:4,v:"JDS"},{t:4,v:"PPL000000132801"}]).

Here are some for known error messages:

// Known error messages
web_global_verification("Text=Authentication failed", "ID=AuthenticationFailed", LAST);
web_global_verification("Text=The incident location information is invalid", "ID=IncidentLocationInformationIsInvalid", LAST);
web_global_verification("Text=Network protocol/data error when performing data operation", "ID=NetworkProtocolDataError", LAST);
web_global_verification("Text=Please contact administrator", "ID=PleaseContactAdministrator", LAST);
web_global_verification("Text=Session is invalid", "ID=SessionIsInvalidOrHasTimedOut", LAST);
web_global_verification("Text=Message not found", "ID=MessageNotFound", LAST);
web_global_verification("Text=User is currently connected from another machine", "ID=UserIsCurrentlyConnectedFromAnotherMachine", LAST);	    

Code Snippets

For some reason ARS will create a hash of the password before sending it to the server. While it is possible to reinvent their hashing function, there is no point. When you parameterise the password, just put the hashed value in your parameter file, and ensure that all accounts have the same password.

web_submit_data("LoginServlet", 
	"Action=https://{ServerName}/arsys/servlet/LoginServlet", 
	"Method=POST", 
	"TargetFrame=", 
	"RecContentType=text/html", 
	"Referer=https://{ServerName}/arsys/shared/login.jsp", 
	"Snapshot=t2.inf", 
	"Mode=HTML", 
	ITEMDATA, 
	"Name=username", "Value=bpm01", ENDITEM, 
	"Name=pwd", "Value=pthkhphshjhkhk", ENDITEM, // welcome4
	"Name=auth", "Value=", ENDITEM, 
	"Name=timezone", "Value=AET", ENDITEM, 
	"Name=encpwd", "Value=1", ENDITEM, 
	"Name=goto", "Value=", ENDITEM, 
	"Name=server", "Value=", ENDITEM, 
	"Name=ipoverride", "Value=0", ENDITEM, 
	"Name=initialState", "Value=0", ENDITEM, 
	"Name=returnBack", "Value=null", ENDITEM, 
	LAST);

Timestamps are used in many requests. These are either in seconds or milliseconds since 1/Jan/1970. Be careful not to confuse a timestamp in seconds with a length argument after it for a timestamp in milliseconds. This will cause a "Network protocol/data error when performing data operation" error.

// Example in C
// Timestamp in seconds
lr_save_int(time(NULL), "pTimestamp_1"); // 1249950809
// Timestamp in milliseconds
web_save_timestamp_param("pTimestamp_2"); // 1249950434718

// Example in Java
// Timestamp in seconds
lr.save_int((int) (System.currentTimeMillis()/1000), "pTimestamp_1"); // 1249950809
// Timestamp in milliseconds
lr.save_string(System.currentTimeMillis() + "", "pTimestamp_2"); // 1249950434718

The following code example deals with the situation where the account is already logged onto the system. It will confirm the dialog window (SetOverride) and retry the request.

// Handle case where dialog window pops up saying "User is currently connected from another machine".
web_reg_find("Text=this.result", "SaveCount=TextCheckCount", LAST); // This happens normally.
web_reg_find("Text=User is currently connected from another machine", "SaveCount=LoggedInCount", LAST); // This happens when the account is already logged on.
web_custom_request("BackChannel_2", 
	// https://{ServerName}/arsys/BackChannel/?param=170/GetEntryList/22/{ServerName}31/HPD:Incident Management Console22/{ServerName}15/AST:AppSettings0/16/4\1\2\2\1\2\2\1\2/0/2/0/2/0/1/21/313/1/9/400127400"
	"URL=https://{ServerName}/arsys/BackChannel/?param=170%2FGetEntryList%2F22%2F{ServerName}31%2FHPD%3AIncident%20Management%20Console22%2F{ServerName}15%2FAST%3AAppSettings0%2F16%2F4%5C1%5C2%5C2%5C1%5C2%5C2%5C1%5C2%2F0%2F2%2F0%2F2%2F0%2F1%2F21%2F313%2F1%2F9%2F400127400", 
	"Method=GET", 
	"TargetFrame=", 
	"Resource=0", 
	"RecContentType=text/plain", 
	"Referer=https://{ServerName}/arsys/forms/{ServerName}/HPD%3AIncident+Management+Console/Default+User+View+%28Support%29/?cacheid={pCacheID_2}", 
	"Snapshot=t9.inf", 
	"Mode=HTML", 
	"EncType=text/plain; charset=UTF-8", 
	LAST);
// RETURNS: retry=this.Override([{cId:1,t:2,m:"User is currently connected from another machine",n:9093,a:""}]);
// RETURNS: CurWFC.status([{cId:1,t:2,m:"User is currently connected from another machine",n:9084,a:""}]); // if this response is received, there does not seem to be a way to bypass it.
// RETURNS: this.result={n:1,f:[{t:4,v:"BMC.ASSET"},{ts:1249950451}]};

if (strcmp(lr_eval_string("{LoggedInCount}"), "1") == 0) {
	// User is already logged in.
	// Confirm dialog window and continue.
	lr_output_message("Account is in use by another session. Script will confirm dialog and continue. LoggedInCount: %s, TextCheckCount: %s", lr_eval_string("{LoggedInCount}"), lr_eval_string("{TextCheckCount}"));

	web_reg_find("Text=this.result=true", LAST);  
	web_custom_request("BackChannel_2b", 
		// https://{ServerName}/arsys/BackChannel/?param=15/SetOverride/1/1"
		"URL=https://{ServerName}/arsys/BackChannel/?param=15%2FSetOverride%2F1%2F1",  
		"Method=GET", 
		"TargetFrame=", 
		"Resource=0", 
		"RecContentType=text/plain", 
		"Referer=https://{ServerName}/arsys/forms/{ServerName}/HPD%3AIncident+Management+Console/Default+User+View+%28Support%29/?cacheid={pCacheID_2}", 
		"Snapshot=t9.inf", 
		"Mode=HTML", 
		"EncType=text/plain; charset=UTF-8", 
		LAST);
	// RETURNS: this.result=true;

	web_reg_find("Text=this.result", LAST);
	web_reg_find("Text=User is currently connected from another machine", "Fail=Found", LAST);
	web_custom_request("BackChannel_2c", 
	// https://{ServerName}/arsys/BackChannel/?param=170/GetEntryList/22/{ServerName}31/HPD:Incident Management Console22/{ServerName}15/AST:AppSettings0/16/4\1\2\2\1\2\2\1\2/0/2/0/2/0/1/21/313/1/9/400127400"
		"URL=https://{ServerName}/arsys/BackChannel/?param=170%2FGetEntryList%2F22%2F{ServerName}31%2FHPD%3AIncident%20Management%20Console22%2F{ServerName}15%2FAST%3AAppSettings0%2F16%2F4%5C1%5C2%5C2%5C1%5C2%5C2%5C1%5C2%2F0%2F2%2F0%2F2%2F0%2F1%2F21%2F313%2F1%2F9%2F400127400",
		"Method=GET", 
		"TargetFrame=", 
		"Resource=0", 
		"RecContentType=text/plain", 
		"Referer=https://{ServerName}/arsys/forms/{ServerName}/HPD%3AIncident+Management+Console/Default+User+View+%28Support%29/?cacheid={pCacheID_2}", 
		"Snapshot=t9.inf", 
		"Mode=HTML", 
		"EncType=text/plain; charset=UTF-8", 
		LAST);
	// RETURNS: this.result={n:1,f:[{t:4,v:"BMC.ASSET"},{ts:1249950451}]};
	// RETURNS: CurWFC.status([{cId:1,t:2,m:"User is currently connected from another machine",n:9084,a:""}]); <-- this is returned if the dialog confirm did not work.

} else if (strcmp(lr_eval_string("{TextCheckCount}"), "1") == 0) {
	// Everything is working as expected. Continue script execution.
	lr_output_message("Account is not in use by another session, and everything is working as expected. LoggedInCount: %s, TextCheckCount: %s", lr_eval_string("{LoggedInCount}"), lr_eval_string("{TextCheckCount}"));
} else {
	// Did not find expected text
	lr_error_message("Error. Did not get \"user is currently connected\", but also did not get expected text. Something is wrong with Remedy ARS.");
	lr_exit(LR_EXIT_VUSER, LR_FAIL);
}

Correlation

Correlation is quite difficult for Remedy ARS. My approach was very manual:

• Record the business process twice with the same input data, then WDiff to find request values that change between iterations.
• Record the business process twice with different input data (different username, different "incident" data etc), then WDiff to find other request values that change.
• VuGen will break long strings (like the BackChannel URL) into multiple lines. This will be a problem for you if you are trying to do a search and replace for values you are correlating/parameterising. I put all long strings onto a single line.
• The BackChannel URLs are hard to read in their URL encoded form. I put the URL decoded value above in a comment above the URL.
• I found that I had a clearer idea of what the application was doing during the business process when I cut and pasted the JSON response below each BackChannel request.
• If you find that you have to do difficult string processing or use someone else's JSON library, you may find it easier to convert your C-based script into a Java-based script. Note that there are some bugs with HP's sed script that does the conversion. It does not add "new String[]{" to the end of the web_custom_request "URL" argument, when the URL agument spans multiple lines. It will also change any occurances of LASTID into LAST}}ID and LASTCOUNT into LAST}}COUNT.

Finally, you will be amazed at the values that look like they should be correlated that actually never change. Remember that the only way to be sure is to use WDiff (Tools > Compare with Script). Just don't expect these values to stay the same if the version of Remedy ARS changes, or you user profiles change.