What is Oracle WebCenter

Oracle WebCenter is a technology used by Oracle as its internal intranet, supporting 100,000 users world-wide.  Oracle WebCenter leverages the potential of all members of an organization.  Tuning this system to perform is a non-trivial task.

Oracle WebCenter is highly customizable dynamic intranet.  There are many different widgets that can be added into the system.  As a consequence of the level of customization, performance issues result from any of these customizations made during the implementation of the system.

Oracle WebCenter Correlation

The core technology of Oracle WebCenter is Oracle Application Development Framework (ADF). Oracle released a White Paper in March of 2011, Techniques for Testing Performance/Scalability and Stress-Testing ADF Applications.  This white paper acknowledges that the system has issues with sessions releasing memory.

“Session timeout should be set as low as can be tolerated. A good rule of thumb is to set the session timeout so low that users of  the application complain, and gradually add a little more time until the volume of complaints tapers off.”

The white paper takes a swing at Performance Testing tools, particularly correlation within the script.

“This process, often called “correlation”, is critically important. Without it, the application will not behave correctly when running the simulated user load, and will leak memory that otherwise wouldn’t happen.”

If you have handled the _afrLoop, _adf.ctrl-state and ViewState variables, you have taken care of the correlation items critical to scripting success.  There are occasionally other variables which need correlation, but these are the core dynamic values returned.

Client Side variable

The only generated client side variable that I found was ‘unique’.  This variable is embedded into an URL.

&unique=1334545456422
 
web_save_timestamp_param(strMyTimestamp, LAST);
lr_param_sprintf("UniqueID", "%s", strMyTimestamp);
&unique={UniqueID}

General Architecture

  • F5                                           – Load Balancer
  • Application Server(s)      – Series of JVMs
  • SES Application Server   – Secure Enterprise Search
  • Database Server(s)         – Oracle database
  • Authentication                  – OAM, OID, LDAP, AD
Machines for an implementation of Oracle's WebCenter
A typical architectural deployment of Oracle’s WebCenter

Monitoring

Deploying performance counters against CPU, Memory and NIC for each server is important.  Deploying counters against the Oracle database employed by this application is important.  Even more important is determining a method for monitoring the JVMs that serve up the application.

Application Servers – WebCenter JVMs:

  • WC_SPACES                       –              WebCenter core intranet application
  • WC_UCM                            –              Universal Content Manager
  • WC_PORTLET                     –              ISR168 Portlets – may or may not be implemented
  • WC_COLLABORATION   –              Discussion Forums
  • WC_UTILITIES                    –              Analytics and Personalization
  • SOA_SERVER                     –              Service Oriented Architecture
  • IBR_SERVER                       –              Enable image conversion
  • CPS_SERVER                      –              UCM Content Portlets

WC_SPACES JVMS form the core of the Oracle WebCenter system.  Logging into the system a session is created within SPACES.

Preparing to Test WebCenter

Building a number of users equal to or greater than the number of virtual users called or in test runs is important for testing WebCenter.  Each of these should be used interchangeably on all of the different scripts.  Each user carries its own context and session.  The context and session information carried by each user not only makes the test more realistic, but also the additional context forces the system to work harder.   I found this out the hard way by testing using the same credentials for each VU.  The system could handle this without issue.  Running the same test and then using different credentials the application failed to perform.

When running these tests either use Unique Once for the retrieval of user credential data.  Using sequential in any format for user credential data caused unnatural data behavior in the system, and can provide positive test results that mask issues still lurking within the system.

Many performance test engagements require creating data, which can be ploughed through during testing.  This consumable data for a WebCenter engagement is ‘workspaces’.  Workspace creation and configuration is a two-step process.  The first process is creating workspaces.  The second process is adding all of the test users, as MODERATORs to these workspaces.  Both of these processes should be scripted.

Your created workspaces and added virtual users allow different operations to be performed:

  • Create Discussions
  • Commenting on Discussions
  • Searching spaces
  • Uploading Documents

Workspaces have limits on the number of documents that can be uploaded to them, so creating a few extra workspaces will be beneficial in case a limit is reached during testing.

Generating Load

With Oracle’s acknowledgement that ADF technologies have session related resource issues, the traditional performance test script methodology of logging on once and rolling through a series of transactions to determine response times and system performance can cause issues which will extend the engagement.  Concentrate on identifying and resolving issues introduced by code customization or general configuration issues.

Script configurations

The following are configuration types which can be helpful in performance testing Oracle WebCenter.  These running configuration avoid known issues with ADF technologies and allow the focus of the investigation to shift to customization and configuration issues.  There are 3 base configurations that I have employed to work around this issue.

  • Random Iterations
  • Time Out
  • Login/Logout each iteration

Random Iterations

The runtime logic allows the application to login more times than once.  Controlling the number of times that a login or a logout occurs can be done using whatever programmatic method the scripter desires.

Runtime Logic

  • Vuser_Init
    • Once Only business processes
    • Run
      • Login
      • Business Processes
      • Logout
      • Vuser_End
        • Wrap up once only business processes

Pacing

Complete start of iteration 1 to start of iteration 2 and has no relationship to the session timeout, as users will be closing out their opened sessions.  Please note that when a user logs out of WebCenter there is an anonymous session that is kicked off and continues through.

Random Iteration Code additions

MAXLOOPS is defined in globals.h as a positive integer, and it represents the maximum number of iterations that any VU will run before logging off/on.

INITIALIZELOOPS is defined in globals.h as 0.

    //            vuser_init
    nMaxLoops = time(NULL) % MAXLOOPS;
    nMaxLoops++;
    nIterationCount = nMaxLoops;
    //            Login - Decider
    if(nIterationCount >= nMaxLoops) {
        nIterationCount = INITIALIZELOOPS;
    } else {
        nIterationCount++;
        lr_exit(LR_EXIT_ACTION_AND_CONTINUE, LR_AUTO);
    }
    //            Logout
    if(nIterationCount < nMaxLoops)
    lr_exit(LR_EXIT_ACTION_AND_CONTINUE, LR_AUTO);

Timed Out Users

Each time the user logs in, the user performs a limited number of transactions and then the user remains dormant until they are timed out.

Runtime Logic

  • Vuser_Init
    • Once Only business processes
    • Run
      • Login
      • Business Processes
      • Vuser_End
        • Logout not necessary

Pacing

Implemented at the end of the business process to be slightly longer than the amount of time before the system times out, forcing the application to timeout and therefore clean up the sessions.  Each time the user starts up the user logs into the system.

Login Each Iteration

This mechanism contains a flaw of over taxing the system in the creation of sessions.  Session creation is always a costly operation, so be careful when employing this type of script behavior.

Runtime Logic

  • Vuser_Init
    • Once Only business processes
    • Run
      • Login
      • Business Processes
      • Logout
      • Vuser_End
        • Wrap up once only business processes

Utilizing these different configurations of virtual users and mixing these different types of Vuser groups causes difficulty in calculating complete business processes per hour.   Be careful when making promises about transaction levels.  So be careful in your deployment of the virtual users in order to achieve desired levels of activity.  This is quite costly from a session overhead perspective.

Tuning this application is quite difficult as there are so many moving parts.  If the configuration of the system under test is not configured along the same method as is recommended by F5, then any imbalances shown in monitoring UCM, SOA or the other JVMs running need to be investigated.  Do not be surprised if these investigations lead you to the F5.

Leave a Reply