*** Adaptive Load scenario is not currently available. We will soon publish an alternative ***

Every good test plan should include testing your application under different load scenarios such as: gradual load, stress, endurance, etc.

However before applying any load scenario it is a good practice to follow two very important steps:

• Define your performance expectations
• Identify your application’s performance limits

If these steps are skipped, we will run the risk of under or over testing our application.

Adaptive load scenario adjusts the load level according to your application’s performance in order to identify its limits.

One way to identify our application’s performance limits is by trying out numerous load levels to identify the appropriate load level that our application can support with an acceptable end-user experience. This process can require a few iterations and is not always accurate. To simplify the process of identifying performance limitations, we have recently introduced the adaptive load scenario . This scenario adapts the load level to the performance level of the application under test. The adaptation is made according to an acceptable service level that is defined prior to the test.

The adaptive load scenario decreases the load once errors are encountered or when the response time becomes unacceptable. The unacceptable response time is defined prior to the test.

The best way to utilize such a scenario is to use a larger load capacity. During the test, if the limits are reached the test will adapt and the limits will be identified.

Don’t be surprised if the identified limits are lower than expected. This will give you the chance to fine tune your application and test it again in order to make sure that the application can in fact support the required performance service level.

To illustrate the power of the adaptive load scenario , I will compare two test sessions ran against a website which is running on a 512MB server from Rackspace . For the two tests I used a simple business process which contains a short user sequence. In our scenario a user opens a browser, downloads a large file (4MB), waits for 10 seconds then loads a small HTML file (25KB) and then stops.

The first test will include a gradual load scenario. This scenario includes a gradual user ramp up from 0 to up to 100 concurrent users. The gradual load does not stop in case of errors or in case a response time is higher than defined. The second test will include an adaptive load scenario .

Gradual Load Scenario Results

During this test we ran the gradual load scenario which is not adaptive. Looking at the results of this test, it is clear that up to a level of 20 simultaneous users the response time presented an almost flat line. From that point, the response time began to increase. The load continued to grow and so did the response time. Although this test allowed us to identify the cause of the poor performance level (this issue was discussed in the post: Benchmarking Amazon Web Services Vs Rackspace ), it provided less intelligence regarding the performance limits. One could obviously guess that the limits are somewhere around the 20-30 simultaneous users, but a guess always provides less confidence.

Adaptive Load Scenario Results

For the adaptive load scenario , we defined 30 seconds as the unacceptable response time. This would mean that if a response time of over 30 seconds is met, the load level is reduced by one simultaneous user. An error, if encountered,would also decrease the load level by one simultaneous user.

Looking at the results of the second test, it is clear that the performance limit was reached and identified. Under the assertion that a response time of over 30 seconds is unacceptable, it is clear that the application can not support more than 20 simultaneous users.

When ever the load level was trying to go over it’s limit, it resulted in an unacceptable response time therefore decreasing the load level. Increasing the load level resulted again and again in an unacceptable response times.

Looking further into the reports reveals that out of 3,600 samples, 63 resulted in an unacceptable response time. The average response time was under 10 seconds.

The report on the right reveals the exact distribution of response time between responses.

Conclusions

The adaptive load scenario provides us with our application’s limits. It is very important to identify these limits as a first step prior to applying additional load scenarios.

Using the adaptive load scenario is only one alternative to identify these limits. Other scenarios can be used as well, however they are less accurate and may require multiple iterations.

To use the adaptive load scenario , simply use the PX lite interface. Add a test and choose the adaptive load scenario . Once you choose this scenario, a text box will appear asking you to choose the unacceptable response time, then save and start.

*** Adaptive Load scenario is not currently available. We will soon publish an alternative ***