Adaptive Load Scenario - How to Identify Performance Limits

*** Adaptive Load scenario is not currently available.
 Please use
Real-Time JMeter Properties Control ***

Every good test plan should include testing your application under different load scenarios such as: gradual load, stress, endurance, etc.

However before applying any load scenario it is a good practice to follow two very important steps:

  • Define your performance expectations
  • Identify your application’s performance limits

If these steps are skipped, we will run the risk of under or over testing our application.

Adaptive load scenario adjusts the load level according to your application’s performance in order to identify its limits.

One way to identify our application’s performance limits is by trying out numerous load levels to identify the appropriate load level that our application can support with an acceptable end-user experience. This process can require a few iterations and is not always accurate. To simplify the process of identifying performance limitations, we have recently introduced the adaptive load scenario . This scenario adapts the load level to the performance level of the application under test. The adaptation is made according to an acceptable service level that is defined prior to the test.

The adaptive load scenario decreases the load once errors are encountered or when the response time becomes unacceptable. The unacceptable response time is defined prior to the test.

The best way to utilize such a scenario is to use a larger load capacity. During the test, if the limits are reached the test will adapt and the limits will be identified.

Don’t be surprised if the identified limits are lower than expected. This will give you the chance to fine tune your application and test it again in order to make sure that the application can in fact support the required performance service level.

To illustrate the power of the adaptive load scenario , I will compare two test sessions ran against a website which is running on a 512MB server from Rackspace . For the two tests I used a simple business process which contains a short user sequence. In our scenario a user opens a browser, downloads a large file (4MB), waits for 10 seconds then loads a small HTML file (25KB) and then stops.

The first test will include a gradual load scenario. This scenario includes a gradual user ramp up from 0 to up to 100 concurrent users. The gradual load does not stop in case of errors or in case a response time is higher than defined. The second test will include an adaptive load scenario .

Gradual Load Scenario Results

During this test we ran the gradual load scenario which is not adaptive. Looking at the results of this test, it is clear that up to a level of 20 simultaneous users the response time presented an almost flat line. From that point, the response time began to increase. The load continued to grow and so did the response time. Although this test allowed us to identify the cause of the poor performance level (this issue was discussed in the post: Benchmarking Amazon Web Services Vs Rackspace ), it provided less intelligence regarding the performance limits. One could obviously guess that the limits are somewhere around the 20-30 simultaneous users, but a guess always provides less confidence.

Adaptive Load Scenario Results

For the adaptive load scenario , we defined 30 seconds as the unacceptable response time. This would mean that if a response time of over 30 seconds is met, the load level is reduced by one simultaneous user. An error, if encountered,would also decrease the load level by one simultaneous user.

Looking at the results of the second test, it is clear that the performance limit was reached and identified. Under the assertion that a response time of over 30 seconds is unacceptable, it is clear that the application can not support more than 20 simultaneous users.

When ever the load level was trying to go over it’s limit, it resulted in an unacceptable response time therefore decreasing the load level. Increasing the load level resulted again and again in an unacceptable response times.

Looking further into the reports reveals that out of 3,600 samples, 63 resulted in an unacceptable response time. The average response time was under 10 seconds.

The report on the right reveals the exact distribution of response time between responses.

Conclusions

The adaptive load scenario provides us with our application’s limits. It is very important to identify these limits as a first step prior to applying additional load scenarios.

Using the adaptive load scenario is only one alternative to identify these limits. Other scenarios can be used as well, however they are less accurate and may require multiple iterations.

To use the adaptive load scenario , simply use the PX lite interface. Add a test and choose the adaptive load scenario . Once you choose this scenario, a text box will appear asking you to choose the unacceptable response time, then save and start.



*** Adaptive Load scenario is not currently available.
Please use 
Real-Time JMeter Properties Control ***


BlazeMeter

provides a performance testing solution that's 100% compatible with Apache JMeter™

BlazeMeter Tutorials

  1. 200 MPH in neutral - Cloud-testing compared to traditional testing within the corporate LAN
  2. Analyze the Results of a Load Test Using BlazeMeter
  3. Real Time JMeter Properties Control with BlazeMeter
  4. BlazeMeter Formatters: Throughput
  5. Read Only Access to your Test Dashboard in Real-time
  6. Test X-Large Capacity Facebook Applications
  7. Dynamic Master/Slave Tests
  8. Load Test WordPress Sites
  9. Excluding Certain Domains from the Load Test
  10. Adaptive Load Scenario - How to Identify Performance Limits
  11. Folder Subscription
  12. Customize JTL Properties
  13. Download files from the Files tab
  14. How BlazeMeter simplifies maintaining JMeter scripts
  15. How can I download my site report?
  16. How can I make my report more detailed and present also embedded resources statistics
  17. How do I balance the "number of simulated users" on a test server?
  18. Using MSSQL JDBC Driver
  19. How to override basic JMeter properties from the test dashboard?
  20. How to setup a different CSV file for each JMeter engine?
  21. How to simulate .htaccess login
  22. Ignore Requests in the Reports
  23. IP spoofing / Hosts file modification
  24. JTL files and Real Time Reports
  25. Leveraging Cloud Computing for Load Testing of Web Applications
  26. Monitoring the JMeterEngine(s) Vital Signs
  27. Post Test Actions
  28. Presenting JTL files in BlazeMeter
  29. Recording HTTP traffic in the Cloud
  30. Running a SOAK test
  31. Select the Number of JMeter Engines
  32. Set JMeter Engine Command Line Arguments
  33. Set Load Geographical Origin
  34. Set the Console Command Line Arguments
  35. Set the JMeter Version
  36. Simulating logged in session using BlazeMeter
  37. [Master/Slave] Synchronize and Aggregate the Results from Several Test Sessions into a Single Aggregated Test
  38. Testing Amazon ELBs
  39. The Load Reports
  40. Upload Files
  41. Upload Proprietary JAR file
  42. Upload user.properties, system.properties and jmeter.properties Files
  43. Upload your JMeter Script
  44. Use Dedicated IP Addresses
  45. BlazeMeter Delayed Load Start
  46. Using Concurrent Pool Size - JMeter 2.5+
  47. Using CSV files with BlazeMeter
  48. Using Include Controller
  49. Using non-standard port. Unable to verify my site. Any workaround?
  50. Using Oracle JDBC Driver
  51. Using the Auto Scripting Feature with BlazeMeter
  52. Viewing Reports in GUI Mode
  53. SSL Manager and JKS files
  54. Using MySQL JDBC Driver
  55. Set the Time Zone
  56. How to Upload Many Files at Once
  57. Load Testing with HTTP Basic Authentication
  58. Input field and form submission
  59. Tests can end prematurely using JMeter 2.9
  60. Common File Storage
  61. Using BlazeMeter's Jenkins Plugin Behind a Corporate Proxy

Feedback and Knowledge Base