Website automation with Selenium and JMeter: part 1

/ DevOps / By / Leave a Comment / Automation, JMeter, Selenium, Web

This post is part 1 of a series:

One of our customers asked us to troubleshoot their web application: “out of memory” errors were causing page loads to fail. To understand and isolate the problem we needed a way to reproduce it, and observe it in a monitored scenario. We decided on a strategy for both server and web user:

  • Server: deploy a build that monitors for memory leaks, a first-pass step to diagnose the out of memory errors.
  • Client: simulate web requests from one or more users.

It was straightforward to monitor the server application for memory leaks. On the client side we needed a way to generate repeating user requests, potentially at high volume, because the errors didn’t occur for every request. This post describes how we built a client-side process to automate web requests.

Tools

Selenium WebDriver automates the web browser. Apache JMeter is for load testing. These tools solve quite different problems. It’s unusual to combine them but for our troubleshooting scenario, doing so made it easy to build a client-side simulator quickly.

We used WebDriver to script a web browser for automating a sequence of user interactions, like typing text or clicking a button/hyperlink. We also set up JMeter to run the script on multiple threads simulating concurrent users, repeating until the error occurred.

Selenium is a popular suite of tools for automating web browsers. These tools include:

  • Selenium IDE, a web browser plugin that records a sequence of user interactions and produces a script for playback. This is useful for testing in different scenarios, or for automating repetitive browser-based tasks.
  • Selenium WebDriver, a framework primarily for building automated regression tests. It’s designed for flexibility with an architecture that:
    • Provides a programmable API that supports several languages.
    • Automates supported browsers via browser-specific software drivers.

ChromeDriver is a WebDriver-compatible software driver for the Google Chrome browser. Drivers are available for other browsers.

JMeter is a tool for load and performance testing, primarily for network resources like web services. Its architecture is based on plugins. A JMeter script is composed of elements for configuration, flow control, sampling, and other elements.

Selenium/WebDriver Support is a JMeter plugin that integrates WebDriver and JMeter. It bundles a number of JMeter elements, including a sampler for WebDriver, and configuration elements for various browser-specific drivers. WebDriver Sampler wraps WebDriver script inside a JMeter script. We scripted the WebDriver API using JavaScript; several other languages are supported.

 

Example Browser Automation

I can’t show details of our customer’s software, but the example below demonstrates similar techniques for handling subtleties that only become apparent when automating a web browser. For example, an HTML page element can be clicked only if it’s visible. It may be invisible because it belongs to part of the page that hasn’t yet loaded, or because the element is hidden by a progressive disclosure element.

My example website is the OpenShift web console, a browser interface for viewing/managing OpenShift projects, applications, and resources. In the screenshot below:

  • The logged-in OpenShift user is developer (top-right);
  • The selected project is my-project (top-left);
  • The Builds view is active;
  • The detail area shows build #1 for deployment nodejs-centos-ex; and
  • The Logs tab is active with the full build log displayed.

 

 

To build the example I did the following:

  1. Use Minishift to quickly provision an OpenShift cluster in a virtual machine;
  2. Use the OpenShift web console to create a Node.js application running in a CentOS container;
  3. Write a script for WebDriver + JMeter that automates login to the web console and simulates mouse clicks to view the application deployment and build log.

The short screencast below shows the automation in action. I modified the script for this, inserting several artificial delays to slow down the automation that’s normally too fast for a human viewer.

 

 

JMeter Script

This relatively simple script demonstrates some nice features of WebDriver and JMeter. The full source code is available on GitHub and part 2 of this article explains how to install the tools.

JMeter properties provide a way to add flexibility. We can define properties in a script for values like numbers and strings. Properties can have default values. Then at the time of executing the script we supply custom values to override the defaults.

The syntax  ${__P(k,v)}  instructs JMeter to apply its built-in function  __P  (shorthand for __property) to get the value of property  k  whose default value is  v. For example, JMeter replaces the expression  ${__P(foo,bar)}  by the value  bar  if a custom value for  foo  was not supplied to the JMeter runner (see below).

I created the script as described in the steps below.

  1. Start the JMeter IDE: in the JMeter bin sub-directory, run either jmeter.bat (Windows) or jmeter.sh (Linux). By default the IDE starts with an empty test plan i.e. only root node Test Plan in the tree view on the left.
  2. Add a thread group for setting the number of concurrent users and iterations (loops): right-click Test Plan, choose Add > Threads (Users) > Thread Group. Define parameters for user count and loop count with 1 as the default value for each. JMeter command-line arguments can override these defaults.
    1. In the text box at Number of Threads (users), enter ${__P(users,1)}
    2. In the text box at Loop Count, enter ${__P(loops,1)}

  3. Add an element for user-defined variables (UDVs): right-click Thread Group, choose Add > Config Element > User Defined Variables. Define user-defined variables (see below); the values will be supplied by JMeter command-line arguments.
    1. Click Add (at the bottom) to add a table row: in the Name column, enter URL, in the Value column enter ${__P(url,)}
    2. Add a row: in the Name column, enter USERNAME, in the Value column enter ${__P(username,)}
    3. Add a row: in the Name column, enter PASSWORD, in the Value column enter ${__P(password,)}
  4. Add an element for configuring Chrome Driver: right-click Thread Group, choose Add > Config Element > jp@gc – Chrome Driver Config. In the text box at Path to Chrome Driver, enter the full path to the installed ChromeDriver binary.
  5. Add an element for scripting WebDriver: right-click Thread Group, choose Add > Sampler > jp@gc – WebDriver Sampler. More on this below, in section WebDriver Sampler JavaScript.
  6. Add an element to capture HTTP requests, responses and errors: right-click Thread Group, choose Add > Listener > View Results Tree.
    This element is commonly used in load testing, to show statistics like average response time. Here it was useful while writing the script, for quick feedback while testing interactively; click one of the “play” buttons (green triangle) to execute.

 

Executing the Script

Step 3 above defines user-defined variables as parameters without default values, e.g. parameter url here: URL=${__P(url,)}. The script code assumes the values are supplied during execution:

 

var url      = WDS.vars.get('URL')
var username = WDS.vars.get('USERNAME')
var password = WDS.vars.get('PASSWORD')

 

Generally the JMeter command line interface (CLI) is used when testing, whereas the IDE shown above is good for composing script elements. Below is an example Windows command session to execute a script:

 

> set JMETER_HOME=C:\Program Files\Apache Software Foundation\apache-jmeter-4.0
> C:\Program Files\Apache Software Foundation\apache-jmeter-4.0\bin\jmeter.bat -n -t nodejs-centos-ex.jmx
  -Jurl=https://192.168.3.134:8443/console/ -Jusername=developer -Jpassword=developer

 

The CLI arguments used here are explained below:

  • -n
    Run JMeter in non-GUI mode
  • -t nodejs-centos-ex.jmx
    Specifies the JMeter script file to execute
  • -j test.log
    Specifies an output file for the JMeter log
  • -Jusername=developer
    Defines a parameter username with value “developer”

Note: if the command omits the -j option (log file), JMeter writes to its default log file in the installation directory.

 

Fail Fast, Test Fast

By combining JMeter and WebDriver we were able to rapidly build an automation process to simulate one or more web users. Our objective was to induce server activity in a repeatable way, in order to both diagnose the problem and test the solution. We used the automation to:

  1. Reproduce the server errors;
  2. Confirm a trend of increasing memory usage;
  3. Confirm steady (non-increasing) memory usage after the memory leak was fixed.

We may have been able to do this without WebDriver, because we weren’t testing browser-specific behaviour like layout or image rendering. An alternative automation script could use JMeter’s HTTP Request sampler to generate web server requests: ultimately our goal was to “test” the server, not the browser. With such a script a single laptop could simulate hundreds of concurrent users. On the other hand, it’s more time-consuming to get the sequence of HTTP requests right, even with the help of a recorder tool to capture the requests. With the WebDriver Sampler, a machine with four CPU cores can support three concurrent users (JMeter itself requires one core). But 2-3 concurrent users were enough, to induce the problem and test the solution.

That’s all for this post. In the next part I’ll show actual script code to handle asynchronous browser behaviour, and how to install the tools.

 

Credits

Featured image:
https://westergaard.eu/wp-content/uploads/2017/07/edorasware-illustration-memory-leak-horizontal.jpg

Leave a Comment

Scroll to Top