UI automation testing

The first thing a user sees about your software product is the interface. Testing the user interface (UI) is critical and you need to be confident that all UI elements work as expected and look as you intended under any circumstance.

There are a lot of technical and human aspects that influence the speed, visibility, and flexibility of web services’ layout and as such, all should be verified.

There are two solutions to ensure everything you need on the UI side: manual and automation testing.

While not dead yet, manuals can be costly and the results don’t always match the price. In this piece, we will focus on the second one, automation testing.

It’s important to analyze different scenarios under different conditions across various browsers, dimensions, devices, and so on. Automation testing helps pick up the pace, mitigate human errors in the process, port elements, save money and time, etc. While some consider automation testing strenuous, highly-technical work, trust us, it is all worth it.

Let’s dive in!

UI applications testing approaches

There are a lot of different approaches for testing UI applications but two come to mind as the more common ones: record-and-playback testing and model-based testing. Let’s explore each one.

Record-and-playback testing

Record-and-playback testing is based on the way an engineer records all their interactions with a web application and then repeats the sequence as a kind of script. Scripts are played again and again without human involvement and it comes in useful for simple web applications with a small number of features and functionality.

This approach doesn’t come without its drawbacks. The most unpleasant pitfall is that this practice needs high managing capacity. As expected, scripts will fail even in cases like primitive updates, the relocation of UI elements, or new IDs or classes. Sometimes, it can be simpler to check the fault manually than to rewrite an entire script. Of course, this depends on the tools you use as many offer easier maintenance and troubleshooting. Another drawback is that this approach to testing takes quite a significant amount of memory, which can translate into a high-priced solution for you. 

Model-based testing

Model-based testing is a solution best suited for scenarios where engineers create a model with the application’s description and builds the whole testing process around this model and behavior.

One of the biggest drawbacks of this approach is that it required deep knowledge and experience in automation and coding, and sometimes it can take a significant amount of time to write a testing framework (not only to record a script). It can become increasingly challenging for non-tech users to read and understand the flow. These issues can be solved with up-to-date testing “weapons”. Overall, this approach has more pros than the record-and-play testing approach.

Model-based testing can help save money and time; it can also be easily adapted for several environments, machines, browsers, and more. It doesn’t require large investments like the record-and-play testing approach when it comes to covering new features. Last but not least, model-based testing is easy to catch bugs and debug. 

Tools

Selenium IDE

It is mostly used to record and playback cases with limited amounts of data, which need only one window. It can be used as a health-check for UI elements [buttons, labels, text, etc.] and responses for user actions [clicking, entering data, typing, etc.]. Unfortunately, it can’t test more complex cases like connection to a database, dynamic-generated data or elements, and other, more challenging scenarios.

Watir Webdriver (+ rspec)

Watir (Web Application Testing In Ruby) is an open-source Ruby library that can be used for UI automation testing as a way of simulating user actions. It tests existing elements and availability by clicking on links and other user-like actions. In addition, it has some powerful features like the ability to take screenshots, headless testing, and response time measuring and performance, to name a few.
Rspec is also a Ruby testing library that can be used in a behavior-driven development (BDD).

Example

require 'rspec'
require 'watir-webdriver'

browser = Watir::Browser.new

describe "check the main page" do
  before(:each) do
    @browser.goto("http://main.page/")
  end


  context "check no error" do
    it "should not return an 400 error" do
      @browser.text.should_not include('Bad request')
    end
    it "should not return an 401 error" do
      @browser.text.should_not include('Not authorised’')
    end
   end

  context "page include main elements" do 
    it "check input field" do
      @browser.text_field({name: "input"}).set 'MyName'
      @browser.form({name: "submit name"}).submit
      Sleep 4
      expect(@browser.title).to eq(“Your Name Form")
    end
   end
end

Cucumber

Cucumber is widely used for BDD and its main feature is that it offers an easy to read format. Test cases can be described in a recognizable way using the English language. This tool is common among engineers and non-tech specialists. 

It uses the Gherkin language which has five main keywords: Scenario, GIven, WHen, Then, And. These keywords describe the main case you want to test, start a condition, the user’s actions, the result, and addition for any of the three operators. 

Example

Feature: Checking main page

  Scenario: Positive case
    Given main page is opened 
    When I enter the name “John” 
    And I click on the “Submit” button
    I should see the text “Your name is John”

Jasmine

Jasmine is an open-source testing framework for JavaScript. Considered by its creators as a BDD tool, it can also be used for TDD and unit testing. It has an alternative form of library as a “Protractor.” 

Jasmine is mostly employed for unit testing, but it can be effective for UI automation with the help of additional libs and drivers (Selenium driver, jasmine-jquery).

Example

const {Builder, By, Key, until} = require('selenium-webdriver');

require('chromedriver');
const driver = new Builder()
    .forBrowser('chrome')
    .build();
var login = async function() {
    let loginEl = By.css('.some-login');
    await driver.wait(until.elementLocated(loginEl), 10 * 1000);
    console.log('Login done')
}
describe('base page, function () {
    beforeEach(async function() {
        await login();
    });
    afterAll(async function() {
       console.log('Test done')
    });

    var urlAddress = '/some-jasmine-uiSpec';
    describeUi("base page", fixtureAddress, function () {
        function currentBaseUrl() {
            return window.location.pathname;
        }
        it("URL is correct", function () {
            expect(currentBaseUrl()).toBe(urlAddress);
        });

    describeUi("base page with hash", fixtureAddress+’#098’, function() {
        it("has is in url", function() {
            expect(location.hash).toBe('#098');
        });
    });
});

Postman

While not a full-fledged UI or automation testing tool, Postman is useful in its unique way. Web pages are the bridge between the server and the user. With that in mind, Postman checks API requests to figure out what’s wrong either on the client or server side, it verifies response codes, it tests time responses, and much more.

Cypress

Cypress is a JavaScript tool designed for UI testing. It’s easy for beginners and it’s also suitable for E2E testing. Cypress offers many features for engineers like automatic waiting without having to insert it, easy debugging via well-known tools, snapshots for each part that was run, stubs, and much more.

Example

describe('Post Name, () => {
  it('Open a page', () => {
    cy.visit('/main-page')     
    cy.get('input.name') 
      .type(‘My name is John’)  

    cy.contains('Submit Name')      
      .click()                 
    cy.get('#main-content')               
      .should('contain', 'My name is John')
  })
})

Appium

Once upon a time, if you ever needed to automate UI testing for your mobile app, chances are you were using Appium. This widespread instrument is used to test iOS or Android and it allows communication with both mobile webpages (with Chrome or other browsers) and Chrome-backed apps. It delegates commands to the Chromedriver and its tests can be executed both with real devices and with emulators. 

Example

// setup classes which will describe app path and so on
 
def testEmailInput(self):
        email = self.driver.find_element_by_accessibility_id('emailTextField')
        email.send_keys("my@gmail.com")
        email.send_keys(Keys.RETURN)
        sleep(2)
        self.assertEqual(email.get_attribute("value"), "my@gmail.com")

Robot Framework

Robot Framework is similar to Cucumber. It allows you to describe acceptance criteria and test cases with the help of high-level language which is understandable for everyone. This tool is Python-based, but for the full-fledged UI testing, it can be expanded with additional libraries that are using other programming languages. 

Example

*** Settings ***
Library  SeleniumLibrary
Suite Setup     Open browser    ${URL}   ${BROWSER}
Suite Teardown  Close All Browsers

*** Variables ***
${URL}          http://some-url.com/
${BROWSER}      Chrome

*** Test Cases ***
The user can save his name
    [Tags]       search_flights
    Find Input element xpath://select[@name='inputName']  John
    Click Button    css:input[type='submitName']
    @{name}=  Get WebElements    css:table[class='table']>tbody tr
    Should Not Be Empty     ${name}

// also it includes Python classes which describe WebDriver session

UI testing pitfalls

During UI testing, engineers can face painful pitfalls and headaches. Let’s examine some of them along with some ideas to solve them.

Long-running time

• The most common approach to go over this point is to run tests in parallel or to use more than one virtual environment. 
• You could group tests based on preconditions and run in scopes where it’s possible without recreating the environment.
• Execute happy/short path first. You don't need to test complicated and long-run scope if anything failed since the beginning.
• Run it as nightly execution.
• Check several basic E2E ways at the beginning.
• Allow to rerun suites in the pipeline without restarting an entire suite.
• Move some parts on the backend side. For example, if you are creating new users for each feature, you can verify it on the UI side and move to the backend in the next cycle.

Unstable/dynamic elements

• Use a type of “updates encapsulated” environment for automation. Or, don’t update the environment during test running. 
• Change waiters in seconds to waiters “element.when_visible”
• Use retries in reasonable quantities to check if elements are present.
• Take a screenshot for misstep research.

UI changes are missed by tests

• You can use tools [or write it by yourself] to compare screenshots from main pages, tabs, points, etcetera. If there are any changes that weren’t caught by the tests, the correlation will return a failure. 

Conclusion

The choice to use an automation tool is complex. You need to take into account several factors like human resources, technical skills, required patterns and the application’s behavior, portability, independence components, input data, and conditions, resulting view, and reporting structure to name a few. Fortunately, each of these factors can be covered by most modern tools.

With a thought-out, well-balanced decision to implement your testing solutions, you can achieve your QA strategy and take it to a higher level.