Vipul Gupta's Blog

How To Automate Using TestNG In Selenium? [TestNG Tutorial]

How To Automate Using TestNG In Selenium? [TestNG Tutorial]

Vipul Gupta's photo
Vipul Gupta
·

28 min read

Table of contents

Automation testing is a fast-growing industry, and every tester tends to opt for tools and frameworks that are self-sufficient and offer useful features out of the box. Though there are a number of test automation frameworks like Selenium, Cypress, etc; I still prefer using Selenium.

The reason for choosing Selenium is very simple — Selenium supports the Java programming language.

The features offered by Selenium multiply when it is used in conjunction with Java-based test automation frameworks like TestNG, JUnit, etc. An ideal framework in such a case would be robust enough and complement the advantages of Selenium WebDriver & Java.

Of all the candidates available to serve these needs, the one that stands out is TestNG. TestNG is not only easy to implement and learn but also supports reporting of execution and logs. It makes parallel testing such an easy task! Ask any Selenium TestNG user to validate this claim, a majority of them will answer in affirmation. As per the State of open source testing survey, TestNG is one of the best Java testing frameworks preferred by developers. Selenium is the only test automation framework that is ahead of TestNG. Close to 50% of the organizations prefer the TestNG framework.

TestNG, where NG stands for Next Generation, is a testing framework developed by Cedric Beust. In this Selenium TestNG tutorial, I will deep dive into how to install TestNG and cover the essentials on writing your first automation script with TestNG for Selenium automation testing.

What is TestNG?

TestNG is an open-source advanced testing framework built on the capabilities of JUnit and NUnit. The framework is not only robust but also provides awesome features like TestNG annotations, grouping, parameterizations, etc., that help keep tests well-structured and easy to maintain.

At the time of writing this blog, the latest version of TestNG is 7.5. However, the preferred version is 6.14.3. I will be using the said version for demonstrating the features of TestNG Selenium in this Selenium TestNG tutorial.

The TestNG framework is designed to simplify a broad range of testing needs, from unit testing (testing a class in isolation of the others) to integration testing (testing entire systems made of several classes, several packages and even several external frameworks, such as application servers).

Salient features of TestNG:

  • Provides HTML Reports in an efficient easy to read format, thus catering to WebDriver’s limitation to generate such reports

  • Support to Group multiple test cases to be executed together whenever that particular group is invoked

  • Ability to set Priority among test cases which help to decide which case has to be executed first

  • Provides the ability to execute only the failed cases with the help of testng-failed.xml

  • Support for Cross Browser testing, Data Parameterization, and Parallel testing

  • Easy to understand Annotations, which helps control the sequence of execution in automation script without a static main method

  • Handles Uncaught Exceptions inherently, thus preventing sudden test termination.

This Cypress test automation tutorial will help you learn the benefits of Cypress automation, and how to install Cypress.

Installation and Setup of TestNG in Selenium

Having understood the features of TestNG, the next would be to know how to set it up on the system before I deep dive into the practical demonstration. In this tutorial on TestNG in Selenium, I will cover the installation process for 2 of the most used IDEs -Eclipse and IntelliJ.

Watch this video to learn how TestNG has become one of the most robust test automation frameworks and all you need to know to get started with TestNG in Selenium.

https://youtu.be/90F7dePW1vo

How to install TestNG in Eclipse

To install TestNG in Eclipse, download and install Eclipse IDE on your system.

Step 1: Launch Eclipse, click on Help, and select Eclipse Marketplace.

Step 2: Inside the Eclipse Marketplace window, go to the Search tab and find TestNG. If it is already installed, you will see the Installed button, and no need to continue with these steps; otherwise, it would say Install. Click Install in such a case.

Step 3: On the consecutive window, make sure the TestNG checkbox is checked and click Confirm.

Step 4: It is recommended to restart Eclipse after the installation is completed for changes to take effect correctly.

Step 5: After restarting, verify the installation by right-clicking on any project and checking if the TestNG menu is visible to create a TestNG class or not.

How to install TestNG in IntelliJ

In this section of the blog on TestNG in Selenium, I will cover how to install TestNG in IntelliJ. For IntelliJ, we will run tests on Java where we will download the JAR from the Maven Repositories website and add it.

But before proceeding to add the JAR, make sure you already have IntelliJ installed on the system. If not, you can download IntelliJ by navigating to the JetBrains website.

Step 1: Visit Maven Repositories, search for TestNG and select the latest most used version (i.e. 7.1.0). In this tutorial, I will be using TestNG version 6.14.3. Hence, the POM file that I will be using for demonstration will be having traces of TestNG 6.14.3.

Step 2: Select and download the jar file. We will be adding this jar to IntelliJ.

Step 3: Launch IntelliJ, open the File menu, and select Project Structure.

Step 4: Open the Modules panel, go to the Dependencies tab, click on + sign, and select JARs or directories.

Step 5: Navigate to the path where we have the downloaded jar from previous steps and select it.

Step 6: It would appear on the list of dependencies. Just check it and click the Apply button followed by the OK button.

And with this, we are done with installation on both IntelliJ and Eclipse.

This Playwright automation testing tutorial will guide you through the setup of the Playwright framework, which will enable you to write end-to-end tests for your future projects.

Project Setup in Eclipse

For this TestNG in Selenium tutorial, we would be going ahead with Eclipse IDE. In this section, we will learn how to create a TestNG project in Eclipse. However, if you are more comfortable using IntelliJ, then please go ahead with it. The steps mentioned would be the same irrespective of the IDE used. So let us create the project structure and add a test class to verify if it is being executed as TestNG Run or not.

In this entire tutorial, we will be using the same project structure as discussed above and add a new class file each time we want to practice something by executing and analyzing the output.

  • Create a Maven project in Eclipse and name it testng_selenium.

  • Update the pom.xml to have Selenium and TestNG dependencies.

POM, short for Project Object model, is an XML file that makes the basic fundamental unit of Maven. It stores the information about the project like version, description, etc., and also contains configuration details like dependencies, which Maven uses to build the project and make our scripts work.

<project xmlns="[http://maven.apache.org/POM/4.0.0](http://maven.apache.org/POM/4.0.0)" xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance](http://www.w3.org/2001/XMLSchema-instance)" xsi:schemaLocation="[http://maven.apache.org/POM/4.0.0](http://maven.apache.org/POM/4.0.0) [https://maven.apache.org/xsd/maven-4.0.0.xsd](https://maven.apache.org/xsd/maven-4.0.0.xsd)">
  <modelVersion>4.0.0</modelVersion>
  <groupId>testng_selenium</groupId>
  <artifactId>testng_selenium</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <build>
    <sourceDirectory>src</sourceDirectory>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.1</version>
        <configuration>
          <release>16</release>
        </configuration>
      </plugin>
    </plugins>
  </build>
   <dependencies>
   <dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>3.141.59</version>
   </dependency>
   <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.14.3</version>
            <scope>test</scope>
        </dependency>
  </dependencies>
</project>

Step 1: Create a package and name it test. This package will contain all our test class files.

Step 2: Inside this package, add a new class VerifySetup, add the following code, and save it.

package test;

import org.testng.annotations.Test;

public class VerifySetup {

[@Test](http://twitter.com/Test)
 public void verifySetup()
 {
  System.out.println("TestNG setup is working fine");
 }
}

Step 3: Now execute this as a TestNG test to verify the setup. For this select the class name, right-click on it, select Run As and then click on TestNG Test.

Upon successful execution, the output should look like the below. This verifies our TestNG setup and the same step can be done to execute more classes that we add in this tutorial.

TestNG Annotations and Attributes

Annotation means a note or a comment in any document that is used to provide it certain meaning. TestNG annotations also serve a similar purpose. These are used to provide meaning to any function in the test script and describe its behavior.

Annotations in TestNG are some predefined keywords that help to control the execution of the automation script in a proper flow. One must note, any test which is not annotated in TestNG is ignored while running tests. So it is mandatory to add required annotations for the test cases. Here is one example to understand this.

package test;

import org.testng.annotations.Test;

public class UnAnnotatedTestExample {

[@Test](http://twitter.com/Test)
 public void testWithAnnotation() 
 {
  System.out.println("This test is annotated");
 }

public void testWithoutAnnotation() 
 {
  System.out.println("This test is not annotated");
 }
}

As you can see, in the above example, we have added two test cases, one with @Test annotation and one without it. So the expectation here is only the one with annotation is executed and another one is skipped. Let’s execute the class and verify this.

It is clear from the output, that only the case with annotation got executed. One more thing to note here is that the Test run on the output also shows the total number of test cases as 1 since TestNG reads the number of test cases from @Test annotation and we had only one such case in the file.

TestNG annotations also provide relevant information about the method, class, or test suite they are associated with. These are represented by adding @ prefix. Let us have a look at all these annotations one by one, which we will use in this Selenium TestNG tutorial.

Types of Annotations

TestNG currently supports ten types of annotations.

@Test

This is the most important annotation as it covers the actual test case logic and executes the automation run. There are some attributes associated with this annotation that helps to serve various use cases. We will learn these attributes in the following sections.

[@Test](http://twitter.com/Test)
public void testAnnotation()
{
//Write your test case logic here.
}

@BeforeTest

This annotation is executed before running any test with the @Test annotation in a class. One example of using this can be to maximize the browser for test cases in a class.

[@BeforeTest](http://twitter.com/BeforeTest)
public void browserSetup()
{
//Code to maximize the browser
}

@AfterTest

This annotation is executed after all the test cases in a class are executed. One use case of this annotation would be to write the method to compile a test run report after all tests are run.

[@AfterTest](http://twitter.com/AfterTest)
public void compileReport()
{
//code to compile a report
}

@BeforeMethod

This annotation is executed before any and every test method, i.e., before every @Test annotation method in the class. For example, we have cases to test some dashboard that comes up after login. So instead of writing login steps in each test case, @BeforeMethod annotation can be used to write those steps once in a method to be called before every test case.

[@BeforeMethod](http://twitter.com/BeforeMethod)
public void doLogin()
{
//code to do login
}

@AfterMethod

This annotation is executed after any and every test method, i.e., after every @Test annotation method in the class. In continuation to the above example, let’s say we need to log out after every test case. In such a scenario, this annotation would help.

[@AfterMethod](http://twitter.com/AfterMethod)
public void doLogout()
{
//code to do logout
}

@BeforeClass

This annotation is executed before the first test method with @Test annotation in the class is executed. For example, this can be used to navigate to the URL before executing the test.

[@BeforeClass](http://twitter.com/BeforeClass)
public void navigateUrl()
{
//code to navigate to URL
}

@AfterClass

This annotation is executed after the last @Test method is executed. The method with this annotation can be used to close the driver after the automation run.

[@AfterClass](http://twitter.com/AfterClass)
public void closeDriver()
{
//code to close the driver
}

@BeforeSuite

This annotation marks the entry point of any automation script execution on TestNG in Selenium. Any method with this annotation is executed before any test in all the classes in that suite. This can be used to do generic setup steps for the execution, like initializing WebDriver for execution.

[@BeforeSuite](http://twitter.com/BeforeSuite)
public void initializeDriver()
{
//code to initialize the driver
}

@AfterSuite

This annotation in TestNG is the last to be executed after all test methods of all the classes within the suite have been executed. A method with this annotation is mostly used for cleanup activities like closing the active driver sessions.

[@AfterSuite](http://twitter.com/AfterSuite)
public void cleanUp()
{
//code to close all sessions
}

@BeforeGroups

TestNG provides the capability to group the tests with similar functionalities in a group with the help of the group attribute of @Test annotation. This helps when we want to execute only a particular type of test case. The method with @BeforeGroup annotation is executed before the first test case for the given group is executed. We can mention the group name inside brackets next to the annotation.

[@BeforeGroups](http://twitter.com/BeforeGroups)("groupName")
public void startGroupExecution()
{
//code to execute before particular group test cases.
}

@AfterGroups

This annotation in TestNG is executed after all the test cases of a given group are executed.

[@AfterGroups](http://twitter.com/AfterGroups)("groupName")
public void stopGroupExecution()
{
//code to execute after particular group test cases.
}

With this, we have understood all the annotations used in TestNG. These annotations are highly usable as:

  • Very easy to learn as there are no rules on which annotation to use with which method. The tester can use them as per free will and understanding.

  • Ease of parametrization, which helps in creating groups and other tasks.

  • Strongly typed, which saves time by highlighting errors while implementing only.

  • Saves the time and effort to extend classes to define the order of execution, unlike JUnit

Watch this video to learn about the TestNG Annotations and how they help provide better structure and readability to the code.

https://youtu.be/zI7Xbaj4RIs

Hierarchy of Annotations

TestNG annotations have a predefined hierarchy of execution which helps in executing the source code. These are executed in the below order :

[@BeforeSuite](http://twitter.com/BeforeSuite) -> [@BeforeTest](http://twitter.com/BeforeTest) -> [@BeforeClass](http://twitter.com/BeforeClass) -> [@BeforeMethod](http://twitter.com/BeforeMethod) -> [@Test](http://twitter.com/Test) -> [@AfterMethod](http://twitter.com/AfterMethod) -> [@AfterClass](http://twitter.com/AfterClass) -> [@AfterTest](http://twitter.com/AfterTest) -> [@AfterSuite](http://twitter.com/AfterSuite)

Let us have a look at the following code example to demonstrate the hierarchy better.

TestNGAnnotationsHierarchy.java

package test;
import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;


public class TestNGAnnotationsHierarchy {
 [@Test](http://twitter.com/Test)
 public void testCase1() {
   System.out.println("This is Test Case 1"); 
 }

 [@Test](http://twitter.com/Test)
 public void testCase2() {
   System.out.println("This is Test Case 2"); 
 }

[@BeforeMethod](http://twitter.com/BeforeMethod)
 public void beforeMethod() {
   System.out.println("This will execute before every Test Method");
 }

 [@AfterMethod](http://twitter.com/AfterMethod)
 public void afterMethod() {
   System.out.println("This will execute after every Test Method");
 }

 [@BeforeClass](http://twitter.com/BeforeClass)
 public void beforeClass() {
   System.out.println("This will execute before the Class");
 }

 [@AfterClass](http://twitter.com/AfterClass)
 public void afterClass() {
   System.out.println("This will execute after the Class");
 }

 [@BeforeTest](http://twitter.com/BeforeTest)
 public void beforeTest() {
   System.out.println("This will execute before the First Test");
 }

 [@AfterTest](http://twitter.com/AfterTest)
 public void afterTest() {
   System.out.println("This will execute after the Last Test");
 }

 [@BeforeSuite](http://twitter.com/BeforeSuite)
 public void beforeSuite() {
   System.out.println("This will execute before the Test Suite");
 }

 [@AfterSuite](http://twitter.com/AfterSuite)
 public void afterSuite() {
   System.out.println("This will execute after the Test Suite");
 }
}

Upon execution, this is how the output for this code snippet would be where all the methods get executed as per the hierarchy of the annotations.

Looking at this code example and the output, it is evident that TestNG religiously follows the order of execution of annotations.

One might wonder when you see this example is that since every annotation has a particular sequence defined, then how do the two @Test methods in this example decide which one to execute when. This is something that is handled using test case priority in Selenium TestNG. The same will be covered in the upcoming sections of the blog.

TestNG Attributes

Just like a method in Java, TestNG annotations have attributes that help in making our tests more defined and provide more information about the test script while performing Selenium automation testing with TestNG. These attributes are:

description: It helps to define the purpose of the test

[@Test](http://twitter.com/Test)(description = "this is the definition of the test")

groups: This attribute helps to group test cases of the same functionality under a common group by giving it a group name. It is very useful when the user wants to execute cases of a particular group or wants to exclude cases of a particular group from execution.

[@Test](http://twitter.com/Test)(groups = "DashboardCases")

dependsOnMethods: The test having this attribute is only executed if the dependent test method passes. If it fails or is not executed, then the test with this attribute is skipped from execution.

[@Test](http://twitter.com/Test)(dependsOnMethods = "LoginFunctionality")

alwaysRun: It ensures that this test method always runs and is not dependent on the result of the methods that it depends upon.

[@Test](http://twitter.com/Test)(alwaysRun = true)

dataProvider: This attribute is used when we want to run a test case on a particular data set (e.g. running the same test case on different browsers). This attribute provides the data to the test case for which it is used with the help of another method annotated with @DataProvider annotation of TestNG.

[@Test](http://twitter.com/Test)(dataProvider = "CrossBrowserTesting")

  • We will learn more about DataProviders in a further section of this blog.

  • enabled: This attribute is helpful when we want to skip executing a particular test case. It helps to do so by setting the value to false.

    @Test(enabled = false)

timeOut: This attribute is used to define the max execution time for a test case. In other words, the test case should be completed in the given time, which is given in milliseconds, else the test execution would be terminated, and the case would be marked as failed with timeout exception (org.testng.internal.thread.ThreadTimeoutException)

[@Test](http://twitter.com/Test)(timeOut = 5000)

invocationCount: This attribute is used when we want to execute the test case in a loop for a given number of times.

[@Test](http://twitter.com/Test)(invocationCount = 3)

  • Would execute the test case 3 times

  • invocationTimeOut: This attribute is used in combination with the invocationCount attribute. This attribute defines the timeframe within which the test should execute for a given number of times.

 [@Test](http://twitter.com/Test)(invocationCount = 3, invocationTimeOut = 20)

  • Means execute the case 3 times in 20 milliseconds.

  • expectedExceptions: This attribute helps to handle the exceptions that the test method is expected to throw. If the test method throws the exception that is set on the attribute, then the test case passes, and for any other unhandled exception, the test method fails.

    [@Test](http://twitter.com/Test)(expectedExceptions = {ArithmeticException.class})

priority: This attribute is used to define the order of execution by assigning priorities to the test cases. A test method with lower priority is always executed first.

[@Test](http://twitter.com/Test)(priority=1), [@Test](http://twitter.com/Test)(priority=2)
  • In this case, the test with priority 1 is executed first, followed by the test with priority 2. Move to the next section to know more about priorities.

Consider the below code example to understand attributes implementation.

package test;

import org.testng.annotations.AfterGroups;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.Test;

public class TestNGAttributesExample {

[@Test](http://twitter.com/Test)(enabled = false, description = "This test will be skipped as enabled is set to false")
 public void disabledTest() {
  System.out.println("This method will be skipped from the test run using the attribute enabled=false");
 }

[@Test](http://twitter.com/Test)(invocationCount = 5, invocationTimeOut = 20, groups = "testngAttributes")
 public void invocationCountTest() {
  System.out.println("This method will be executed by 5 times");
 }

[@Test](http://twitter.com/Test)(timeOut = 500, groups = "testngAttributes")
 public void baseTest() {
  System.out.println("This is base test method");
 }

[@Test](http://twitter.com/Test)(dependsOnMethods = "baseTest", groups = "testngAttributes")
 public void firstDependentTest() {
  System.out.println("This is dependent method 1 and will only execute if baseTest passes");
 }

[@Test](http://twitter.com/Test)(alwaysRun = true, dependsOnMethods = "baseTest", groups = "testngAttributes")
 public void secondDependentTest() {
  System.out.println(
    "This is dependent method 2 and will execute irrespective of baseTest result because of alwaysRun");
 }

[@BeforeGroups](http://twitter.com/BeforeGroups)("testngAttributes")
 public void beforeGroupMethod() {
  System.out.println("This method is executed before testngAttributes group test cases");
 }

[@AfterGroups](http://twitter.com/AfterGroups)("testngAttributes")
 public void afterGroupMethod() {
  System.out.println("This method is executed after testngAttributes group test cases");
 }
}

Running the above code would give the following output:

TestNG Priorities

As mentioned before in this TestNG in Selenium tutorial, test case execution in TestNG is governed by the priority of test cases. But what if no priority is assigned to test cases. Then what decides the order of execution.

If there are multiple test cases, then TestNG executes them in alphabetical order. So for the following code, aTest is executed before bTest.

[@Test](http://twitter.com/Test)
public void aTest()
{}

[@Test](http://twitter.com/Test) 
public void bTest()
{}

Now, let us assume a scenario where we want to execute bTest first. In such scenarios, a priority attribute is used. We can achieve this by assigning lower priority to the test we want to execute first. Priorities in TestNG can be assigned starting from 0, where lower priority cases are executed first.

[@Test](http://twitter.com/Test)(priority = 2)
public void aTest()
{}

[@Test](http://twitter.com/Test)(priority = 1)
public void bTest()
{}

In this case, bTest will be executed before aTest.

The next question that comes up is, what if the same priority is assigned to both test cases.

[@Test](http://twitter.com/Test)(priority = 1)
public void aTest()
{}

[@Test](http://twitter.com/Test)(priority = 1)
public void bTest()
{}

In such a case, TestNG runs the cases in alphabetical order, i.e., aTest will be executed before bTest.

Another scenario could be that the test class file is a combination of test cases with and without priority like below

package test;

import org.testng.annotations.Test;

public class TestNGPrioritiesExample {
 [@Test](http://twitter.com/Test)(priority = 1)
 public void aTest() {
  System.out.println("This is test a.");
 }

[@Test](http://twitter.com/Test)(priority = 2)
 public void bTest() {
  System.out.println("This is test b.");
 }

[@Test](http://twitter.com/Test)
 public void cTest() {
  System.out.println("This is test c.");
 }

[@Test](http://twitter.com/Test)(priority = 1)
 public void dTest() {
  System.out.println("This is test d.");
 }

[@Test](http://twitter.com/Test)
 public void eTest() {
  System.out.println("This is test e.");
 }

}

In such a case, TestNG, by default, assigns a priority = 0 to such cases that do not have the attribute. Those are executed first in alphabetical order, followed by other cases as per priorities logic. So the output for the above code would be

This is all about priorities, and these would be highly beneficial to use in any automation script while working with TestNG in Selenium. To learn more about the test case priority, you can go through our earlier blog on how to set test case priority in TestNG with Selenium.

DataProviders in TestNG

DataProviders in TestNG is a part of the inbuilt TestNG data-driven testing approach and is used to pass different values to the same TestNG test class through @DataProvider annotation.

TestNG DataProvider sample:

[@DataProvider](http://twitter.com/DataProvider)(name = "dataProvider_name")
public Object[][] dataProviderMethod() {
return new Object[][] {<values>};
}

This annotation consists of only one attribute- name and it is always of type String. If the name is not provided then the name of the method becomes the data provider name by default, dataProviderMethod in this case.

DataProvider always returns a 2-D list of objects, on which the test method performs the execution using a data-driven approach on each passed argument.

Let’s take an example of a simple cross browser code where we want to navigate to a website on both Chrome and Firefox browsers.

package test;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class DataProviderExample {

[@DataProvider](http://twitter.com/DataProvider)(name = "browserName")
 public Object[][] browserName() 
 {
  return new Object[][] { { "chrome" }, { "firefox" } };
 }

[@Test](http://twitter.com/Test)(dataProvider = "browserName")
 public void crossBrowserTest(String browser) 
 {
  WebDriver driver = null;

  System.out.println("Launching browser : " + browser);

  switch(browser)
  {
   case "chrome":
    System.setProperty("webdriver.chrome.driver", "<Path_to_your_chrome_driver>");
    driver = new ChromeDriver();
    break;
   case "firefox":
    System.setProperty("webdriver.firefox.driver", "<Path_to_your_firefox_driver>");
    driver = new FirefoxDriver();
    break;
   default:
    System.out.println("Invalid browser name passed");
  }

  driver.navigate().to("[http://www.lambdatest.com/](http://www.lambdatest.com/)");
  System.out.println("Navigated Successfully to the website");
 }

}

In the above code, we have implemented DataProvider having a name as browserName, which passes 2 values chrome and firefox. These values are passed to the test method crossBrowserTest using the parameter value and the test case is executed one by one on both browsers.

This is a classic example of cross-browser testing code implementation where we want to execute the same script on multiple browsers.

Executing the above code would give an output like:

One thing to note here is that instead of having only 1 @Test annotated test case, the execution shows two test cases are executed. This is because we passed 2 different values to the test case and hence TestNG treats it as 2 separate test cases since the test data is different.

TestNG Assertions

Assertions provide the means to verify if the actual and expected results of the test case execution are a match or not thus deciding the outcome of the test case whether it is a pass or fail. These are widely used to validate the results in any script of TestNG in Selenium.

An example of TestNG Assertions would be to validate the username on the dashboard after a user logs in to a website.

The syntax for TestNG Assert:

Assert.Method(actual, expected, message)

  • actual: the actual value which is fetched from the test case as a part of the execution.

  • expected: the expected value with which we want to compare the actual value.

  • message: string message to be displayed in case of assert failure.

Want to perform browser test automation cloud on the most powerful cloud infrastructure? Leverage LambdaTest automation testing for faster, reliable, and scalable experience on cloud.

Types of TestNG Asserts

Hard Assert: These are the default type of asserts in TestNG. These assert stop the test case execution when any assert fails and subsequent statements are not executed/validated. These are used in cases when subsequent steps in the test cases are not to be executed when a particular step assertion fails.

Soft Assert: These are the opposite of Hard Asserts. In the case of soft assert, the test case execution of subsequent steps continues even if some assert fails in between. These are used when we want to continue the execution and see the results for all steps even if a few fails in between. To use soft asserts, org.testng.asserts.SoftAssert package needs to be included.

The most commonly used TestNG asserts are:

assertTrue This assertion verifies whether the given condition is true. If false, it will fail the test case.

Assert.assertTrue(condition, message);

assertFalse This assertion verifies whether the given condition is false. If true, it will fail the test case.

Assert.assertFalse(condition, message);

assertEquals This assertion verifies if the actual and expected are a match. If they match, it passes the test case otherwise fails.

Assert.assertEquals(actual, expected, message);

assertNotEquals This assertion verifies if the actual and expected are not a match. If they match, it fails the test case otherwise passes.

Assert.assertNotEquals(actual, expected, message);

TestNG Selenium Demonstration

By now, this TestNG in Selenium tutorial has covered almost all the main features of the TestNG framework that you would need to write your first Java automation script. In the following code example, we will combine all the annotations, attributes, priorities, data providers, and assertions that we have learned so far.

The following test scenario will be executed by this code (To print and verify the sum of 2 numbers)

  1. Create a dataProvider that passes values as firstValue, secondValue, expectedSum.

  2. Navigate to https://www.lambdatest.com/selenium-playground/simple-form-demo.

  3. Run Test 1 with the first set of values, {1, 4, 5}. This test should pass as the expected sum is correct.

  4. Clear the old values from the form.

  5. Run Test 2 with the second set of values, {2, 4, 7}. This test should fail as the expected sum is not correct. We are intentionally passing incorrect expected values to see the differences in output and test reports for passed and failed cases. Always remember that ideally the expected should always be the correct value. This is done just for the tutorial purpose here.

  6. Execute the code and analyze the reports.

In addition to this test case, we will add two more test cases with priority to showcase the usage, along with one of them having invocationCount attribute to show case getting executed multiple times as per priority.

package test;

import java.net.MalformedURLException;
import java.net.URL;
import org.openqa.selenium.By;
import org.openqa.selenium.remote.*;
import org.testng.Assert;
import org.testng.annotations.*;

public class SeleniumTestNGExample {

public RemoteWebDriver driver = null;
 String username = "<lambdatest_username>";
 String accessKey = "<lambdatest_accesskey>";

[@BeforeSuite](http://twitter.com/BeforeSuite)
 public void setUp() {
  DesiredCapabilities capabilities = new DesiredCapabilities();
  capabilities.setCapability("browserName", "Chrome");
  capabilities.setCapability("version", "92.0");
  capabilities.setCapability("platform", "Windows 10");
  capabilities.setCapability("resolution", "1024x768");
  capabilities.setCapability("build", "TestNG Selenium Tutorial");
  capabilities.setCapability("name", "TestNG Selenium JAVA");

try {
   driver = new RemoteWebDriver(
     new URL("https://" + username + ":" + accessKey + "[@hub](http://twitter.com/hub).lambdatest.com/wd/hub"), capabilities);
  } catch (MalformedURLException e) {
   System.out.println("Invalid grid URL");
  }
  System.out.println("The driver setup process is completed using BeforeSuite");
 }

[@BeforeTest](http://twitter.com/BeforeTest)
 public void browserProfile() {
  driver.manage().window().maximize();
  System.out.println("The browser profile is updated using BeforeTest");

}

[@BeforeClass](http://twitter.com/BeforeClass)
 public void navigateToUrl() {
  driver.get("[https://www.lambdatest.com/selenium-playground/simple-form-demo](https://www.lambdatest.com/selenium-playground/simple-form-demo)");
  System.out.println("Navigated to URL using BeforeClass");
 }

[@Test](http://twitter.com/Test)(description = "Test case with priority", priority = 1)
 public void testPriotity()
 {
  System.out.println("This case has priority 1");
 }

 [@Test](http://twitter.com/Test)(description = "Test case with priority and invocation count", priority = 2, invocationCount = 2)
 public void testInvocationCount()
 {
  System.out.println("This case has priority 2 and invocation count");
 }

[@DataProvider](http://twitter.com/DataProvider)(name = "SanityTestData")
 public Object[][] sanityTestDataProvider() {
  String[][] testData = { { "1", "4", "5" }, { "2", "4", "7" } };
  return testData;
 }

[@Test](http://twitter.com/Test)(dataProvider = "SanityTestData", description = "Test case with group and data provider but without priority
", alwaysRun = true, groups = { "sanity" })
 public void testSumOfTwoValues(String firstValue, String secondValue, String expectedSum) {
  // to enter data and submit
  driver.findElement(By.id("sum1")).sendKeys(firstValue);
  driver.findElement(By.id("sum2")).sendKeys(secondValue);
  driver.findElement(By.xpath("//button[text()='Get values']")).click();

// to fetch actual result
  String actualSum = driver.findElement(By.id("addmessage")).getText();

// to assert actual and expected result
  Assert.assertEquals(actualSum, expectedSum,
    "Expected and actual results do not match. Expected : " + expectedSum + " and Actual : " + actualSum);

}

[@AfterMethod](http://twitter.com/AfterMethod)()
 public void clearOldData() {
  driver.findElement(By.id("sum1")).clear();
  driver.findElement(By.id("sum2")).clear();
 }

[@AfterClass](http://twitter.com/AfterClass)
 public void closeDriver() {
  driver.quit();
  System.out.println("Quit the driver using AfterClass");
 }

[@AfterTest](http://twitter.com/AfterTest)
 public void testReporting() {
  System.out.println("Test Report generated using AfterTest");
 }
}

You can see that this code uses a number of annotations, dataProviders, assertions, priorities and attributes which we have learnt and gives an idea of how these can be used together while working with automation scripts on TestNG in Selenium to use the features at full extent.

In this TestNG in Selenium tutorial, I have created an instance of Selenium RemoteWebDriver as the TestNG framework is best utilized with a Cloud Selenium Grid. The reason being, along with the advantages of Selenium Grid like executing multiple cases together across a variety of browsers and OS, a Cloud Grid also offers the speed and scalability which makes executions faster and more reliable with improved debugging and troubleshooting support.

So, in this code example, we will be executing our code on LambdaTest, which is one such Selenium Cloud Grid for performing cross browser testing at scale. It helps to execute testing across 3000+ real online browsers and OS without the hassle of maintaining in-house Selenium Grid. Selenium testing tools like LambdaTest provides interactive reporting dashboards for results analysis, a great variety of logs, and integrations with top CI/CD tools, project management tools, etc.

Next, we add the LambdaTest username and access key from the LambdaTest profile section to be used to run the script.

Code Walkthrough

Inside the setUp() method, browser capabilities are being set, and a remote grid is launched, which is to be used for executing the run.

browserProfile() method will help to execute the step to maximize the browser window once it is launched.

We have used the navigateToUrl() method to open the website on which the test would be executed.

We have added two test cases that will run on local only to showcase the usage of priority and invocationCount usage, namely testPriotity() and testInvocationCount().

sanityTestDataProvider() provides two sets of test data on which the test method would be executed of which one should pass and the other should fail, as already stated above.

testSumOfTwoValues() is the actual test method where we have added the steps to enter the values and fetch the results to be compared using assertion. This method is executed once for each type of test data and helps to decide whether it is a pass or fail by means of assert.

Next to fetch the result of execution, we use the getText() method with the identifier and store it as a String. Post this, we add Assertion to help validate the result.

In this code, we have implemented assertEquals which will compare the values and print a message if there is a mismatch.

clearOldData() function helps to clear the data filled in the input text boxes as a part of previous test execution.

After every execution, it is a good practice to close any driver instances that we have created during the course of execution. The same has been implemented by the closeDriver() method.

In order to signify the completion of the executed test run and provide the tester with a message that the test report is ready to be analyzed, we have used the testReporting() method.

The two most important parts of the sample code are the dataProvider method and the main test method.

The above helps to find the web elements and enter data in the input boxes and then it by clicking the button. To fetch web element locators, we navigate to the page and right-click, and then do Inspect to find the locators as desired by any strategy like id, XPath, CSS, etc.

This Selenium testing tutorial for beginners will help you interact with web elements in Selenium WebDriver.

https://youtu.be/VeV_sup5S8E

You can follow the LambdaTest YouTube Channel and stay updated with the latest tutorials around Selenium testing, Cypress testing, CI/CD, and more.

Having understood the sample code, now execute the class as TestNG Test to get output.

In this screenshot, we can see the test case gets executed as per priority, and the one with invocation count as 2 gets executed two times.

You can see that Default test numbers show total Tests run: 3 since we have three @Test annotated test cases in the class of which 1 has failures and hence Failures: 1.

However, the Default suite section shows the actual number of test cases along with the actual passed and failed number of cases.

Total tests run: 5 -> testPriotity() executed 1 time + testInvocationCount() executed 2 times + testSumOfTwoValues() executed once for each data set (total 2 times) -> 1 + 2 + 2 = 5

For the failed test case, the log shows the message we added on the assert to appear in case of failure: Expected and actual results do not match. Expected: 7 and Actual: 6

Also, we can log in to the LambdaTest Dashboard to check the execution results for the automation script we have executed.

You can see more detailed logs and reporting for this run by clicking on the recent test run and navigating to the LambdaTest Automation Logs section, as shown below.

In addition to the Automation Logs and related info with it, the LambdaTest Dashboard now provides users with more detailed views and data about test case execution and associated trends in graphical representation by means of LambdaTest Analytics Dashboard available under the Automation Tab.

Analytics also provides you the ability to analyze the test trends and resource utilization by means of multiple filters like users, OS, browsers, etc.

TestNG Reports Generation in Selenium

The same results can be analyzed in a more formatted way with the help of HTML reports generated by TestNG. To access the same, refresh your project by right-clicking on the project name. This is advised to update the report to the latest run.

Next, navigate to the test-output folder and from it open the index.html file. This is the default format report for the TestNG run.

If you select the test from the Info section, it shows the number of classes executed and the number of failed tests.

Selecting the group section shows the name of the group and test case methods executed as a part of it.

Chronological View shows the order in which all the methods in the class were executed, along with the time taken by each function on each step.

Last, and most important is the Results section, which shows the complete detail of the executed cases with proper exception/asserts failure in case of failed test cases. We can hide/show only passed or only failed test cases as well using this section as per the requirement.

If you are a test automation expert and want to demonstrate your proficiency in the TestNG framework, you can earn this TestNG certification offered by LambdaTest for FREE

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from LambdaTest:

https://youtu.be/dzXX2hJhuCY

Conclusion

With this Selenium TestNG tutorial, we can easily conclude that using TestNG in Selenium makes the tests more structured and easy to maintain. Moreover, with the support of its additional features like reporting, dataProviders, more robust annotations, etc., over its predecessors, TestNG makes the automation testing experience user-friendly. Having said that, it is time for you to get started with your first TestNG in Selenium automation project, and remember to use the annotations wisely as they are to make life easier.

Happy Testing!!

testngseleniumautomation testing Tutorial