TestNG Tutorial | TestNG Annotations | TestNG Selenium Reference

TestNG Tutorial – TestNG Selenium – TestNG Annotations – TestNG vs JUnit and TestNG examples. These are the most common search keywords for many testers and in this TestNG tutorial article, I will explain TestNG framework. We can write our tests by using Selenium and JUnit or TestNG. Before, I wrote a comprehensive tutorial for selenium tutorial with JUnit. Also, you can find detailed articles for JUnit here. Now, we can learn TestNG and its features. Let’s start to this awesome TestNG tutorial.

TestNG Tutorial Starts with What is TestNG?

TestNG is a powerful testing framework which helps to write the test easier and more functional. It is abbreviation is TestNG = Test Next Generation.

With TestNG, we will write more flexible and powerful tests with help of easy annotations, grouping, sequencing & parametrizing, and parallel execution. We will learn all of this features in this TestNG tutorial article.

How to Write a Test with TestNG?

• Add TestNG maven dependency to your pom.xml
• Write your test scenario with TestNG Annotations.
• Structure your test in testng.xml file. (Classes, methods, groups, parameters, listeners, parallelization, etc.)
• Run the test.

TestNG Annotations

@BeforeSuite: The annotated method runs before all tests in this suite.

  @BeforeTest: The annotated method runs before test classes.

    @BeforeGroups: The annotated method runs before the group’s tests.

      @BeforeClass: The annotated method runs before the first test method which belongs to any of the test groups. 

        @BeforeMethod: The annotated method runs before each test method.

          @Test: The annotated method is a test method. It executes the test.

        @AfterMethod: The annotated method runs after each test method.

      @AfterClass: The annotated method runs after all the test methods in the current test class.

    @AfterGroups: The annotated method runs after the last test method which belongs to any of the test groups.

  @AfterTest: The annotated method runs after test classes.

@AfterSuite: The annotated method runs after all tests.

Levels in TestNG is shown below:

Example Test Code for TestNG Annotations:

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.*;

/*****************************************************************************
 * Author:      Onur Baskirt
 * Description: It is an example of TestNG annotations.
 *******************************************************************************/

public class TestNGAnnotations {
    //-----------------------------------Global Variables-----------------------------------
    //Declare a Webdriver variable
    public WebDriver driver;

    //Declare a test URL variable
    public String testURL = "https:/";

    //-----------------------------------Suite Suite-----------------------------------
    //You can set suite level setup such as test environment, Database connection string etc.
    @BeforeSuite
    public void setupSuite () {
        System.out.println("@BeforeSuite started.");
    }

    //-----------------------------------All Tests Setup-----------------------------------
    //All Tests setup
    @BeforeTest
    public void setupTests () {
        System.out.println("@BeforeTest started.");
    }

    //-----------------------------------Class Setup-----------------------------------
    //All Tests setup
    @BeforeClass
    public void setupClass () {
        System.out.println("@BeforeClass started.");
    }

    //-----------------------------------Test Setup-----------------------------------
    @BeforeMethod
    public void setupTest (){
        //Print console that code is in @BeforeMethod!
        System.out.println("@BeforeMethod has started.");

        //Create a new ChromeDriver
        driver = new ChromeDriver();
    }

    //-----------------------------------Tests-----------------------------------
    @Test
    public void firstTest () {
        System.out.println("First Test is Started.");

        //Go to www.swtestacademy.com
        driver.navigate().to(testURL);

        //Get page title
        String title = driver.getTitle();

        //Print page's title
        System.out.println("Page Title: " + title);
    }

    @Test
    public void secondTest () {
        System.out.println("Second Test is Started.");

        //Go to www.amazon.com
        driver.navigate().to("https://www.amazon.com/");

        //Get page title
        String title = driver.getTitle();

        //Print page's title
        System.out.println("Page Title: " + title);
    }

    //-----------------------------------Test TearDown-----------------------------------
    @AfterMethod
    public void teardownTest (){
        System.out.println("@AfterMethod has started.");
        //Close browser and end the session
        driver.quit();
    }

    //-----------------------------------Class TearDown-----------------------------------
    @AfterClass
    public void teardownClass (){
        System.out.println("@AfterClass has started.");
    }

    //-----------------------------------All Tests TearDown-----------------------------------
    @AfterTest
    public void teardownTests (){
        System.out.println("@AfterTest has started.");
    }

    //-----------------------------------Suite TearDown-----------------------------------
    @AfterSuite
    public void teardownSuite (){
        System.out.println("@AfterSuite has started.");
    }
}

Example’s console output is shown below:

TestNG XML File

A sample testng.xml structure is shown below:

First, we need to give our Suite Name,

then our Test Name,

then Test Class Name,

and we can also Include or Exclude Test Methods. You can find an example TestNG XML file for our TestNG tutorial.

Sample TestNG XML file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="AllTestsSuite">
    <test name="Testing Annotations Test">
        <classes>
            <class name="TestNGAnnotations">
                <methods>
                    <include name="firstTest" />
                    <exclude name="secondTest" />
                </methods>
            </class>
        </classes>
    </test>
</suite>

Test Case Grouping in TestNG

In testng.xml file, you need to write which groups of tests will you run as shown below.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<!--Grouping Example-->
<suite name="Selenium Training Suite">
    <test name="Selenium TestNG Test">
        <groups>
            <run>
                <include name="Smoke"/>
                <include name="Regression"/>
            </run>
        </groups>
        <classes>
            <class name="testng.TestNGGrouping"></class>
        </classes>
    </test>
</suite>

Example Test Code for TestNG Grouping:

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.*;

/**
 * Created by obaskirt on 02-Jun-17.
 */
public class TestNGGrouping {

    //-----------------------------------Test Setup-----------------------------------
    @BeforeTest
    public void setupTest (){
        //Print console that code is in @BeforeMethod!
        System.out.println("@BeforeTest has started.");
    }

    //-----------------------------------Tests-----------------------------------
    @Test (groups = {"Regression", "Smoke"})
    public void firstTest () {
        System.out.println("1st Test is Started.");
    }

    @Test (groups = {"Regression", "Smoke"})
    public void secondTest () {
        System.out.println("2nd Test is Started.");
    }

    @Test (groups = {"Regression"})
    public void thirdTest () {
        System.out.println("3rd Test is Started.");
    }

    @Test (groups = {"Medium"})
    public void fourthTest () {System.out.println("4th Test is Started.");}

    @Test (groups = {"Regression"})
    public void fifthTest () {
        System.out.println("5th Test is Started.");
    }

    @Test (groups = {"Medium"})
    public void sixthTest () {
        System.out.println("6th Test is Started.");
    }

    //-----------------------------------Test TearDown-----------------------------------
    @AfterTest
    public void teardownTest (){
        System.out.println("@AfterTest has started.");
    }
}

Test Results:

In TestNG.xml file we defined “Smoke” and “Regression” group tests so these tests run as you see in below figure.

Dependent Test Cases in TestNG | dependsOnMethods & dependsOnGroups 

If you are testing a user journey, you may need to use dependsOnMethods and dependsOnGroups annotations of TestNG. I will show you an example of dependsOnMethods for our TestNG tutorial.

loginWebsite depends on openWebsite method
goToBasket depends on loginWebsite
logOutFromWebsite depends on goToBasket

@Test(dependsOnMethods = {"openWebsite"})
public void loginWebsite () {
    System.out.println("Logged-in website!");
}

@Test
public void openWebsite () {
    System.out.println("Website opened!");
}

@Test (dependsOnMethods = {"loginWebsite"})
public void goToBasket () {
    System.out.println("We are in basket!");
}

@Test (dependsOnMethods = {"goToBasket"})
public void logOutFromWebsite () {
    System.out.println("Logged-out from website!");
}

Example Test Results:

TestNG Prioritization | Priority Feature of TestNG

You have to use the priority parameter, to prioritize the test methods as shown below. In below code, we ordered the tests reversely. Thus, the first 3rd test will run, then the 2nd test will run and finally 1st test will run.

@Test(priority = 2)
public void firstTest () {
    System.out.println("1st Test is Started.");
}

@Test (priority = 1)
public void secondTest () {
    System.out.println("2nd Test is Started.");
}

@Test (priority = 0)
public void thirdTest () {
    System.out.println("3rd Test is Started.");
}

TestNG Priority Example Result:

How to Skipping Tests in TestNG

You can skip any test by using “enabled=false” parameter.

@Test(priority = 0)
public void firstTest () {
    System.out.println("1st Test is Started.");
}

@Test (priority = 1, enabled = false)
public void secondTest () {
    System.out.println("2nd Test is Started.");
}

@Test (priority = 2)
public void thirdTest () {
    System.out.println("3rd Test is Started.");
}

TestNG skipping test cases example result:

TestNG Parameters & Cross Browser Testing in TestNG

We can set parameters in testng.xml file and use them in our tests. Especially we can set global variables such as browser etc.

First, we need to set browser parameters in our testng.xml

<suite name="AllTestsSuite">
    <test name="TestNG Basics Test">
        <parameter name="browser" value="chrome" />
        <!--<parameter name="browser" value="firefox" />-->
        <classes>
            <class name="TestNGCrossBrowser">
            </class>
        </classes>
    </test>
</suite>

Then we should get the parameter by using @Parameters annotation and write an if/else or switch block code snippet for browser selection in the setup method.

@BeforeMethod
@Parameters("browser")
public void setupTest (String browser){
    //Print console that code is in @BeforeMethod!
    System.out.println("@BeforeMethod has started.");

    //Browser Selection based on browser parameter
    if(browser.equals("chrome")){
        driver = new ChromeDriver();
    } else if (browser.equals("firefox")) {
        driver = new FirefoxDriver();
    } else {
        System.out.println("You set a wrong browser parameter!");
    }
}

TestNG DataProvider

A Data Provider is a method annotated with @DataProvider and it returns an array of objects. First, you need to write your DataProvider method and declare your data parameters values.

@DataProvider (name = "LoginData")
public static Object[][] loginData () {
    //Userdata is formatted as username and password
    return new Object[][] { { "user1", "pass1" }, { "user2", "pass2" }};
}

Then use your DataProvider in your test method.

@Test (dataProvider = "LoginData")
public void dataProviderTest (String username, String password) {
    System.out.println("Test is Started...");

    System.out.println("Username: " + username);
    System.out.println("Password: " + password );
}

DataProvider Test Result:

TestNG Assertions

We will do our checks and controls with Assertions. TestNG provides several assertions the most common assertions are Assert.assertTrue(), Assert.assertFalse(), Assert.assertEquals().

Assert.assertTrue(Condition, Message)

If the condition is not true, an AssertionError, with the given message, is thrown.

@Test
public void assertTrueExample () {
    //If check is true, test will pass.
    System.out.println("AssertTrue Example");
    String myString = "swtestacademy.com";
    //Condition, Message
    Assert.assertTrue(myString.equals("swtestacademy.com"), "AssertTrue test is failed!");
    System.out.println("AssertTrue test is Passed!\n");
}

Assert.assertFalse(Condition, Message)

If the condition is not false, an AssertionError is thrown.

@Test
public void assertFalseExample () {
    //If check is false, test will pass.
    System.out.println("AssertFalse Example");
    String myString = "swtestacademy.com";
    //Condition, Message
    Assert.assertFalse(myString.equals("Wrong-String"), "AssertFalse test is failed!");
    System.out.println("AssertFalse Test is Passed!\n");
}

Assert.assertEquals(String actual,String expected, String message)

If actual and expected conditions are not equal, AssertionError with the given message is thrown.

@Test
public void assertEqualsExample () {
    System.out.println("AssertEquals Example");
    String myString = "swtestacademy.com";
    //Actual String, Expected String, Message
    Assert.assertEquals("swtestacademy.com", myString, "AssertEquals test is failed!");
    System.out.println("AssertEquals Test is Passed!\n");
}

Run TestNG Tests in Random Order

In order to run tests in random order, you need to add below class into your project.

import java.util.Collections;
import java.util.List;
import java.util.Random;

import org.testng.IMethodInterceptor;
import org.testng.ITestContext;

public class TestOrderRandomizer implements IMethodInterceptor {
    @Override
    public List intercept(List methods, ITestContext context) {
        long seed = System.nanoTime();
        Collections.shuffle(methods, new Random(seed));
        return methods;
    }
}

Then, you need to add TestOrderRandomizer class in TestNG.xml’s listener tags as shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="Test Suite">

<listeners>
    <listener class-name="Utility.TestOrderRandomizer"/>
</listeners>

<test name="AdayWebSite">
    <parameter name="browser" value="chrome"/>
    <classes>
        <class name="Test1"/>
        <class name="Test2"/>
        <class name="Test3"/>
    </classes>
</test>
</suite>

Run TestNG Tests in Sequential Order

In order to run tests in sequential order, we should use group-by-instances=”true” in TestNG.xml file as shown below example:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="Test-Suite">

    <test name="Example Tests" group-by-instances="true">
        <classes>
            <class name="tests.NavigateToScreensTest" />
            <class name="tests.LoginTests">
                <methods>
                    <include name="invalidTypeUsernameTest"/>
                    <include name="wrongUsernameTest"/>
                    <include name="wrongPasswordTest"/>
                    <include name="successfulLoginTest"/>
                </methods>
            </class>
        </classes>
</suite>

How to Use Multiple Suites in TestNG

You need to define multiple suites in an XML as shown below.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="ETS Suite">
    <suite-files>
        <suite-file path="test-suite1.xml" />
        <suite-file path="test-suite2.xml" />
    </suite-files>
</suite>

Retry Tests with TestNG

Retrying the failed test cases is described in this article.

TestNG Listeners

TestNG Listeners is described in this article.

How to Run Tests in Parallel with TestNG

Run your tests in parallel without Selenium Grid (on your local computer) is described in this article.

Run your tests in parallel with Selenium Grid and TestNG is described in this article.

Run your mobile tests in parallel with TestNG and Appium is described in this article.

TestNG has many powerful features. In this TestNG Tutorial article, I shared the most common and important features of TestNG. I hope you enjoyed reading.

[fusion_widget_area name=”avada-custom-sidebar-seleniumwidget” title_size=”” title_color=”” background_color=”” padding_top=”” padding_right=”” padding_bottom=”” padding_left=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=””][/fusion_widget_area]

Thanks.
Onur Baskirt

Onur Baskirt

Onur Baskirt is a Software Engineering Leader with international experience in world-class companies. Now, he is a Software Engineering Lead at Emirates Airlines in Dubai.