Listening to TestNG lifecycle events

The listener {javadocs-base-url}/org/testng/IExecutionListener.html[IExecutionListener] allows you to be notified whenever TestNG is about to commence/conclude its execution. This listener can be used to perform setup and teardown activities at the test application layer itself ( for e.g., you could leverage this listener to boot up docker containers that are required for your application and shutdown them gracefully.)

Please note that this is the first listener TestNG would execute before it commences executing any of the <suite> found and also this would be the last listener to be invoked by TestNG (after the reporting listeners) before TestNG exits.

This listener should be declared, as explained in the section about TestNG listeners.

Here’s a sample listener implementation.

import org.testng.IExecutionListener;

public class SimpleExecutionListener implements IExecutionListener {

    @Override
    public void onExecutionStart() {
        System.err.println("TestNG is commencing execution");
    }

    @Override
    public void onExecutionFinish() {
        System.err.println("TestNG is finished execution");
    }
}

Here’s a sample test class that uses this above listener.

import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(SimpleExecutionListener.class)
public class SampleTestCase {

    @Test
    public void testMethod1() {}
}

The execution output would look like below:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
TestNG is commencing execution

===============================================
Default Suite
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

TestNG is finished execution

Process finished with exit code 0

Listening to method invocations

The listener {javadocs-base-url}/org/testng/IInvokedMethodListener.html[IInvokedMethodListener] allows you to be notified whenever TestNG is about to invoke a test (annotated with @Test) or configuration (annotated with any of the @Before or @After annotation) method and declare it as a listener, as explained in the section about TestNG listeners.

Here’s a sample listener implementation.

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;

public class SimpleInvokedMethodListener implements IInvokedMethodListener {
    @Override
    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
        log("Commencing", method);
    }

    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
        log("Completed", method);
    }

    private static void log(String prefix, IInvokedMethod method) {
        String type = "Configuration";
        if (method.isTestMethod()) {
            type = "Test";
        }
        String msg = prefix + " executing [" + type + "] method "
                + method.getTestMethod().getQualifiedName() + "()";
        System.err.println(msg);
    }
}

Here’s a sample test class that’s using this above listener.

import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(SimpleInvokedMethodListener.class)
public class SampleTestCase {

    @BeforeMethod
    public void setup() {}

    @Test
    public void testMethod1() {}

    @Test
    public void testMethod2() {}
}

Here’s how the execution console would look like:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Commencing executing [Configuration] method org.testng.demo.configs.SampleTestCase.setup()
Completed executing [Configuration] method org.testng.demo.configs.SampleTestCase.setup()
Commencing executing [Test] method org.testng.demo.configs.SampleTestCase.testMethod1()
Completed executing [Test] method org.testng.demo.configs.SampleTestCase.testMethod1()
Commencing executing [Configuration] method org.testng.demo.configs.SampleTestCase.setup()
Completed executing [Configuration] method org.testng.demo.configs.SampleTestCase.setup()
Commencing executing [Test] method org.testng.demo.configs.SampleTestCase.testMethod2()
Completed executing [Test] method org.testng.demo.configs.SampleTestCase.testMethod2()

===============================================
Default Suite
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================

Listening to configuration invocations

The listener {javadocs-base-url}/org/testng/IConfigurationListener.html[IConfigurationListener] allows you to be notified whenever TestNG is about to invoke a configuration (annotated with any of the @Before or @After annotation) method and declare it as a listener, as explained in the section about TestNG listeners. This listener also lets you be notified about whether a configuration passed, failed (or) if it got skipped.

Here’s a sample listener implementation.

import org.testng.IConfigurationListener;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;

public class MyConfigurationListener implements IConfigurationListener {

    @Override
    public void beforeConfiguration(ITestResult tr, ITestNGMethod tm) {
        //The "ITestNGMethod" will be a valid object ONLY for @BeforeMethod and @AfterMethod
        System.err.println("Commencing execution of Config method " + tr.getMethod().getQualifiedName() +
                " for the test method " + tm.getQualifiedName());
    }

    @Override
    public void onConfigurationSuccess(ITestResult tr, ITestNGMethod tm) {
        //The "ITestNGMethod" will be a valid object ONLY for @BeforeMethod and @AfterMethod
        System.err.println("Successfully executed Config method " + tr.getMethod().getQualifiedName() +
                " for the test method " + tm.getQualifiedName());
    }
}

Here’s a sample test class that’s using this above listener.

import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(MyConfigurationListener.class)
public class SampleTestClass {

    @BeforeMethod
    public void beforeMethodConfig() {
        System.err.println("Executing config method beforeMethodConfig()");
    }

    @Test
    public void testMethod() {
        System.err.println("Executing test method");
    }
}

Here’s how the execution console would look like:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Commencing execution of Config method org.testng.demo.SampleTestClass.beforeClass for the test method org.testng.demo.SampleTestClass.testMethod
Successfully executed Config method org.testng.demo.SampleTestClass.beforeClass for the test method org.testng.demo.SampleTestClass.testMethod

===============================================
Default Suite
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

Listening to class level invocations

The listener {javadocs-base-url}/org/testng/IClassListener.html[IClassListener] allows you to be notified whenever TestNG is about to start processing a test class and invoke its configuration/tests.

Add the listener implementation, as explained in the section about TestNG listeners.

Here’s a sample listener implementation.

import org.testng.IClassListener;
import org.testng.ITestClass;

public class ClassLevelListener implements IClassListener {

    @Override
    public void onBeforeClass(ITestClass testClass) {
        System.err.println("Commencing execution for the test class : " + testClass.getRealClass().getName());
    }

    @Override
    public void onAfterClass(ITestClass testClass) {
        System.err.println("Completed execution for the test class : " + testClass.getRealClass().getName());
    }
}

Here’s a sample test class that consumes the above shown sample listener.

import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(ClassLevelListener.class)
public class SampleTestCase {

    @BeforeMethod
    public void setup() {}

    @Test
    public void testMethod1() {}

    @Test
    public void testMethod2() {}
}

Execution output would be as below:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Commencing execution for the test class : org.testng.demo.configs.SampleTestCase
Completed execution for the test class : org.testng.demo.configs.SampleTestCase

===============================================
Default Suite
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================

Listening to data provider invocations

The listener {javadocs-base-url}/org/testng/IDataProviderListener.html[IDataProviderListener] allows you to be notified whenever TestNG is about invoke data provider methods. Refer to here to learn how to work with data driven tests.

Add the listener implementation, as explained in the section about TestNG listeners.

Here’s a sample listener implementation.

import org.testng.IDataProviderListener;
import org.testng.IDataProviderMethod;
import org.testng.ITestContext;
import org.testng.ITestNGMethod;

public static class SimpleDataProviderListener implements IDataProviderListener {

    @Override
    public void beforeDataProviderExecution(IDataProviderMethod dataProviderMethod, ITestNGMethod method, ITestContext iTestContext) {
        log("Commencing", dataProviderMethod, method);
    }

    @Override
    public void afterDataProviderExecution(IDataProviderMethod dataProviderMethod, ITestNGMethod method, ITestContext iTestContext) {
        log("Completed", dataProviderMethod, method);
    }

    private static void log(String prefix, IDataProviderMethod dataProviderMethod, ITestNGMethod method) {
        String msg = prefix + " execution of data provider : " + dataProviderMethod.getMethod().getName()
                + "() associated with the test method " + method.getQualifiedName() + "()";
        System.err.println(msg);
    }

    @Override
    public void onDataProviderFailure(ITestNGMethod method, ITestContext ctx, RuntimeException t) {
        String msg = "The data provider " + method.getQualifiedName() + "() failed because of "
                + t.getMessage();
        System.err.println(msg);
    }
}

A sample test class that is using this listener.

import org.testng.annotations.DataProvider;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(SimpleDataProviderListener.class)
public class SampleDataDrivenTestCase {

    @Test(dataProvider = "passing")
    public void passingTest(int ignored) {}

    @DataProvider(name = "passing")
    public Object[][] getPassingDataProvider() {
        return new Object[][] {
                {1}, {2}
        };
    }

    @Test(dataProvider = "failing")
    public void skippedTest(int ignored) {}

    @DataProvider(name = "failing")
    public Object[][] getFailingDataProvider() {
        throw new IllegalStateException("Initialisation failed");
    }
}

The execution output would look like below:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Commencing execution of data provider : getPassingDataProvider() associated with the test method org.testng.demo.SampleDataDrivenTestCase.passingTest()
Completed execution of data provider : getPassingDataProvider() associated with the test method org.testng.demo.SampleDataDrivenTestCase.passingTest()
Commencing execution of data provider : getFailingDataProvider() associated with the test method org.testng.demo.SampleDataDrivenTestCase.skippedTest()
The data provider org.testng.demo.SampleDataDrivenTestCase.skippedTest() failed because of java.lang.IllegalStateException: Initialisation failed

... # Rest of the output omitted for brevity
===============================================
Default Suite
Total tests run: 3, Passes: 2, Failures: 1, Skips: 0
===============================================

Listening to Suite level invocations

The listener {javadocs-base-url}/org/testng/ISuiteListener.html[ISuiteListener] allows you to be notified whenever TestNG is about to start processing a <suite>.

This listener can be used to perform setup/teardown activities at the suite level.

Add the listener implementation, as explained in the section about TestNG listeners.

Here’s a sample listener implementation.

import org.testng.ISuite;
import org.testng.ISuiteListener;

public class SimpleSuiteListener implements ISuiteListener {

    @Override
    public void onStart(ISuite suite) {
        log("Commencing", suite);
    }

    @Override
    public void onFinish(ISuite suite) {
        log("Completed", suite);
    }

    private static void log(String prefix, ISuite suite) {
        String msg = prefix + " execution for the suite named <" + suite.getName() + ">";
        System.err.println(msg);
    }
}

A suite <suite> xml file could look like below:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Regression-test-suite" verbose="2">
    <listeners>
        <listener class-name="org.testng.demo.SimpleSuiteListener"/>
    </listeners>
    <test name="P1-Build-Certification-Tests" verbose="2">
        <classes>
            <class name="org.testng.demo.SampleTestCase"/>
        </classes>
    </test>
</suite>

The execution output would look like below:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
...
... TestNG 7.9.0 by Cédric Beust (cedric@beust.com)
...

Commencing execution for the suite named <Regression-test-suite>
PASSED: org.testng.demo.configs.SampleTestCase.testMethod1

===============================================
    P1-Build-Certification-Tests
    Tests run: 1, Failures: 0, Skips: 0
===============================================

Completed execution for the suite named <Regression-test-suite>

===============================================
Regression-test-suite
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

Process finished with exit code 0