Parameters
TestNG allows you to pass an arbitrary number of parameters to each of your tests using the `@Parameters` annotation.
There are three ways to set these parameters
-
The testng.xml file
-
Programmatically
-
Java system properties
Parameters from testng.xml
If you are using simple values for your parameters, you can specify them in your testng.xml
:
@Parameters({ "first-name" })
@Test
public void testSingleString(String firstName) {
System.out.println("Invoked testString " + firstName);
assert "Cedric".equals(firstName);
}
In this code, we specify that the parameter firstName
of your Java method should receive the value of the XML parameter called first-name
. This XML parameter is defined in testng.xml
:
<suite name="My suite">
<parameter name="first-name" value="Cedric"/>
<test name="Simple example">
<!-- ... -->
</test>
</suite>
The same technique can be used for @Before
/@After
and @Factory
annotations:
@Parameters({ "datasource", "jdbc-driver" })
@BeforeMethod
public void beforeTest(String ds, String driver) {
m_dataSource = buildDataSource(); // look up the value of datasource
m_jdbcDriver = driver;
}
This time, the two Java parameter ds
and driver
will receive the value given to the properties datasource
and jdbc-driver
respectively.
Parameters can be declared optional with the {javadocs-base-url}/org/testng/annotations/Optional.html[org.testng.annotations.Optional] annotation:
@Parameters("db")
@Test
public void testNonExistentParameter(@Optional("mysql") String db) {
//more code
}
If no parameter named "db" is found in your testng.xml
file, your test method will receive the default value specified inside the @Optional
annotation: "mysql".
The @Parameters
annotation can be placed at the following locations:
-
On any method that already has a
@Test
,@Before
/@After
or@Factory
annotation. -
On at most one constructor of your test class. In this case, TestNG will invoke this particular constructor with the parameters initialized to the values specified in
testng.xml
whenever it needs to instantiate your test class. This feature can be used to initialize fields inside your classes to values that will then be used by your test methods.
Important:
-
The XML parameters are mapped to the Java parameters in the same order as they are found in the annotation, and TestNG will issue an error if the numbers don’t match.
-
Parameters are scoped. In testng.xml, you can declare them either under:
-
<suite>
tag or -
<test>
tag or -
<class>
tag or -
<methods>
tag.
-
-
The order of precedence (lowest to highest) in terms of resolving values for parameters with same names is
<suite> --> <test> --> <class> --> <methods>
-
If two parameters have the same name, it’s the one defined in
<methods>
that has precedence. This is convenient if you need to specify a parameter applicable to all your tests and override its value only for certain test method.
Parameters with DataProviders
Specifying parameters in testng.xml might not be sufficient if you need to pass complex parameters, or parameters that need to be created from Java (complex objects, objects read from a property file or a database, etc…). In this case, you can use a Data Provider to supply the values you need to test. A Data Provider is a method on your class that returns an array of array of objects. This method is annotated with @DataProvider
:
//This method will provide data to any test method that declares that its Data Provider
//is named "test1"
@DataProvider(name = "test1")
public Object[][] createData1() {
return new Object[][] {
{ "Cedric", 36 },
{ "Anne", 37},
};
}
//This test method declares that its data should be supplied by the Data Provider
//named "test1"
@Test(dataProvider = "test1")
public void verifyData1(String n1, Integer n2) {
System.out.println(n1 + " " + n2);
}
will print
Cedric 36
Anne 37
A @Test
method specifies its Data Provider with the dataProvider
attribute. This name must correspond to a method on the same class annotated with @DataProvider(name="…")
with a matching name.
By default, the data provider will be looked for in the current test class or one of its base classes. If you want to put your data provider in a different class, it needs to be a static method or a class with a non-arg constructor, and you specify the class where it can be found in the dataProviderClass
attribute:
public class StaticProvider {
@DataProvider(name = "create")
public static Object[][] createData() {
return new Object[][] {
new Object[] { 42 }
};
}
}
public class MyTest {
@Test(dataProvider = "create", dataProviderClass = StaticProvider.class)
public void test(Integer n) {
// ...
}
}
The data provider supports injection too. TestNG will use the test context for the injection. The Data Provider method can return one of the following types:
-
An array of array of objects (
Object[][]
) where the first dimension’s size is the number of times the test method will be invoked and the second dimension size contains an array of objects that must be compatible with the parameter types of the test method. This is the case illustrated by the example above. -
An
Iterator<Object[]>
. The only difference withObject[][]
is that anIterator
lets you create your test data lazily. TestNG will invoke the iterator and then the test method with the parameters returned by this iterator one by one. This is particularly useful if you have a lot of parameter sets to pass to the method and you don’t want to create all of them upfront.-
An array of objects (
Object[]
). This is similar toIterator<Object[]>
but causes the test method to be invoked once for each element of the source array. -
An
Iterator<Object>>
. Lazy alternative ofObject[]
. Causes the test method to be invoked once for each element of the iterator.
-
It must be said that return type is not limited to Object only thus MyCustomData[][]
or Iterator<Supplier>
are also possible. The only limitation is that in case of iterator its parameter type can’t be explicitly parameterized itself. Here is an example of this feature:
@DataProvider(name = "test1")
public Iterator<Object[]> createData() {
return new MyIterator(DATA);
}
Using MyCustomData[]
as a return type
@DataProvider(name = "test1")
public MyCustomData[] createData() {
return new MyCustomData[]{ new MyCustomData() };
}
Or its lazy option with Iterator<MyCustomData>
@DataProvider(name = "test1")
public Iterator<MyCustomData> createData() {
return Arrays.asList(new MyCustomData()).iterator();
}
Parameter type (Stream
) of Iterator
can’t be explicitly parametrized
@DataProvider(name = "test1")
public Iterator<Stream> createData() {
return Arrays.asList(Stream.of("a", "b", "c")).iterator();
}
If you declare your @DataProvider
as taking a java.lang.reflect.Method
as first parameter, TestNG will pass the current test method for this first parameter. This is particularly useful when several test methods use the same @DataProvider
and you want it to return different values depending on which test method it is supplying data for.
For example, the following code prints the name of the test method inside its @DataProvider
:
@DataProvider(name = "dp")
public Object[][] createData(Method m) {
System.out.println(m.getName()); // print test method name
return new Object[][] { new Object[] { "Cedric" }};
}
@Test(dataProvider = "dp")
public void test1(String s) {
}
@Test(dataProvider = "dp")
public void test2(String s) {
}
and will therefore display:
test1
test2
Data providers can run in parallel with the attribute parallel:
@DataProvider(parallel = true)
public Object[][] getTestData() {
// ...
}
Each of the parallel data providers running from an XML file runs with a thread pool which has a size of 10
by default. You can modify this value in the <suite>
tag of your XML file:
<suite name="Suite1" data-provider-thread-count="20" >
<!-- content ignored for brevity -->
</suite>
If you want to run a few specific data providers in a different thread pool, you need to run them from a different XML file.
Retries and data providers
TestNG allows you to retry a data provider incase it has encountered any issues when calling it the first time.
This is similar to how regular test methods can be retried as explained in this section.
To be able to retry a data provider, the following needs to be done.
-
First we would need to implement the interface
org.testng.IRetryDataProvider
. -
Next you would need to tie this implementation to the data provider annotation using the attribute
retryUsing
of the@DataProvider
annotation. -
With that we can now retry a failed ata provider.
Here’s a sample retry implementation:
import org.testng.IDataProviderMethod;
import org.testng.IRetryDataProvider;
import java.util.concurrent.atomic.AtomicInteger;
public class RetryDataProvider implements IRetryDataProvider {
private final AtomicInteger counter = new AtomicInteger(1);
@Override
public boolean retry(IDataProviderMethod dataProvider) {
boolean status = counter.getAndIncrement() <= 2;
String clazz = dataProvider.getMethod().getDeclaringClass().getName();
String dataProviderMethodName = dataProvider.getMethod().getName();
String methodName = clazz + "." + dataProviderMethodName + "()";
System.err.println("Retry the data provider method " + methodName + " ? " + status);
return status;
}
}
Here’s how a test class that consumes this retry mechanism can look like:
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
public class RetriableDataProviderSample {
private boolean throwException = true;
@Test(dataProvider = "test-data")
public void sampleTestMethod(int input) {
System.err.println("Input value = " + input);
}
@DataProvider(retryUsing = RetryDataProvider.class, name = "test-data")
public Object[][] testDataSupplier() {
if (throwException) {
throwException = false;
System.err.println("Simulating a problem when invoking the data provider");
throw new IllegalStateException("Simulating a failure in data provider");
}
return new Object[][]{
{1}, {2}
};
}
}
And when you run this sample, the output would look something like below:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Simulating a problem when invoking the data provider
Retry the data provider method org.testng.demo.RetriableDataProviderSample.testDataSupplier() ? true
Input value = 1
Input value = 2
===============================================
Default Suite
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================
Controlling ThreadPool Usage
Starting from TestNG 7.9.0
, there are some additional ways in which the thread-pools that run the parallel tests can be controlled. For these new features to be consumed, update your suite file to use the testng-1.1.dtd
(as seen below) so that your IDE can provide you with autocompletion:
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.1.dtd" >
<suite name="sample">
<!-- content ignored for brevity -->
</suite>
-
share-thread-pool-for-data-providers
- When this attribute is set to true at the suite level, TestNG will start using a shared thread pool for all the data driven tests in a given<suite>
. The size of the thread pool is determined using the attributedata-provider-thread-count
. This attribute has a default value offalse
. -
use-global-thread-pool
- When this attribute is set to true at the suite level, TestNG will start using a common thread pool for running both your regular test methods and data driven test methods. The size of the thread pool is determined using the attribute thread-count. This attribute has a default value offalse
.
Parameters from System Properties
TestNG can be passed parameters on the command line of the Java Virtual Machine using system properties (-D
). Parameters passed in this way are not required to be pre-defined in testng.xml
, but will override any parameters defined there.
java -Dfirst-name=Cedrick -Dlast-name="von Braun" org.testng.TestNG testng.xml
The Java system property variable is a string with no spaces that represents the name of the property. The value variable is a string that represents the value of the property. If the value is a string with spaces, then enclose it in quotation marks.
In TestNG 6.x parameters defined in testng.xml could not be overwritten by system properties |
Parameters in reports
Parameters used to invoke your test methods are shown in the HTML reports generated by TestNG. Here is an example: