Skip to content

Dynamic and Parameterised Tests

Estimated time to read: 13 minutes

Dynamic Tests

@Test

With JUnit, test methods are specified using the @Test annotation, these can not be changed at runtime.

For example, if the below method needed to be run on multiple sets of data...

@Test
void testRewardProgram() {
 //...
 assertEquals(type, reward.getType());
}

... one way would be to create a list and iterate it to act and assert on different data each time.

@Test
void testRewardProgram() {
 /// ...
 List<TestData> list = createTestData();
 for(TestDate data : list) {
  // ...
  assertEquals(type, reward.getType());
 }
}

@RepeatedTest

The @RepeatedTest annotation could also be used along with the RepetitionInfo interface and removal of the for loop. However, this still acts and asserts on different data each time.

@RepeatedTest(10)
void testRewardProgram(RepetitionInfo repetitionInfo) {
 // ...

 TestData data = list.get(repetitionInfo.getCurrentRepetition() -1));

 // ...

 assertEquals(type, reward.getType());
}

@TestFactory (Dynamic Tests)

JUnit 5 introduced dynamic tests, which are generated by a method annotated with @TestFactory. @TestFactory is not a test method by itself, but rather a factory of tests.

Note

The @TestFactory method must not be private or static. Otherwise, it will not be executed by JUnit.

Data Sources

The @TestFactory can use the following as data sources:

  • Collections
  • Iterable Interfaces
  • Iterator Interfaces
  • Streams
  • Arrays

DynamicNode

Data sources must contain elements of type DynamicNode. DynamicNode is an abstract class that is the parent of DynamicContainer and DynamicTest.

DynamicContainer

DynamicContainer is a container of DynamicTest that has its own display name and contains either an iterable or a stream of DynamicNodes. It can contain other DynamicContainers or DynamicTests.

DynamicTest

DynamicTest represents the tests generated at runtime. It is composed of a display name and an Executable. The Executable is a functional interface that wraps the code of the test so it can be provided as a lambda expression or method reference.

Note

@BeforeEach and @AfterEach are not executed for each dynamic test!

Note

Dynamic tests have no notion of the lifecycle.

Example

public class RewardByGiftServiceDynamicTest {

 @BeforeEach
 void setUp() { // The setUp method will only be executed once for all tests within the @TestFactory, not for each dynamic test.
  System.out.prinln("BeforeEach");
 }

 @TestFactory // Add the `@TestFactory` annotation. 
 Collection<DynamicTest> dynamicTestsFromCollection() { // Create a collection of DynamicTest objects
  return Arrays.asList(
      dynamicTest( // Static dynamic test method
          "1st dynamic test" // Display name
          () -> assertEquals(1,1)), // Lambda with implements the `Executable` interfaces.
      dynamicTest(
          "2nd dynamic test"
          () -> assertEquals(1,1))
  );
 }

 private LongStream getStreamOfRandomNumbers() {
  Random r = new Random();
  return r.longs(1000, 2000)
 }

 private List<Product> getSampleOrder() {
  Product smallDecaf = new Product(1, "Small Decaff", 1.99);
  Product bigDecaf = new Product(2, "Big Decaff", 2.49);
  Product bigLatte = new Product(3, "Big Latte", 2.99);
  Product bigTea = new Product(4, "Big Tea", 2.99);
  Product espresso = new Product(5, "Espresso", 2.99);

  return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
 }
}
public class RewardByGiftServiceDynamicTest {

 @BeforeEach
 void setUp() { // The setUp method will only be executed once for all tests within the @TestFactory, not for each dynamic test.
  System.out.prinln("BeforeEach");
 }


 @TestFactory
 Iterator<DynamicTest> dynamicTestsFromCollection() {
  return Arrays.asList(
     dynamicTest( // Static dynamic test method
         "1st dynamic test" // Display name
         () -> assertEquals(1,1)), // Lambda with implements the `Executable` interfaces.
     dynamicTest(
         "2nd dynamic test"
         () -> assertEquals(1,1))
  ).iterator();
 }

 private LongStream getStreamOfRandomNumbers() {
  Random r = new Random();
  return r.longs(1000, 2000)
 }

 private List<Product> getSampleOrder() {
  Product smallDecaf = new Product(1, "Small Decaff", 1.99);
  Product bigDecaf = new Product(2, "Big Decaff", 2.49);
  Product bigLatte = new Product(3, "Big Latte", 2.99);
  Product bigTea = new Product(4, "Big Tea", 2.99);
  Product espresso = new Product(5, "Espresso", 2.99);

  return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
 }
}
public class RewardByGiftServiceDynamicTest {

 private RewardByGiftService reward;

  @BeforeEach
  void setUp() {
  reward = new RewardByGiftService();
  reward.setNeededPoints(100);
  System.out.prinln("BeforeEach");
 }

 @TestFactory
 Stream<DynamicTest> giftProductNotInOrderRewardNotApplied() {
  return getSTreamOfRandomNumbers() // Get a stream of random numbers 
      .limit(5) // Limit the stream to five results
      .mapToObj(random Id -> // Use a map method to take the randomId and call the `DynamicTest` static method
            dyanmicTest(
             "Testing Product ID" + randomId, // Set the display name
             () -> { // Lambda expression
              reward.setGiftProductId(randomId); // Set the random number as the giftProductId
              RewardInformation info = reward.applyReward(getSampleOrder(), 200); // Apply the reward
              assertEquals(0, info.getDiscount()); // Assert that the discount is 0
              assertEquals(0. info.getPointsRedeemed()); // Assert that the pointsRedeemed is 0
             }
            )
 }

 private LongStream getStreamOfRandomNumbers() {
  Random r = new Random();
  return r.longs(1000, 2000)
 }

 private List<Product> getSampleOrder() {
  Product smallDecaf = new Product(1, "Small Decaff", 1.99);
  Product bigDecaf = new Product(2, "Big Decaff", 2.49);
  Product bigLatte = new Product(3, "Big Latte", 2.99);
  Product bigTea = new Product(4, "Big Tea", 2.99);
  Product espresso = new Product(5, "Espresso", 2.99);

  return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
 }
}
public class RewardByGiftServiceDynamicTest {

 private RewardByGiftService reward;

  @BeforeEach
  void setUp() {
  reward = new RewardByGiftService();
  reward.setNeededPoints(100);
  System.out.prinln("BeforeEach");
 }


//   @TestFactory
//  Stream<DynamicTest> giftProductNotInOrderRewardNotApplied() {
//   return DynamicTest.stream( // DynamicTest.stream takes three parameters. The latter two must take the same type as the initial generator. See the example.
//    inputGeneratorIterator, // An iterator the serves as a dynamic input generator.
//    displayNameGeneratorFuction, //The interface function generates a displayName based on the input value.
//    testExecutorThrowingConsumer // A consumer the executes a test based on the input value.
//   )
//  }

 @TestFactory
 Stream<DynamicTest> giftProductNotInOrderRewardNotApplied() {
  Iterator<Long> inputGeneratorIterator = getStreamOfRandomNumbers().limit(5).iterator(); // Streams have a method to convert a stream to an iterator.

  Function<Long, String> displayNameGeneratorFunction = randomId - > "Testing Product ID " + randomId;

  ThrowingConsumer<Long> testExecutorThrowingConsumer = randomId -> {
   reward.setGiftProductId(randomId);
   RewardInformation info = reward.applyReward(getSampleOrder(), 200);

   assertEquals(0, info.getDiscount());
   assertEquals(0, info.getPointsRedeemed());
  };

  return DynamicTest.stream(
   inputGeneratorIterator, 
   displayNameGeneratorFunction, 
   testExecutorThrowingConsumer 
  );
 }

 private LongStream getStreamOfRandomNumbers() {
  Random r = new Random();
  return r.longs(1000, 2000)
 }

 private List<Product> getSampleOrder() {
  Product smallDecaf = new Product(1, "Small Decaff", 1.99);
  Product bigDecaf = new Product(2, "Big Decaff", 2.49);
  Product bigLatte = new Product(3, "Big Latte", 2.99);
  Product bigTea = new Product(4, "Big Tea", 2.99);
  Product espresso = new Product(5, "Espresso", 2.99);

  return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
 }
}
public class RewardByGiftServiceDynamicTest {

 private RewardByGiftService reward;

  @BeforeEach
  void setUp() {
  reward = new RewardByGiftService();
  reward.setNeededPoints(100);
  System.out.println("BeforeEach");
 }

 @TestFactory
 Stream<DynamicContainer> dynamicTestsWithContainers() {
  return longStream.range(1,6) // Create a stream of numbers from 1 to 6
      .mapToObj(productId -> dynamicContainer( // Map / convert the objects to DynamicContainer objects.
       "Container for ID" + productId, // Display name
       Stream.of( // DynamicContainer can take a stream of DynamicNodes
        dynamicTest("Valid Id", () -> assertTrue(productId > 0)) // DynamicTest to assert that productId is greater than 0
        dynamicContainer("Test", Stream.of( // dynamicContainer which contains a dynamicTest, to test that a discount has been applied as the numbers generated are the ones that happen to be in the sample order
         dynamicTest("Discount applied", () -> {
          reward.setGiftProductId(productId);
          RewardInformation info = reward.applyReward(getSampleOrder(), 200);

          assertTrue(info.getDiscount() > 0);
         }) 
        )
       ) 
      ));
 }

 private LongStream getStreamOfRandomNumbers() {
  Random r = new Random();
  return r.longs(1000, 2000)
 }

 private List<Product> getSampleOrder() {
  Product smallDecaf = new Product(1, "Small Decaf", 1.99);
  Product bigDecaf = new Product(2, "Big Decaf", 2.49);
  Product bigLatte = new Product(3, "Big Latte", 2.99);
  Product bigTea = new Product(4, "Big Tea", 2.99);
  Product espresso = new Product(5, "Espresso", 2.99);

  return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
 }
}

Parameterised Tests

Tests can be run multiple times with different arguments by using parameterised tests.

This can be done through the @ParameterizedTest annotation.

Note

@ParameterizedTest replaces the @Test annotation.

Note

@ParamterizedTests have the same life cycle as regular test methods.

Consuming Parameters

A source must be provided to @ParamterizedTest which will provide the arguments for each test invocation.

Parameterised tests consume arguments form a source whilst adhering to the following rules:

  • Zero or more indexed arguments must be declared. There is typically a 1-1 relationship between the argument source index and the method parameter index.
  • Zero or more aggregators must be declared next. Multiple parameters can be aggregated into one.
  • Zero or more arguments supplied by a ParameterResolved must be declared last.

Indexed Arguments

Indexed arguments are provided by an implementation of the ArgumentsProvider interface. ArgumentsProvider provides indexed arguments to a @ParameterizedTest method.

Each argument provided corresponds to a single method parameter; arguments are passed at the same index in the method's parameter list.

An ArgumentsProvider can be registered with the @ArgumentsSource annotation.

Argument Aggregators

ArgumentsAccessor can be used to aggregate multiple arguments into one. An instance of the object is automatically injected into any parameter of the same type. ArgumentsAccessor defined an API for accessing multiple arguments through a single one passed to the test method.

Custom aggregators can also be created by implementing the ArgumentsAggregator interface. It can then be registered with the @AggregateWith annotation.

Inject Paramaters

The ParameterResolver interface can be used to inject other types of parameters into a test. The ParameterResolver interface defines an API to dynamically resolve parameters at runtime.

Three built in resolvers are registered automatically, these are:

  • TestInfoParameterResolver - To inject parameters of type TestInfo in all types of test and lifecycle methods; @Test, @RepeatedTest, @ParameterizedTest, @TestFactory.
  • RepetitionInfoParameterResolver - To inject parameters of type RepetitionInfo for methods annotated with @RepeatedTest, @BeforeEach and @AfterEach
  • TestReporterParameterResolver - To inject parameters of type TestReporter in all types of test methods; @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, @BeforeEach, @AfterEach

JUnit Dependency

To use parameterised tests, the following dependency is required.

Group ID: org.junit.jupiter Artifact ID: junit-jupiter-params Version: 5.x.x

Custom Display Names

@ParameterizedTest's can have custom display names, like other types of test. There is a default format, however, this can be overridden.

  • {displayName} - The displayName of the method
  • {index} - The current invocation index of the parameter source, starting from one
  • {arguments} - The complete comma separated arguments list
  • {argumentsWithNames} - The complete comma separated argument list with names included
  • {0}, {1}, ... - Each argument can be called by using its identifier. Note that these start at 0

Example

junit-jupiter includes junit-jupiter-params as a transitive dependency. Ensure that JUnit is included in the pom.xml for the project.

pom.xml
 // ...
 <dependencies>
  <dependency>
   <groupId>org.junit.jupiter</groupId>
   <artifactId>junit-jupiter</artifactId>
   <scope>test</scope>
  </dependency>
 </dependencies>

 // ...
ParamTest
// ...

public class RewardByGiftServiceParameterizedTest {
 private RewardByGiftService = null;

 @BeforeEach
 void setUp() {
  reward = new RewardByGiftService();
  reward.setNeededPoints(100);
  System.out.println("BeforeEach");
 }

//  @ParameterizedTest // Annotate the test with @ParamterizedTest
 @ParameterizedTest(name = "Test #{index}: productId={0}") // Passing the name argument to the annotation, along with a value, will name each test individually using the variables for ParameterizedTest.
 @ValueSource(longs = {1, 2, 3, 4}) // ValueSource allows specifying an array of strings or primitive types. It provides a single parameter to the method. This just happens to be the single argument expected in the code below. 
 // Note that the ValueSource has been created with `longs`, this means the argument value must also be long.
 void discountShouldBeApplied(long productId) {
  reward.setGiftProductId(productId);
  RewardInformation info = reward.applyReward(getSampleOrder(), 200);

  assertTrue(info.getDiscount() > 0);

 }


 @ParameterizedTest(name = "Test #{index}: productId={0}")
 @ValueSource(longs = {5, 6, 7, 8})
 @DisplayName("Display Name")
 void discountShouldBeApplied(long productId, TestInfo testInfo, TestReporter testReporter) {
  System.out.println("Display name: " + testInfo.getDisplayName());
  testReporter.publishEntry("ProductID", String.valueOf(productId));

  reward.setGiftProductId(productId);
  RewardInformation info = reward.applyReward(getSampleOrder(), 200);

  assertTrue(info.getDiscount() > 0);

 }

 // ...
}

Note

The @BeforeEach method will still run before each parameterized test as it follows the lifecycle of regular test methods. This is in contrast to @TestFactory!

Argument Sources

In parameterized tests, arguments can be provided by sources via the use of annotations.

There are three rules to consider when using argument sources:

  • For every test method, there should be at least one source, else the test will not be executed
  • Each source must provide arguments for all expected method parameters
  • The test will be executed once for each group of arguments

@ValueSource

@ValueSource allows defining arrays of type:

  • java.lang.String
  • java.lang.Class
  • Primitives (int, boolean, et al)

Can only be used on test methods that accept a single parameter.

Example

 @ParameterizedTest(name = "Test #{index}: productId={0}")
 @ValueSource(longs = {5, 6, 7, 8})
 @DisplayName("Display Name")
 void discountShouldBeApplied(long productId, TestInfo testInfo, TestReporter testReporter) {
  System.out.println("Display name: " + testInfo.getDisplayName());
  testReporter.publishEntry("ProductID", String.valueOf(productId));

  reward.setGiftProductId(productId);
  RewardInformation info = reward.applyReward(getSampleOrder(), 200);

  assertTrue(info.getDiscount() > 0);
 }

@EnumSource

@EnumSource can be used to run a test with the values of a provided enum.

It takes, as optional parameters:

  • Names of the enum values to be included / excluded
  • Mode of the parameter, depending on the value

Can only be used on test methods that accept a single parameter.

Example

@ParameterizedTest
@EnumSource(SpecialProductsEnum.class)
void discountShouldBeAppliedEnumSource(SpecialProductsEnum product) {
 reward.setGiftProductId(product.getProductId());
 RewardInformation info = reward.applyReward(getSampleOrder(), 200);

 assertTrue(info.getDiscount() >0);
}

@ParameterizedTest
@EnumSource(SpecialProductsEnum.class, names= = {"BIG_LATTE", "BIG_TEA"}) // Specifying names allows only certain values to be tested from the Enum.
void discountShouldBeAppliedEnumSourceLimited(SpecialProductsEnum product) {
 reward.setGiftProductId(product.getProductId());
 RewardInformation info = reward.applyReward(getSampleOrder(), 200);

 assertTrue(info.getDiscount() >0);
}
public enum SpecialProductsEnum {
 SMALL_DECAF(1),
 BIG_DECAF(2),
 BIG_LATTE(3),
 BIG_TEA(4),
 ESPRESSO(5);

 private final int productId;

 private SpecialProductsEnum(int productId) {
  this.productId = productId;
 }

 public int getProductId(){
  return productId;
 }
}

@MethodSource

@MethodSource allows the specification of one or more methods that will provide the arguments for the test.

For single parameter tests, the parsed methods can:

  • Return a Stream of the parameter type
  • Return a Stream of primitive types

For multiple parameter tests, the parsed methods can:

  • Return a Stream, Iterable, Iterator, or array of elements with type Arguments. Arguments is an interface for wrapping an array of objects.

The methods parsed by @MethodSource must be static, unless a @TestInstance(Lifecycle.PER_CLASS) lifecycle is in use; in this case, methods can be defined in an external class as long as a fully qualified method name is given.

Example

@MethodSource - Single param
@ParameterizedTest
@MethodSource("productIds")
void discountShouldBeAppliedSource(long productId) {
 reward.setGiftProductId(product.getProductId());
 RewardInformation info = reward.applyReward(getSampleOrder(), 200);

 assertTrue(info.getDiscount() >0);
}

private static LongStream productIds() { // This method can be private, however, it must be static!
 return LongStream.range(1,6);
}
@Method Source - Multi param
@ParameterizedTest
@MethodSource("productIdsCustomerPoints")
void discountShouldBeAppliedSourceMultiParam(long productId, long customerPoints) {
 reward.setGiftProductId(product.getProductId());
 RewardInformation info = reward.applyReward(getSampleOrder(), 200);

 assertTrue(info.getDiscount() >0);
}

static Stream<Arguments> productIdsCustomerPoints() { // Returns a string of `Arguments` object.
 return productIds().mapToObj(productId -> Arguments.of(productId, 100 * productId));
 // Outputs 2, 200
}

@CsvSource

@CsvSource allows the declaration of arguments as a comma-separated list of strings.

It takes:

  • delimiter - To specify the delimiting character. This is , by default.
  • delimiterString

Single quotes ' are used as quote characters.

@CsvFileSource

@CsvFileSource takes one or more CSV files from the CLASSPATH or the local file system.

It takes:

  • files - To specify files from the file system
  • resources - To specify files from the CLASSPATH
  • attributes - To specify the files' encoding
  • lineSeparator
  • delimiter - To specify the delimiting character. This is , by default.
  • delimiterString

Each line of the CSV files results in a test invocation.

Lines beginning as a # will interpreted as a comment within the file.

Double quotes " are used as quote characters.

Example

@ParameterizedTest
@CsvFileSource(resources ="/product-point-data.csv")
void discountShouldBeAppliedCsvFileSource(long productId, long customerPoints) {
 reward.setGiftProductId(product.getProductId());
 RewardInformation info = reward.applyReward(getSampleOrder(), 200);

 assertTrue(info.getDiscount() >0);
}

Null and Empty sources

  • @NullSource provides a null argument. It can not be used for primitive type arguments.
  • @EmptySource provides an empty value for parameters of type String, List, Set, Map or arrays.
  • @NullAndEmptySource combines the functionality of @NullSource and @EmptySource.

@ArgumentsSource

@ArgumentsSrouce allows defining custom sources through an ArgumentsProvider interface implementation. This interface returns a string of elements of type Arguments and takes an object of type ExtensionContext.

interface ArgumentsProvider {
 Stream<? extends Arguments>
  provideArguments(ExtensionContext context)
   throws Exception;
}

Example

ProductIdArgumentsProvider.java
public class ProductIdArgumentsProvider implements ArgumentsProvider {

    @Override
    public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
        return LongStream.range(1, 6)
                .mapToObj(
                        productId ->
                                Arguments.of(productId, 200 * productId)
                );
    }
}
@ParameterizedTest
    @ArgumentsSource(ProductIdArgumentsProvider.class)
    void discountShouldBeAppliedArgumentsSource(long productId, long customerPoints) {
        reward.setGiftProductId(productId);
        RewardInformation info = reward.applyReward(
                getSampleOrder(), customerPoints);

        assertTrue(info.getDiscount() > 0);
    }

Argument Conversion

When working with parameterized tests, if the argument and the parameter types do not match, the argument type can be converted implicitly.

String to Object Conversion

Implicit Conversion

In particular, for the annotations @ValueSource, @CsvSource and @CsvFileSource, the implicit conversion is performed when the argument is provided as a string.

A String type can be converted implicitly to:

  • Primitive types and their wrapper classes
  • Enums
  • java.time classes
  • localDate
  • period
  • et al
  • Path
  • Currency
  • Locale
  • And more...

Factory Conversion

Strings can also be converted to a particular type if the type declares exactly one non-private static factory method that accepts a single string argument and returns an instance of the type.

The name of the method is irrelevant. However, if there are multiple factory methods with these characteristics, no conversion will happen.

MyTest.java
public class MyTest {
 @ParameterizedTest
 @ValueSource(strings = "Latte")
 void testProduct(Product p) {
  // ...
 }
}
Product.java
public class Product {
 private long id;
 private String name;
 private double price;

 // ...

 public static Product factoryMethod(String name) {
  return new Product(-1, name, 0.0);
 }
}

Constructor Conversion

JUnit can also use a non-private constructor of the type that accepts a single string argument, but the type must be declared either as a top-level class or as a static nested class.

If a factory method and a constructor are discovered, then the factory method will be used.

MyTest.java
public class MyTest {
 @ParameterizedTest
 @ValueSource(strings = "Latte")
 void testProduct(Product p) {
  // ...
 }
}
Product.java
public class Product {
 private long id;
 private String name;
 private double price;

 // ...

 public Product(String name) {
  this.id = -1;
  this.name = name;
  this.price = 0.0;
 }
}

Custom Converters

A custom converter can be created using the ArgumentConverter interface and then put into use by using the @ConvertWith annotation.

ArgumentConverter
interface ArgumentConverter {
 Object convert(Object source, ParameterContext context)
  throws ArgumentConversionException;
}

The abstract class SimpleArgumentConverter extends ArgumentConverter. Its convert method receives the target type that the source object should be converted in to.

SimpleArgumentConverter
abstract class SimpleArgumentConverter implements ArgumentConverter {
 protected abstract Object convert(Object source, Class<?> targetType)
  throws ArgumentConversionException;
}

The abstract class TypedArgumentConverter extends ArgumentConverter and avoid type checks

TypedArgumentConverter
abstract class TypedArgumentConverter<S, T> implements ArgumentConverter {
 protected abstract T convert(S source)
  throws ArgumentConversionException;
}

Conversion Example

discountShouldBeAppliedCustomConverter.java
@ParameterizedTest
@ValueSource(string = { "1; Small Decaf; 1..99", "2; Big Decaf; 2.49"})
void discountShouldBeAppliedCustomConverter(@ConvertWith(ProductArgumentConverter.class)Product product) {
 System.out.println("Testing product " + product.getName());
 reward.setGiftProductId(product.getId());
 RewardInformation info - reward.applyReward(getSampleOrder(), 200)

 assertTrue(info.getDiscount() > 0);
}

ProductArgumentConverter extends from TypedArgumentConveter. It takes a String as the input type and Product as the output type.

ProductArgumentConverter.java
// ...

public class ProductArgumentConverter extends TypedArgumentConverter<String, Product> {
 protected ProductArgumentConverter() { // Implement the constructor and include the input/output types
  super(String.class, Product.class); 
 }

 @Override
 protected Product convert(String source) { 
  String[] productString = source.split(";");

  Product product = new Product(
      Long.parseLong(productString[0]),
      productString[1].trim(),
      Double.parseDouble(productString[2])
  );
  return product;
}