TestNG testing framework —Test Name in Extent Report

5 min readAug 5, 2023

We went over the following topics:

Test NG Listener
Test Parameterization
Extent report

Now let's discuss on how we can make Test Name unique without hard-coding it

What we have till now? (Hard coded Test Name specified in test methods)

Let's Capture Class Name & Method Name at runtime before test is initialized

private static String getTestName(ITestResult iTestResult){
        String[] resultDataArray = iTestResult.getMethod().getQualifiedName().split("[.]");
        String testName = resultDataArray[resultDataArray.length - 2] + "." + resultDataArray[resultDataArray.length - 1];
        return testName;
    }

This above method would return us className.methodName at runtime.

e.g., (Taking above class into consideration)

TestClass.validTestMethod will be our testName

and our test method would look like

import com.aventstack.extentreports.ExtentReports;
import com.aventstack.extentreports.ExtentTest;
import com.aventstack.extentreports.Status;
import com.aventstack.extentreports.reporter.ExtentSparkReporter;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class TestClass {

    @Test
    public void validTestMethod(){
        int exp = 10, act = 10;
        // logging result
        extentTest.log(exp==act?Status.PASS:Status.FAIL,"Expected: "+exp+" Actual: "+act);
    }
    @Test
    public void inValidTestMethod(){
        int exp = 10, act = 12;
        // logging result
        extentTest.log(exp==act?Status.PASS:Status.FAIL,"Expected: "+exp+" Actual: "+act);
    }
}

now we no longer have to call createTest().

You may still be unclear on what to write where? Let me add a few classes here!

TestListener.java

ITestListener implemented class

package io.github.arvind142.framework.listeners;

import com.aventstack.extentreports.Status;
import io.github.arvind142.framework.constants.IconConstants;
import io.github.arvind142.framework.reporter.TestReporter;
import lombok.extern.slf4j.Slf4j;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;

@Slf4j
public class TestListener implements ITestListener {

    public void onTestStart(ITestResult result) {
        log.trace("onTestStart");
        TestReporter.createTest(result);
    }

    public void onTestSuccess(ITestResult result) {
        log.trace("onTestSuccess");
        TestReporter.log(Status.PASS, IconConstants.TestStatus.getStatusIcon(Status.PASS)+" Test Pass");
    }

    public void onTestFailure(ITestResult result) {
        log.trace("onTestFailure");
        TestReporter.logError(result.getThrowable());
        TestReporter.log(Status.FAIL, IconConstants.TestStatus.getStatusIcon(Status.FAIL)+" Test Fail");
    }

    public void onTestSkipped(ITestResult result) {
        log.trace("onTestSkipped");
        TestReporter.log(Status.SKIP, IconConstants.TestStatus.getStatusIcon(Status.SKIP)+" Test Skip");
    }

    public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
        log.trace("onTestFailedButWithinSuccessPercentage");
    }

    public void onTestFailedWithTimeout(ITestResult result) {
        this.onTestFailure(result);
    }

    public void onStart(ITestContext context) {
        log.trace("onStart");
        TestReporter.init(context);
    }

    public void onFinish(ITestContext context) {
        log.trace("onFinish");
        TestReporter.flushReporting(context);
    }
}

TestReporter:

Class responsible for test reporting

package io.github.arvind142.framework.reporter;

import com.aventstack.extentreports.ExtentTest;
import com.aventstack.extentreports.Status;
import com.aventstack.extentreports.markuputils.CodeLanguage;
import com.aventstack.extentreports.markuputils.ExtentColor;
import com.aventstack.extentreports.markuputils.Markup;
import com.aventstack.extentreports.markuputils.MarkupHelper;
import io.github.arvind142.framework.annotation.TestInfo;
import io.github.arvind142.framework.constants.FrameworkConstants;
import io.github.arvind142.framework.constants.HTMLConstants;
import lombok.extern.slf4j.Slf4j;
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.annotations.Test;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

@Slf4j
public class TestReporter {
    private static ThreadLocal<ExtentTest> extentTestThreadLocal = new ThreadLocal<>();

    private static Reporter reporter;


    private static Map<String,Integer> listOfTCs = new ConcurrentHashMap<>();

    public static void init(ITestContext iTestContext){
        reporter = Reporter.init();
        setTriggerDetails(iTestContext);
    }

    public static void createTest(ITestResult result){
        extentTestThreadLocal
                .set(
                        reporter.getExtentReports().createTest(
                                getTestName(result),
                                getTestDescription(result)
                        )
                );
    }

    public static void assignDevice(String device){
        extentTestThreadLocal.get().assignDevice(device);
    }

    public static void flushReporting(ITestContext iTestContext){
        reporter.flushReporting();
    }

    /*
    ################# Logging methods #################
     */
    public static void logError(Throwable throwable){
        extentTestThreadLocal.get().log(Status.FAIL,throwable);
    }

    public static void log(Status status,String message){
        extentTestThreadLocal.get().log(status,message);
    }
    public static void log(Status status, Markup markup){
        extentTestThreadLocal.get().log(status,markup);
    }
}

TestBase.java

TestBase which should be initialized by Runner classes

package io.github.arvind142.base;

import io.github.arvind142.framework.listeners.TestListener;
import io.github.arvind142.framework.listeners.XMLTestReporter;
import org.testng.annotations.Listeners;

@Listeners({TestListener.class})
public class TestBase {

}

TestRunner.java

Runner class where test methods are written. You’ll not find any call for createTest() method call, because test case name is fetched at runtime.

package io.github.arvind142.runnerClasses;

import io.github.arvind142.base.TestBase;
import io.github.arvind142.framework.annotation.TestInfo;
import io.github.arvind142.framework.builder.DriverBuilder;
import io.qameta.allure.Flaky;
import org.testng.SkipException;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

import static org.testng.Assert.assertEquals;

public class TestRunner extends TestBase {

    @DataProvider(name = "dp")
    public Object[][] dp(){
        return new Object[][]{
                {"1","1"},
                {"2","0"},
                {"3","3"}
        };
    }

    @Test(dataProvider = "dp")
    public void test(String a,String b) {
        assertEquals(a, b);
    }

    @Test
    @TestInfo(author = "Arvind Choudhary", testName = {"Test2","TEst3"})
    public void test2() {
        DriverBuilder.init();
        assertEquals(0, 0);
    }


    @Test
    @TestInfo(author = "Arvind Choudhary", testName = "Test3",testDescription = "abc")
    public void test3() {
        DriverBuilder.init();
        throw new SkipException("Skip");
    }
}

Given current implementation, We will get class name and method name at runtime but just those are not sufficient and the reason behind that is following attribute of @Test method:

dataProvider
invocationCount

which would execute same test method again and again.

Let’s tackle parameterization first

It would hard to differentiate between what was executed with which set of data given current set of code to get class name, method name & first parameter as test name if dataProvider annotation is used.

Before Report:

All Test have the same Test Name, which makes it hard to differentiate.

Let’s rewrite our getTestName method

    private static String getTestName(ITestResult iTestResult){
        String[] resultDataArray = iTestResult.getMethod().getQualifiedName().split("[.]");
        String testName = resultDataArray[resultDataArray.length - 2] + "." + resultDataArray[resultDataArray.length - 1];

        // get parameters if any :)
        List<String> paramString = getParameter(iTestResult);
        if(!paramString.isEmpty()){
            // returning test name
            testName=(testName + " - [ " + paramString.stream().toArray()[0]+" ]");
        }
        return testName;
    }

Now our method checks for parameters if there are any, it would get the first parameter and append it to test name.

After Report:

Now as you see it is easier to identify which test was run with which test parameter

Invocation Count Solution

How do we identify which invocation failed? Let's add invocation count in test name if invocation count attribute is used.

//Test Method

    @Test(invocationCount = 2)
    @TestInfo(author = "Arvind Choudhary", testName = {"Test2","TEst3"})
    public void test2() {
        DriverBuilder.init();
        assertEquals(0, 0);
    }

Before Report:

All Test have the same Test Name, which makes it hard to differentiate.

Let’s rewrite our getTestName method

    private static String getTestName(ITestResult iTestResult){
        String[] resultDataArray = iTestResult.getMethod().getQualifiedName().split("[.]");
        String testName = resultDataArray[resultDataArray.length - 2] + "." + resultDataArray[resultDataArray.length - 1];

        // get parameters if any :)
        List<String> paramString = getParameter(iTestResult);
        if(!paramString.isEmpty()){
            // returning test name
            testName=(testName + " - [ " + paramString.stream().toArray()[0]+" ]");
        }

        // is test invoked again?
        if(listOfTCs.containsKey(testName)){
            listOfTCs.replace(testName,listOfTCs.get(testName)+1);
            testName = testName+" ( Invocation: "+listOfTCs.get(testName)+")";
        }
        else{
            listOfTCs.put(testName,1);
        }
        return testName;
    }