How to Retry Failed Test Cases in Selenium using TestNG?

We have discussed lots more features of TestNG so far. Today we are going to discuss one of the most important features of the TestNG which helps to retry failed test cases. You must be happy as this technique will automatically trigger the failed test cases only, so you do not need to bother about sorting failed test cases and running it manually. At first, let’s discuss some of the reasons for the failure of the test case.

Why do test cases fail in Automation run?

If the reason of failure is not an authentic failure reason, then the following reasons could be the one:

• Network downtimes
• The browser didn’t open
• Web server didn’t respond
• Network is slow
• The page is taking time to load
• Elements are not visible
• Web page loaded but UI messed up

These are some of the reasons for test case failure when failure is not about a mismatch in expected and actual results. It’s worthless if you manually trigger tests when failed due to above reasons. We have observed that above test failure reasons generally occur at a transient time, hence, if we try to execute test again then there are positive chances that the test will get passed or if failed then the reason would the authentic one. Therefore, to reduce the number of test failure by triggering the retry of failed test cases is one of the goals to implement this technique in our Selenium project.

How to Retry Failed Test Cases?

Now the question comes how can we retry failed test cases?

We have two solutions to this question and TestNG better addresses the solution. But, before we discuss the techniques, I’d like to recommend you our tutorial on TestNG Listener. Click below to read the tutorial.

• How to implement TestNG Listener in the Selenium project?

Technique# 1: By running testng-failed.xml file

When any test gets failed then TestNG create a separate XML file in which it lists all the failed tests. A folder gets added to our Selenium project after the run and the folder name is, test-output. When you expand this folder, you find a file called testng-failed.xml. Here is the folder structure:

You just run this XML file like you run your testng.xml and you will see all the failed test cases start running. Isn’t it interesting? 😊

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Failed suite [Suite]">
  <listeners>
    <listener class-name="Test.ListenersDefinitionClass"/>
    <listener class-name="Test.AnnotationTransformer"/>
  </listeners>
  <test name="Test(failed)">
    <classes>
      <class name="Test.TestngRetryFailedTestDemo">
        <methods>
          <include name="failTest"/>
        </methods>
      </class> <!-- Test.TestngRetryFailedTestDemo -->
    </classes>
  </test> <!-- Test(failed) -->
</suite> <!-- Failed suite [Suite] -->

Technique# 2: By implementing IRetryAnalyzer

What is IRetryAnalyzer?

IRetryAnalyzer is an interface in which there is definition to trigger failed test cases found. We can further expand that method which triggers failed test cases through the implementation of the IRetryAnalyzer interface.

Here is the sample implementation:

package Test;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryFailedTestSettings implements IRetryAnalyzer{

	public boolean retry(ITestResult result) {
		// TODO Auto-generated method stub
		return false;
	}
	
	
}

Steps to implement the technique to Retry failed test cases through IRetryAnalyzer

Step# 1: Implement IRetryAnalyzer

First, implement IRetryAnalyzer and override the method written inside it. Following is the below code:

package Test;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryFailedTestSettings implements IRetryAnalyzer{
	
	int retryCount = 0;
	int maxRetryCount = 1;

	public boolean retry(ITestResult result) {
		if(retryCount<maxRetryCount){
			System.out.println("Retrying failed tests "+result.getName()+" and the count of the retry is "+(retryCount+1));
			retryCount++;
			return true;
		}

		return false;
	}

}

In the above code, we just define initial retry count and maximum retry count. Further, retry of the failed test cases only occur when retryCount is always less than the maRetryCount.

Step# 2: Implement IAnnotationTransformer

IAnnotationTransformer is another interface which set the value to the previous class where IRetryAnalyzer was implemented. Here is the code:

package Test;

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;

import org.testng.IAnnotationTransformer;
import org.testng.IRetryAnalyzer;
import org.testng.annotations.ITestAnnotation;

public class AnnotationTransformer implements IAnnotationTransformer{
	
	public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {
		IRetryAnalyzer retryAnalyzer = annotation.getRetryAnalyzer();
		if(retryAnalyzer==null){
			annotation.setRetryAnalyzer(RetryFailedTestSettings.class);
		}
		
	}

}

Step 3: Design your tests in TestNG class file

Here is the sample test automation program.

package Test;

import java.util.concurrent.TimeUnit;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.SkipException;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;


public class TestngRetryFailedTestDemo {
	WebDriver driver;
  
  
  @Test
  public void failTest() {
	  System.setProperty("webdriver.chrome.driver", "C:\\Selenium\\chromedriver.exe");
	  	 
  	  driver = new ChromeDriver();
  	  
  	  driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
	  
  	  driver.get("https://www.inviul.com/");
	  
  	  driver.manage().window().maximize();
	  
		 Assert.assertEquals(driver.getTitle(), "Test Title");
		 
			driver.close();
			driver.quit();
	
  }
  
  
}

In this program, we use Assert command to verify the presence of some text in the title of the web page. It will fail on mismatch.

Step# 4: Now add these listeners to testng.xml file

Earlier we have seen how we added listener file in testng.xml file. We can follow similar steps here too. Following is the statement which adds Retry listeners to the testng.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
	<listeners>
		<listener class-name="Test.ListenersDefinitionClass"></listener>
		<listener class-name="Test.AnnotationTransformer"></listener>
	</listeners>
  <test thread-count="2" name="Test" >
    <classes>
      <class name="Test.TestngRetryFailedTestDemo"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

Console Output

In the above console output, we see Retry of failed test cases got triggered.

That’s all about the techniques to retry failed test cases in Selenium using TestNG’s IRetryAnalyzer. You can post your queries in a comment below and don’t forget to join our Facebook group.