Picture8

How to Execute failed Test cases?

Posted by

While executing the automation scripts, test cases may fail for several reasons. To optimize our next runs, we need to re-run only failed test cases. How to Execute failed Test cases? What is the best approach? In TestNg class, we can easily re-run the test cases using two methods as explained below:

Method 1:  By using testng-failed.xml file in test-output folder.

Method 2:  By implementing TestNG IRetryAnalyzer.

In this article, we are going to discuss both the approaches in details. If you are new to TestNG, read the TestNG tutorial, for understanding the basics of TestNG framework.

Let’s have a look at each of the methods in detail:

Method 1: Executing failed testcase(s) using testng-failed.xml.

Please follow the below steps for this method:

  1. Create a java class
1

2. Create testng.xml file under project folder.

3. Right click on the testng.xml >> Run As >> TestNG suite.

4. Once the execution is done, automatically test-output folder will be created in the project folder.

3

5. In the test-output folder >> testng-failed.xml file will be created.

5

6. Right click on testng-failed.xml>> Run as TestNg. This will now only re-run the fail testcase(s).

Disadvantage of the first method is we can’t execute single fail testcase multiple times. For that in second method we can easily retry multiple time fail testcases.

Method 2:  By implementing TestNG IRetryAnalyzer

There are many reasons for testcases to fail. Some of them could be:

  • Element not visible,
  • Stale element,
  • Time out exception etc.

If testcases have failed for above reasons, then we can easily re-run such testcases in TestNg by implementing the IRetryAnalyzer class. The steps are explained further below.

import org.testng.IRetryAnalyzer;

import org.testng.ITestResult;

public class RetryFailedTestCases implements IRetryAnalyzer {

    private int retryCnt = 0;

    //You can mention maxRetryCnt (Maximium Retry Count) as per your requirement. This example uses 2 which means that for any failed testcases it retires the run twice.

    private int maxRetryCnt = 2;

    //This method will be called everytime a test fails. It will return TRUE if a test fails and need to be retried, else it returns FALSE

    public boolean retry(ITestResult result) {

        if (retryCnt < maxRetryCnt) {

            System.out.println(“Retrying ” + result.getName() + ” again and the count is ” + (retryCnt+1));

            retryCnt++;

            return true;

        }

        return false;

    }

   }

Create another class RetryListenerClass to implement ‘IAnnonation Transformer’ interface. The implementation of ‘IAnnotationTransformer’ interface may help us set the ‘setRetryAnalyzer’ for ‘ITestAnnotation.

IRetryListerner implements IAnnoantion:

import java.lang.reflect.Constructor;

import java.lang.reflect.Method;

import org.testng.IAnnotationTransformer;

import org.testng.IRetryAnalyzer;

import org.testng.annotations.ITestAnnotation;

public class RetryListenerClass implements IAnnotationTransformer {

 @Override

 public void transform(ITestAnnotation testannotation, Class testClass, Constructor testConstructor, Method testMethod) {

 IRetryAnalyzer retry = testannotation.getRetryAnalyzer();

 if (retry == null) {

 testannotation.setRetryAnalyzer(RetryFailedTestCases.class);

 }

  } }

Create one sample class with two test methods:

package com;

import org.openqa.selenium.By;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.WebElement;

import org.openqa.selenium.chrome.ChromeDriver;

import org.openqa.selenium.firefox.FirefoxDriver;

import org.testng.Assert;

import org.testng.annotations.BeforeClass;

import org.testng.annotations.Test;

public class TestNGExampleTests {

            WebDriver driver;

            String baseURL = “https://www.gmail.com/”;

            @BeforeClass

            public void setup() {

System.setProperty(“webdriver.chrome.driver”, “G:\\SampleWorkSpace\\Sample\\Jar\\chromedriver.exe”);

                        driver = new ChromeDriver();

                        driver.manage().window().maximize();

            }

            @Test(priority=1)

            public void verifycurrentUrl() {

                        driver.navigate().to(baseURL);

                        String ActualResult=driver.getCurrentUrl();

                        System.out.println(ActualResult);

                        String ExceptedResult = “https://accounts.google.com/signin/v2/identifier?service=mail&passive=true&rm=false&continue=https%3A%2F%2Fmail.google.com%2Fmail%2F&ss=1&scc=1&ltmpl=default&ltmplcache=2&emr=1&osid=1&flowName=GlifWebSignIn&flowEntry=ServiceLogin”;

                        Assert.assertEquals(ActualResult, ExceptedResult);

            }

            @Test(priority=2)

            public void VerifyCurrentURl() throws InterruptedException

            {

                        driver.navigate().to(baseURL);

                        Thread.sleep(10000);

                        driver.findElement(By.linkText(“Sign in”)).click();

                        Thread.sleep(5000);

            WebElement element = driver.findElement(By.xpath(“/html/body/div[1]/div[1]/div[2]/div[2]/div/div/div[2]/div/div[1]/div/form/span/section/div/div/div[3]/button”));

                        String ActualResult = element.getText();

                        Assert.assertEquals(ActualResult, “Forgot email?”);

            }         

}

In the above class, Test 2 method is getting failed and it will retry 2 times in Retry class. We need to add below code in testng.xml file.

<listeners>

        <listener class-name=”com.RetryListener”/>

  </listeners>

Sample testng.xml file:

<!DOCTYPE suite SYSTEM “http://testng.org/testng-1.0.dtd” >

<suite name=”Sample Test Suite” verbose=”1″ >

<listeners>

        <listener class-name=”com.RetryListenerClass”/>

  </listeners>

  <test name=”VerifycurrentUrl()” >

   <classes>

       <class name=”com.TestNGExampleTests” />

    </classes>

        </test> <!– Test –>

    <test name=”VerifyText()” >

   <classes>

       <class name=”com.TestNGExampleTests” />

    </classes>

       </test>

</suite>

Once you execute above testng.xml file you will get below result:

Conclusion

In this way we can execute fail testcases in TestNG class.

What’s Next?

In this tutorial, we explained how to execute failed test cases. You can go and read more Selenium tutorials published on this blog.



These articles are written under the guidance of experienced Selenium professionals who also teach at Techcanvass. They use real-life challenges in the training programs. You can get trained by these seasoned professionals by joining our Selenium Certification Training or CP-SAT certification Training program.



Want to become an Automation tester?

Techcanvass offers Automation Testing and Business Analysis certification courses. We are an IIBA endorsed education provider (EEP), iSQI Germany Authorized Training as well as Agile Testing alliance partner for CP-SAT certification training in Selenium.

We offer Selenium training with JavaSelenium Training with PythonCP-SAT certification Training and many short courses on Automation – BDD Automation testing with CucumberContinuous Integration Training with JenkinsAutomation testing Frameworks Training and Continuous Integration with Jenkins, Maven and GitHub.

One of the key features of our courses is the Job readiness package, as shown below:

Selenium-Certification-training-with-Job-readiness