IRetryAnalyzer in TestNG
IRetryAnalyzer tells TestNG to re-run a failed test automatically before recording it as failed. Essential for handling flaky tests in CI pipelines.
Step 1 — Implement IRetryAnalyzer
Java
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class RetryAnalyzer implements IRetryAnalyzer {
private int retryCount = 0;
private static final int MAX_RETRY = 2; // retry up to 2 times (3 total attempts)
@Override
public boolean retry(ITestResult result) {
if (retryCount < MAX_RETRY) {
System.out.println("Retrying test: " + result.getName()
+ " | Attempt: " + (retryCount + 1));
retryCount++;
return true; // retry
}
return false; // stop retrying
}
}
Step 2 — Apply to a Single Test
Java
@Test(retryAnalyzer = RetryAnalyzer.class)
public void flakeyLoginTest() {
// If this fails, TestNG retries up to 2 more times
Assert.assertTrue(driver.getTitle().contains("Dashboard"));
}
Step 3 — Apply to ALL Tests via Listener
Java
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
public class RetryListener implements IAnnotationTransformer {
@Override
public void transform(ITestAnnotation annotation, Class testClass,
Constructor testConstructor, Method testMethod) {
// Apply RetryAnalyzer to every @Test method automatically
annotation.setRetryAnalyzer(RetryAnalyzer.class);
}
}
testng.xml:
XML
<suite name="Suite">
<listeners>
<listener class-name="listeners.RetryListener"/>
</listeners>
<test name="Tests">
<classes>
<class name="tests.LoginTest"/>
</classes>
</test>
</suite>
Clean Up: Skip Retried Tests in Report
By default, TestNG marks retried attempts as SKIP in reports, cluttering the output. Fix:
Java
public class RetryReportListener implements ITestListener {
@Override
public void onTestSkipped(ITestResult result) {
// If skip was caused by retry, remove from skipped count
if (result.getMethod().getRetryAnalyzerClass() != null) {
Reporter.log(result.getName() + " was retried, not truly skipped");
// Can also programmatically remove from reporter
}
}
}
How Retry Works
CODE
Test FAILS on attempt 1 → retry() returns true → attempt 2
Test FAILS on attempt 2 → retry() returns true → attempt 3
Test FAILS on attempt 3 → retry() returns false → FAIL recorded
OR
Test PASSES on attempt 2 → no more retries → PASS recorded
When to Use (and When NOT To)
- ✓✓ Flaky network-dependent tests in CI
- ✓✓ Tests with timing sensitivity
- ✓✗ Do NOT use to hide real bugs — fix root cause instead
- ✓✗ Do NOT use for all tests — only truly flaky ones
