Selenium Java (maven project): TestNG results differs from ReportNG
As your tests got skipped, you may want to check test results xml first to find out the root cause (exception).
On the other hand, I'm just wondering what's the initial intention of using a library, which is not maintained for 4 years? ReportNG is dead. It's a fact. And if you take a look at its sources, you'll see which TestNG version it uses (6.8). You could easily assume that its reporter is based on old TestNG API.
When you add TestNG 6.11 as a dependency, the probability of a jar hell occurrence is quite high. And I wouldn't be surprised, if it's a root cause of your issue.
If you haven't decided which reporting system to use yet, I'd recommend to take a look at a modern Allure 2 framework.
Otherwise, just check a stacktrace first to understand the root cause of your issue.
I was able to resolve this using the specified versions.
By default the ReportNG reports are generated under target/surefire-reports folder as configured by surefire plugin. This is something that can be changed via surefire configuration.
You can add any test classes, but for the sake of ease, I am just including pom file and two sample classes that I have used.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.rationaleemotions</groupId> <artifactId>46923243</artifactId> <version>1.0-SNAPSHOT</version> <packaging>jar</packaging> <name>46923243</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <maven.compiler.source>1.8</maven.compiler.source> <maven.compiler.target>1.8</maven.compiler.target> </properties> <dependencies> <!-- https://mvnrepository.com/artifact/velocity/velocity-dep --> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-java</artifactId> <version>3.5.3</version> </dependency> <!-- https://mvnrepository.com/artifact/com.google.inject/guice --> <dependency> <groupId>com.google.inject</groupId> <artifactId>guice</artifactId> <version>3.0</version> </dependency> <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.12</version> <scope>test</scope> </dependency> <!-- https://mvnrepository.com/artifact/org.uncommons/reportng --> <dependency> <groupId>org.uncommons</groupId> <artifactId>reportng</artifactId> <version>1.1.4</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20.1</version> <configuration> <!-- Suite testng xml file to consider for test execution --> <suiteXmlFiles> <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile> </suiteXmlFiles> <properties> <property> <name>usedefaultlisteners</name> <value>false</value> </property> </properties> </configuration> </plugin> </plugins> </build> <repositories> <repository> <id>jcenter</id> <name>jcenter</name> <url>http://jcenter.bintray.com</url> </repository> </repositories></project>The test classes, that I used
package com.rationaleemotions;import org.openqa.selenium.WebDriver;import org.openqa.selenium.chrome.ChromeDriver;import org.openqa.selenium.chrome.ChromeOptions;import org.testng.Assert;import org.testng.annotations.AfterClass;import org.testng.annotations.BeforeClass;import org.testng.annotations.Test;import java.util.concurrent.TimeUnit;public class MwSites { private WebDriver driver; @BeforeClass public void setup() { ChromeOptions options = new ChromeOptions(); options.addArguments("test-type"); options.addArguments("start-maximized"); options.addArguments("--js-flags=--expose-gc"); options.addArguments("--enable-precise-memory-info"); options.addArguments("--disable-popup-blocking"); options.addArguments("--disable-default-apps"); options.addArguments("test-type=browser"); options.addArguments("disable-infobars"); driver = new ChromeDriver(options); driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS); } @AfterClass public void Quit() throws InterruptedException { driver.quit(); } @Test public void MI_Demo() throws InterruptedException { driver.navigate().to("http://demo.movingwalls.com/mi/#/login"); Assert.assertEquals("Login", driver.getTitle()); if ("Login".equals(driver.getTitle())) { System.out.println("Failed to access MI in demo environment"); } else { System.out.println("MI is successfully accessed in demo environment"); } }}package com.rationaleemotions;import org.testng.Assert;import org.testng.Reporter;import org.testng.annotations.Test;public class TestClassWithMultipleResults { @Test public void testMethodWillPass() { Reporter.log("This test method will pass", true); } @Test public void testMethodWillFail() { Reporter.log("This test method will fail", true); Assert.fail(); } @Test(dependsOnMethods = "testMethodWillFail") public void testMethodWillSkip() { }}Suite xml that I used:
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd"><suite name="Suite"> <listeners> <listener class-name="org.uncommons.reportng.HTMLReporter"/> <listener class-name="org.uncommons.reportng.JUnitXMLReporter"/> </listeners> <test name="Test"> <classes> <class name="com.rationaleemotions.MwSites"/> <class name="com.rationaleemotions.TestClassWithMultipleResults"/> </classes> </test> <!-- Test --></suite> <!-- Suite -->Here are the screenshots from the ReportNG reports
