TestNG doesn't search the directories specified in testng.test.classpath property

[longhua]

Issue

The following is the explanation of testng.test.classpath from TestNG document.

If this property is set, TestNG will use it to look for your test classes instead of the class path. This is convenient if you are using the package tag in your XML file and you have a lot of classes in your classpath, most of them not being test classes.

However, it doesn't search the directory specified in testng.test.classpath during runtime. So no test cases are run.

In my test, there are two test classes in package test and testng.xml is as follow.

<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="testing">
    <test name="suite">
        <packages>
            <package name="test"/>
        </packages>
    </test>
</suite>

Steps to reproduce

1. Check out this project https://github.com/longhua/testng-test-classpath-property-issue-1.
2. Add all TestNG required JAR's to CLASSPATH.
3. Create out directory: mkdir out
4. Compile the test classes: javac src/test/*.java -d out
5. Run the test: java -cp $CLASSPATH:out org.testng.TestNG testng.xml
6. Rerun the test with testng.test.classpath property: java -Dtestng.test.classpath="out/" org.testng.TestNG testng.xml

Observation: Four tests are run in Step 3. However, no tests are run in Step 4.

[austin-bowen]

I was running into the same problem, but I think I figured out how to get it to work.

The key for me was to use the full path in the -Dtestng.test.classpath="<test-classpath>" argument.

Also, in the "Steps to reproduce" above, I think another problem is that you still need the test classpath in the "regular" Java classpath. Try something like this:

java -cp path/to/src;path/to/test -Dtestng.test.classpath=/FULL/path/to/test org.testng.TestNG testng.xml

This should still be considered a bug. The "regular" Java classpath works with relative paths, and so should this imo.

I also found that the -Dtestng.test.classpath setting has a couple other problems:

1. It does not like UNC notation on Windows -- e.g., X:\path\to\test will work, but \\server\share\path\to\test will not.
2. The paths are case-sensitive. This is problematic when the filesystem is not case sensitive, i.e. on Windows.

Note: My tests were with TestNG 7.4.0 and Java 17 on Windows.