Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty test class doesn't show warning, but breaks CI by passing on non-zero STDERR from PHPUnit #293

Open
selfsimilar opened this issue Nov 8, 2023 · 4 comments

Comments

@selfsimilar
Copy link

If there are no tests in a test class file, PHPUnit raises a warning (Event\Facade::emitter()->testRunnerTriggeredWarning() - 'no tests found in class') and will pass on a non-zero value to STDERR. Collision will not display this warning, but does pass on the STDERR value, which led to some confusion when it appeared that all tests were passing, but our CI pipeline was stopping at the test step in failure. However, narrowing it down to this issue took a while as at first I thought it was a bug in the CI runner.

You can recreate this with a default Laravel install:

composer create-project laravel/laravel empty-class
cd empty-class
php artisan test
# All tests pass
echo $?
# STDERR is 0
cat <(echo "<?php\nnamespace Tests\\\\Unit;\nuse PHPUnit\Framework\TestCase;\n\nclass EmptyClassTest extends TestCase{}") > tests/Unit/EmptyClassTest.php
php artisan test
# All tests pass
echo $?
# STDERR is 1
vendor/bin/phpunit
# Warnings!

It's not clear to me if this is limited to this one 'empty class' issue, or if all testRunnerTriggeredWarning events are ignored.

@selfsimilar
Copy link
Author

This also affects downstream test packages like Dusk.

laravel/dusk#1063

@RCady
Copy link

RCady commented Nov 10, 2023

This also affects the --coverage option. No coverage output is generated with empty test classes present.

@selfsimilar selfsimilar changed the title Empty test class doesn't show warning, but breaks CI by passing on non-zero STDERR from PHPHUnit Empty test class doesn't show warning, but breaks CI by passing on non-zero STDERR from PHPUnit Nov 15, 2023
@abobija
Copy link

abobija commented Nov 23, 2023

Hi there,
Just wanted to follow up on this. I've also encountered the same issue.

Do we have any updates on the plans for a fix? Additionally, I'm curious about the status of merging/releasing #294. Could someone also explain why this warning was initially hidden/suppresed? Thanks!

@abobija
Copy link

abobija commented Dec 13, 2023

Bump 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants