Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: invalid dataset not showing as FAILED when running with --parallel #1185

Open
mtbossa opened this issue Jun 29, 2024 · 2 comments
Open
Labels

Comments

@mtbossa
Copy link

mtbossa commented Jun 29, 2024

What Happened

When I have a test that uses a dataset ->with('my dataset'), but this dataset doesn't exist, and I run sail test --parallel, no FAILED message is shown in the output. However, if I just run sail test, then it'll show as FAILED, so I know something is wrong.

How to Reproduce

Install a fresh Laravel App, add Pest, add a test that uses a non-existent dataset, run ./vendor/bin/pest --parallel.

Sample Repository

https://github.com/mtbossa/pest-parallel-bug-laravel

Pest Version

2.34.8

PHP Version

8.3.8

Operation System

Linux

Notes

Screenshot_20240629_082424
Screenshot_20240629_082401

@mtbossa mtbossa added the bug label Jun 29, 2024
@mtbossa mtbossa changed the title [Bug]: invalid datasets not showing as FAILED when running with --parallel [Bug]: invalid dataset not showing as FAILED when running with --parallel Jun 29, 2024
@FrittenKeeZ
Copy link

FrittenKeeZ commented Sep 27, 2024

I'm experiencing the same error - when the fault is introduced it seems like the tests are just skipped.

This is the output for the suite before introducing the error.

Tests:    1190 passed (5990 assertions)
Duration: 3.83s
Parallel: 11 processes

This is the output for the suite after introducing the error.

Tests:    1062 passed (5862 assertions)
Duration: 3.88s
Parallel: 11 processes

It looks like 128 tests have disappeared, but no failures at all.
Running the tests without --parallel will instantly crash with the error being outputted.

Edit: I'm using Pest 3.2

@mtbossa
Copy link
Author

mtbossa commented Sep 27, 2024

I'm experiencing the same error - when the fault is introduced it seems like the tests are just skipped.

This is the output for the suite before introducing the error.

Tests:    1190 passed (5990 assertions)
Duration: 3.83s
Parallel: 11 processes

This is the output for the suite after introducing the error.

Tests:    1062 passed (5862 assertions)
Duration: 3.88s
Parallel: 11 processes

It looks like 128 tests have disappeared, but no failures at all. Running the tests without --parallel will instantly crash with the error being outputted.

Edit: I'm using Pest 3.2

Yes. I used an inexistent dataset in the example, but it also happens if the test fails. This is pretty huge, actually. We need to choose between running with --parallel, but being aware that we won't know if dataset tests have failed. Or run it without --parallel flag, which makes testing very slow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants