Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document default parallelizability of STA tests #4833

Open
jnm2 opened this issue Sep 22, 2024 · 7 comments
Open

Document default parallelizability of STA tests #4833

jnm2 opened this issue Sep 22, 2024 · 7 comments
Labels

Comments

@jnm2
Copy link
Contributor

jnm2 commented Sep 22, 2024

Things I couldn"t figure out by reading https://docs.nunit.org/articles/nunit/writing-tests/attributes/parallelizable.html and https://docs.nunit.org/articles/nunit/writing-tests/attributes/apartment.html:

  • Can any STA test run in parallel with any other STA test, and does this happen by default?
  • Can any STA test run in parallel with any MTA test, and does this happen by default?

MTA tests running in parallel with each other by default seems obvious, but it could be stated too.

Also surprising to me is NUnit creating five STA threads. At one point in the past, I had thought there was only one STA queue. I would be interested in knowing if we could document the queues that NUnit uses by default.

@OsirisTerje
Copy link
Member

Agree, we need to get this properly documented.
It is however many variables involved, including which runner is being used and the different settings involved.

As far as I have understood it:

For fixtures:

Can any STA test run in parallel with any other STA test, and does this happen by default?

All tests in a fixture marked as an STA, run sequentially.
Fixtures marked as an STA can run in parallel with other STA fixtures.

Can any STA test run in parallel with any MTA test, and does this happen by default?

I am pretty sure it can, as an STA fixture can run in parallel with any test in a MTA fixture.

For tests:

A test marked as STA will run in its own thread. Multiple tests marked as such should be able to run in parallel with each other, each in their own thread.

@jnm2
Copy link
Contributor Author

jnm2 commented Sep 22, 2024

Interesting. These thread IDs start coming out the same if ParallelScope.Children is used, unless you are debugging:

using NUnit.Framework;
using System.Threading;

[assembly: Parallelizable(ParallelScope.Children)]

public class Fixture
{
    [Test, Apartment(ApartmentState.STA)]
    public void Test1()
    {
        TestContext.WriteLine(Thread.CurrentThread.ManagedThreadId);
    }

    [Test, Apartment(ApartmentState.STA)]
    public void Test2()
    {
        TestContext.WriteLine(Thread.CurrentThread.ManagedThreadId);
    }
}

@jnm2
Copy link
Contributor Author

jnm2 commented Sep 22, 2024

Fixtures marked as an STA can run in parallel with other STA fixtures.

I was not able to reproduce this. The thread ID is the same for me, so the tests can"t be running in parallel:

using NUnit.Framework;
using System.Threading;

namespace DevExpressRepro;

[Apartment(ApartmentState.STA)]
public class Fixture1
{
    [Test]
    public void Test1() => TestContext.WriteLine(Thread.CurrentThread.ManagedThreadId);
}

[Apartment(ApartmentState.STA)]
public class Fixture2
{
    [Test]
    public void Test1() => TestContext.WriteLine(Thread.CurrentThread.ManagedThreadId);
}

(running using ReSharper test runner, NUnit 4.2.2, NUnit3TestAdapter 4.6.0)

@OsirisTerje
Copy link
Member

OsirisTerje commented Sep 28, 2024

I added your repro code above to https://github.com/nunit/nunit.issues/tree/main/Issue4833
And - I added another fixture.

You are correct with ParallelScope.Children, (or All) everything, both fixtures and tests run on the same thread.
If I change to ParallelScope.Fixtures, or Self, everything, both fixtures and tests run in their own separate threads.
What is weird, is that even with ParallelScope.None, they run in their own separate threads,

So I wonder if something is broken wrt to Parallel running and the control of the ParallelScope.

If I remove (comment out) the [assembly: Parallelizable(ParallelScope, they are still running in parallel.

I added in the WorkerId, and that stay the same within the same fixture. I believed that each worker ran in its own thread, and not that a worker could run multiple threads.

@CharliePoole You know more about the design of this. Can you comment?

@OsirisTerje
Copy link
Member

I did some more accurate testing using the stopwatch .

ParallelScope.Fixtures
image

The fixtures run in parallel, the tests do not.

ParallelScope.Children
image

Everything runs sequentially

ParallelScope.Self
image

Everything runs sequentially

So it doesn"t seem like you can run Tests within a Fixture in parallel using STA. I"ve also tried with different optiosn to each fixture and test, with no change.

@CharliePoole
Copy link
Member

@OsirisTerje Keep in mind that I worked on that code four or five years ago. Both the code and I have changed. :-)

However, each worker has a single "worker thread" for running tests but may start a secondary thread for handling timeout. Note that onetime setup and teardown may not necessarily run on the same worker as the test itself.

Unless anything has changed. :-)

@CharliePoole
Copy link
Member

@OsirisTerje IIRC @jnm2 did some work in that area and may have further insights.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants