Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimized threadpool #12

Open
richardbiely opened this issue Oct 26, 2023 · 4 comments
Open

Optimized threadpool #12

richardbiely opened this issue Oct 26, 2023 · 4 comments
Assignees
Labels
enhancement New feature or request

Comments

@richardbiely
Copy link
Owner

No description provided.

@richardbiely richardbiely converted this from a draft issue Oct 26, 2023
@richardbiely richardbiely added the enhancement New feature or request label Nov 15, 2023
@RazielXYZ
Copy link

I have a couple questions about the design of the threadpool that somewhat tie into potential future optimizations too:
First off, why is it limited to 31 (or 32 with the main thread) threads? If it's because of the processor group limit on windows (wherein one group is at most 64 logical processors and therefore 32 physical cores), would it not be desirable to add support for spanning threads across multiple processor groups?
Secondly, why is the thread pool an explicit singleton? This design/architecture of this threadpool seems like it could work very well in uses like event loop handlers or reactors that want their own pool, if multiple pools could be created and customized.

@richardbiely
Copy link
Owner Author

richardbiely commented Mar 1, 2024

No real reason for it. The project is still very early on and some features will get polished over time.

The current idea was getting something reasonable done that would suit the needs of the project and could be built upon later (e.g. parallel/async execution will need to be supported natively by querries at some point).

Getting a threading API right is not something that is done over night.

I am expecting things to change as the project evolves and adapts to various needs which users have.

Any suggestions are welcome, of course.

@RazielXYZ
Copy link

That's fair! I do think it would be interesting and potentially useful to make the threadpool more flexible like that (by not forcing it to be a singleton), but at that point it's not necessarily in service of the ECS exclusively, and it also makes it easier for users to misuse it, so a bit of a trade-off.
As for the thread limit and especially the mention of processor groups, it was more a personal curiosity, as I've noticed a lot of other popular thread pool and task executor framework implementations don't handle those either (and as someone that does use a windows machine with more than one processor group, I wish more did!)

@richardbiely
Copy link
Owner Author

richardbiely commented Mar 13, 2024

Ability to properly use multiple CPU groups and also big.little architectures is one of the goals.

Some groundwork has already been laid in 0.8.4 via introduction of task priorities (high for perf cores, low for eco cores or downclocked perf cores). This makes use of QoS on platforms that support it (win, macos) and scheduling priorities where it is not.

In the future I would like to build on this system further.

@richardbiely richardbiely moved this from Todo to In Progress in Gaia-ECS Nov 24, 2024
@richardbiely richardbiely self-assigned this Nov 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: In Progress
Development

No branches or pull requests

2 participants