Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In the load pipeline, separate the network request from the post-download CPU work, and throttle them differently #473

Open
nithinp7 opened this issue Mar 29, 2022 · 2 comments · May be fixed by #779
Labels
enhancement New feature or request performance Improvements to performance, including reductions in memory usage

Comments

@nithinp7
Copy link
Contributor

I am not sure how much of an impact it has, but our load thread pool is usually saturated by long network requests. Tiles that can be retrieved from the SQL cache however are extremely fast. This creates a conflict where cache retrievals are inevitably waiting on a load thread to finish resolving a network request. I think it ends up being bad scheduling to have very long-running tasks and very fast tasks in the same threads. If we can dedicate a thread pool to only process cache hits, cache loads may become even faster. The network load pool and cache load pool may need to be dynamically sized to not introduce any new bottlenecks.

@kring
Copy link
Member

kring commented Mar 29, 2022

@nithinp7 load/worker threads shouldn't be blocking waiting for network requests. If you're seeing that, let me know, because it would be extremely concerning.

@kring
Copy link
Member

kring commented Mar 29, 2022

Load threads aren't blocked, but there is that limit of 20 (by default) simultaneous loads, and there's no distinction there between loads from cache and loads from the network. So I guess we can have the situation you're referring to, where tiles that would be quick to load from cache have to wait their turn behind slower requests from the network.

I think the proper solution is to split the network load and the CPU work we do after we receive the response into separate tasks. Loading 20 tiles from the network simultaneously is fairly reasonable (on a decently fast connection, at least), but 20 worker threads doing CPU work at once is too much. In CesiumJS, for comparison, there is some browser dependent number of simultaneous network requests (probably 10 ) but only a single thread doing CPU load work and only 5 tiles can be in the queue at once.

@kring kring changed the title Dedicated thread pool for retrieving cached tiles In the load pipeline, separate the network request from the post-download CPU work, and throttle them differently May 9, 2022
@kring kring added the enhancement New feature or request label May 9, 2022
@kring kring added the performance Improvements to performance, including reductions in memory usage label May 31, 2022
@csciguy8 csciguy8 linked a pull request Jan 16, 2024 that will close this issue
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance Improvements to performance, including reductions in memory usage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants