Improve preloading-request logic #2362
Conversation
| let blockNum = blockedRaw |> Seq.sumBy (fun (_, num, _) -> num) | ||
| let blockPaket4 = TimeSpan.FromMilliseconds(500.0 * float blockNum) | ||
| tracefn " - Runtime Paket 4 (estimated ~500ms respose*): %s" (Utils.TimeSpanToReadableString (realTime blockPaket4)) | ||
| tracefn " * See http://stats.pingdom.com/aqicaf2upspo/1265300 for average response times." |
There was a problem hiding this comment.
I don't think its useful going forward.
| tracefn " - Blocked (%s): %s (%d times)" reason (Utils.TimeSpanToReadableString elapsed) num | ||
| | Profile.Category.ResolverAlgorithmNotBlocked b -> | ||
| let reason = reason b | ||
| tracefn " - Not Blocked (%s): %d times" reason num |
There was a problem hiding this comment.
This is an indication in how well pre-loading requests (or caching - but we should track that differently) work
|
This is 50% slower for me... |
| work.Priority <- prio | ||
| member x.Task = | ||
| let { TaskSource = task } = x | ||
| task.Task |
There was a problem hiding this comment.
@tpetricek Do you have an idea how I can achieve something like the above.
I basically want a queue but it's critical to re-prioritize some work of it on-demand.
Basically the paket resolver is queuing up work before it actually needs it, but when it needs the info the resolver blocks and therefore the work should be started immediately if not started already. So instead of re-prioritizing another way would be to take the work out of the queue and start it on the resolver directly.
Re-prioritizing generally has the nice property that we can further improve that in the future (ie the more insights the resolver has the more it changes the priority of some work, or even cancels it).
I think the above is a good start, but has some things I don't like:
- It needs to use sleep so in worst case everything is delayed all the time (I think that's the reason why it's slower than before)
- When the queue is close to empty a lot of workers go into this bad looking fold (minBy with throw errors...)
If you know other people who might be able to help please ping them. I have ideas but most of them will result in several hundred lines of code and are very hard to get right :/
There was a problem hiding this comment.
Ok nevermind, I think I found a better approach...
…ints 60k lines for not resolving dependency files.
|
Sorry @zzDavid I couldn't figure out why the test explodes. I disabled this code place for now, because it doesn't seem to report the correct thing while the resolver is trying to resolve conflicts... |
|
Only red because of mono crash :/ |
This hopefully fixes the flawky test.