Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache refresh memory consumption optimization (#470) #515

Merged
merged 1 commit into from
May 16, 2022
Merged

Cache refresh memory consumption optimization (#470) #515

merged 1 commit into from
May 16, 2022

Conversation

0xERR0R
Copy link
Owner

@0xERR0R 0xERR0R commented May 12, 2022

  • introduced new config parameter "processingConcurrency" to limit working goroutines for cache refresh (Default value 4)
  • rewrite cache refresh logic: collect lines from list files immediately and don"t collect all lines in memory in a slice

closes #470

@0xERR0R 0xERR0R added the 🔨 enhancement New feature or request label May 12, 2022
@0xERR0R 0xERR0R added this to the 0.19 milestone May 12, 2022
@0xERR0R 0xERR0R mentioned this pull request May 12, 2022
Closed
@codecov
Copy link

codecov bot commented May 12, 2022 •
edited
Loading

Codecov Report

Merging #515 (6843517) into development (6772438) will decrease coverage by 0.04%.
The diff coverage is 96.00%.

@@               Coverage Diff               @@
##           development     #515      +/-   ##
===============================================
- Coverage        93.83%   93.78%   -0.05%     
===============================================
  Files               36       36              
  Lines             3081     3089       +8     
===============================================
+ Hits              2891     2897       +6     
- Misses             142      143       +1     
- Partials            48       49       +1
Impacted Files Coverage Δ
config/config.go 98.13% <ø> (ø)
lists/list_cache.go 98.70% <94.73%> (-1.30%) ⬇️
resolver/blocking_resolver.go 98.49% <100.00%> (+0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6772438...6843517. Read the comment docs.

@0xERR0R 0xERR0R merged commit 3392de0 into development May 16, 2022
@delete-merged-branch delete-merged-branch bot deleted the memory-optimization branch May 16, 2022 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
🔨 enhancement New feature or request
Projects
None yet
Milestone
v0.19
Development

Successfully merging this pull request may close these issues.

High memory usage on blacklist refresh
1 participant
@0xERR0R