Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce memory consumption during aggregate state rebuild #368

Conversation

stefanluptak
Copy link
Contributor

@stefanluptak stefanluptak commented Apr 30, 2020

I tried benchmarking both solutions and the time it takes is around 60% of the old implementation.
Even bigger difference is in the amount of memory being allocated and not released during the rebuild. With old implementation it gradualy rises into gigabytes. With new implementation it stays around 100-200 megabytes. I am not sure if the Stream is not used correctly, or if there's some bug in the Stream internals or if it's just my special case, but it could be useful if somebody could do some benchmarks with his data to prove my suspicion.

Before change:
Screenshot 2020-04-30 at 10 52 08
Screenshot 2020-04-30 at 10 52 05

After change:
Screenshot 2020-04-30 at 12 03 30
Screenshot 2020-04-30 at 12 03 24

I tested this on around 100_000 events stored as JSONB in PostgreSQL.

Benchmarking machine:

Operating System: macOS
CPU Information: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
Number of Available Cores: 4
Available memory: 16 GB
Elixir 1.10.2
Erlang 22.2.8

I didn't write new test(s). I assume, this is covered by event_persistence_test.exs.

@slashdotdash
Copy link
Member

This looks great, thank you @EskiMag.

@slashdotdash slashdotdash merged commit 4f85963 into commanded:master Apr 30, 2020
@slashdotdash slashdotdash changed the title Fix high memory consumption during aggregate rebuild Reduce memory consumption during aggregate state rebuild Apr 30, 2020
slashdotdash added a commit that referenced this pull request Apr 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants