You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This a feature request for a tool that allows to compare the results of multiple experiments with the baseline profile. That would allow to validate the results of the experiments:
Check that all mojos that are expected to be executed are actually executed (i.e., no experiment accidentally excludes a mojo, distorting the total time; or that deliberately excluded mojo is definitely not executed)
See anomalies and detect invalid experiments. For example, if the first execution starts with empty target/s, but subsequent — with already compiled classes and do not include clean — the compile time in those experiments may be dramatically smaller, which distorts the results. Such issue applies to any executions that use caches.
Manual comparison is tedious and error-prone in case of multiple experiments, especially if the mojos are sorted by execution time.
Reports
Such tool may produce such reports:
Detailed per-mojo comparison for each module
As a web-page:
Module Foo:
Mojo
Profile 1 (baseline)
Profile 2
…
maven-clean-plugin:3.0.1:clean
100 ms
120 ms ( 20%*)
97 ms (-3%)
maven-compiler-plugin:3.8.1:compile
7.5 s
6.8 s (-10%)
6.3 s (-15%)
maven-surefire-plugin:2.22.1:test
15.5 s
8.3 s (-40%)
0 (:warning: no execution — check your experiment)
…
Total
35 s
27 s
25 s
A summary of all experiments
As console output/web-page:
Profile
Total time
Difference
Profile 1 (baseline)
35 s
—
Profile 2
27 s
-10%
…
Profile N
25 s
-12%
* — percents here and elsewhere are mostly random 🙃
The text was updated successfully, but these errors were encountered:
Overview
This a feature request for a tool that allows to compare the results of multiple experiments with the baseline profile. That would allow to validate the results of the experiments:
target/
s, but subsequent — with already compiled classes and do not includeclean
— the compile time in those experiments may be dramatically smaller, which distorts the results. Such issue applies to any executions that use caches.Manual comparison is tedious and error-prone in case of multiple experiments, especially if the mojos are sorted by execution time.
Reports
Such tool may produce such reports:
Detailed per-mojo comparison for each module
As a web-page:
Module Foo:
A summary of all experiments
As console output/web-page:
* — percents here and elsewhere are mostly random 🙃
The text was updated successfully, but these errors were encountered: