This package is still under development. If you have any trouble running this code, please open an issue on GitHub.

Package to run the experiments for the preprint paper Distributed Convolutional Dictionary Learning (DiCoDiLe): Pattern Discovery in Large Images and Signals.

All the tests should work with python >=3.6. This package depends on the python library numpy, matplotlib, scipy, mpi4py, joblib. The package can be installed with the following command run from the root of the package.

pip install  -e .

Or using the conda environment:

conda env create -f dicodile_env.yml

To build the doc use:

pip install  -e .[doc]
cd docs
make html

To run the tests:

pip install  -e .[test]
pytest .

All experiments are with mpi4py and will try to spawned workers depending on the parameters set in the experiments. If you need to use an hostfile to configure indicate to MPI where to spawn the new workers, you can set the environment variable MPI_HOSTFILE=/path/to/the/hostfile and it will be automatically detected in all the experiments. Note that for each experiments you should provide enough workers to allow the script to run.

All figures can be generated using scripts in benchmarks. Each script will generate and save the data to reproduce the figure. The figure can then be plotted by re-running the same script with the argument --plot. The figures are saved in pdf in the benchmarks_results folder. The computation are cached with joblib to be robust to failures.

$ mpirun -np 1 \
         --mca btl_tcp_if_include wlp2s0 \
         --hostfile hostfile \
         python -m mpi4py examples/plot_mandrill.py

$ mpirun -np 1 \
         --mca btl_tcp_if_exclude docker0 \
         --hostfile hostfile \
         python -m mpi4py examples/plot_mandrill.py

Alternatively, you can also restrict the used interface by setting environment variables OMPI_MCA_btl_tcp_if_include or OMPI_MCA_btl_tcp_if_exclude

$ export OMPI_MCA_btl_tcp_if_include="wlp2s0"

$ export OMPI_MCA_btl_tcp_if_exclude="docker0"``