This implementation of TORS consists of a backend written in C (cTORS), and a front-end written in python (TORS).
The basic project setup uses the structure provided by cmake. The subfolders are subprojects:
- cTORS: The c implementation of TORS
- cTORSTest: The tests for cTORS
- pyTORS: The python interface for cTORS
- TORS: The challenge environment, in python
The following section explains how to compile this source code. Alternatively one can run the code in a docker container. The Dockerfile also shows how this project can be compiled and run step-by-step.
To compile cTORS, cmake 3.11 (or higher) is required and the python development libraries:
apt-get install cmake
apt-get install python3-dev
For windows:
- download and install cmake: https://cmake.org/download/
- python header files already installed with python, see the include folder in your python folder.
This project uses Google Protocol Buffers to read the data files. Installation is required to compile the C source:
- Download the source from the github page (version 3.15.6 is used)
- Follow the instructions to install
You can build cTORS and the pyTORS library with the following command.
mkdir build
python setup.py build
python setup.py install
In the source directory execute the following commands:
mkdir build
cd build
cmake ..
cmake --build .
This has been tested with gcc 9.3. Older versions may not support the c 17 standard.
You can also run TORS in a docker container. To build and run the container, run:
docker build -t tors-base .
docker run --network="host" --rm -it tors-base /bin/bash
To run challenge environment, run the following code
cd TORS
python run.py
Optionally you can change the episode or agent data by changing the parameters
python run.py --agent agent.json --episode episode.json
The --agent
option sets the file that configures the agent.
The --episode
option sets the file that configures the episode.
You can also run the file with the --train
flag to train the agent instead of eveluating its performance.
To use cTORS in python, you need to import they pyTORS
library. E.g.
from pyTORS import Engine
engine = Engine("data/Demo")
scenario = engine.get_scenario("data/Demo/scenario.json")
state = engine.start_session(scenario)
actions = engine.step(state)
engine.apply_action(actions[0])
engine.end_session(state)
The visualizer runs as a flask server. Install the dependencies in TORS/requirements-visualizer
first.
pip install -r TORS/requirements-visualizer
Now flask can be run by running the commands:
cd TORS/visualizer
export FLASK_APP=main.py
export FLASK_ENV=development
export FLASK_RUN_PORT=5000
python -m flask run
The repository also includes example code that wraps cTORS in a gym-environment and uses an RL implementation from stable-baselines3 to learn a policy. To run this example, first install the requirements:
pip install -r TORS/requirements-gym
Then run:
cd TORS
python run_gym.py
You can check the learning progress using tensorboard:
tensorboard --logdir ./log_tensorboard/
TORS can be configured through configuration files. Seperate configuration exists for
- The location
- The scenario
- The simulator
- The episode
- The agent
A location is described by the location.json
file in the data folder.
It describes the shunting yard: how all tracks are connected, what kind of tracks they are, and distances among tracks.
In order to use the visualizer for that location, you need to provided another file vis_config.json
. See the folder data/Demo
and data/KleineBinckhorstVisualizer
for examples.
A scenario is described by the scenario.json
file in the data folder.
It describes the scenario: which employees are available, shunting units' arrivals and departures, and possible disturbances.
The simulator can be configured by the config.json
file in the data folder.
It describes which business rules need to be checked and the parameters for the actions
You can provide an episode configuration and pass it to TORS/run.py
with the --episode
parameter.
This file describes the evaluation/training episode.
It contains the path to the data folder, the number of runs, RL parameters and parameters for scenario generation.
You can provide an agent configuration and pass it to TORS/run.py
with the --agent
parameter.
This file prescribes which agent to use, and passes parameters to the agent.
To run the cTORS tests, execute the commands
cd build
ctest
The documentation in the C code is written in the Doxygen format. Install doxygen (optional) to generate the documentation, or check the full documentation online at algtudelft.github.io/cTORS.
To generate the documentation, install the following programs:
apt-get install -y doxygen graphviz fonts-freefont-ttf
apt-get install -y libclang-dev
python -m pip install git git://github.com/pybind/pybind11_mkdoc.git@master
python -m pip install pybind11_stubgen
With the dependencies installed, cmake automatically generates the documentation. It can also be generated manually by running
cd cTORS
doxygen Doxyfile
cd ..
python -m pybind11_mkdoc -o pyTORS/docstrings.h cTORS/include/*.h -I build/cTORS
This produces as output the cTORS/doc
folder and the pyTORS/docstrings.h
source file. This last file is used in pyTORS/module.cpp
to generate the python docs.
- Mathijs M. de Weerdt: Conceptualization, Supervision, Project administration, Funding acquisition, Writing - review & editing
- Bob Huisman: Conceptualization
- Koos van der Linden: Software, Writing - Original draft
- Jesse Mulderij: Writing - Original draft
- Marjan van den Akker: Supervision of the bachelor team
- Han Hoogeveen: Supervision of the bachelor team
- Joris den Ouden: Conceptualization, Supervision of the bachelor team
- Demian de Ruijter: Conceptualization, Supervision of the bachelor team
- Bachelor-team, consisting of Dennis Arets, Sjoerd Crooijmans, Richard Dirven, Luuk Glorie, Jonathan den Herder, Jens Heuseveldt, Thijs van der Horst, Hanno Ottens, Loriana Pascual, Marco van de Weerthof, Kasper Zwijsen: Software, Visualization