This project aims to provide an efficient data processing framework that prepares large datasets for a deep learning algorithm focused on lateral movement detection in cybersecurity. (for details of algorithm implementation, see Bai et al. 2020.

• Batch processing of large files (over 35GB compressed)
• Rust-based performance for time-efficient data processing
• Application in real-world big data scenarios and cybersecurity

• Rust Programming Language
• Various Rust Libraries (std::env, std::fs, bzip2, serde_json, rayon)

1. Clone the repository
2. Navigate to the project directory
3. Build the project with cargo build

To run the project, execute the following command:

cargo run <path_to_bz2_files>

If you're interested in contributing to this data processing tool aimed at detecting lateral movement in cybersecurity, particularly involving RDP logs, you're welcome to reach out. I am are interested in expanding this tool to accommodate Python users and other real-world scenarios.

For more information, questions, or collaborations, please contact me.