Deficient domestic wastewater management, industrial waste, and floating debris are some leading factors that contribute to inland water pollution. The surplus of minerals and nutrients in overly contaminated zones can lead to the invasion of floating weeds. Lemnoideae, commonly known as duckweed, is a family of floating plants that has no leaves or stems and forms dense colonies with a fast growth rate. If not controlled, duckweed establishes a green layer on the surface and depletes fish and other organisms of oxygen and sunlight.

Consequently, we propose an open-source unmanned surface vehicle (USV) for automatic duckweed removal that costs less than $500. The USV uses 3D-printed parts and common components that can be easily purchased. It is worth mentioning that the proposed approach won the First Global Prize in the OpenCV AI Competition 2022, organized by the OpenCV Foundation. Moreover, further prototyping details and testing results are available in the paper:

Edwin Salcedo, Yamil Uchani, Misael Mamani, and Mariel Fernandez, Towards Continuous Floating Invasive Plant Removal Using Unmanned Surface Vehicles and Computer Vision, IEEE Access 2024.

[Paper] [Video Abstract]

This section contains the CAD designs for the Unmanned Surface Vehicle (USV) proposed for the collection of duckweed. It includes all the parts that were used to 3D print the vehicle, such as the "Control Box", "Main Body", "Floaters", and "Main Grid".

• Aux_1: Support structures.
• Aux_2: Support structures.
• Aux_3: Support structures.
• BackSupport: Used for the net grip that collects water lentils.
• BatteryCover: Covers the battery housing area.
• BoltHolder: Holder for bolts.
• BoxLid: Lid for the control box.
• ElectronicBox: Housing for the electronic components.
• Flipper: Grip for the motors.
• FloaterBase: Base structure for the floaters.
• FloaterLeftBack: The left back part of the floaters.
• FloaterLeftFront: The left front part of the floaters.
• FloaterRightBack: The right back part of the floaters.
• FloaterRightFront: The right front part of the floaters.
• FrontSupport: Frontal support structure.
• Grid: Filtering grid.
• Joint: Joint component.
• UpperStructure: Upper structure of the vehicle.
• VentLid: Lid for ventilation.

• Assembly: The complete assembly of the USV.

The design was created in SolidWorks 2020, and it is recommended to use the same or a newer version for compatibility.

1. Download the desired part or assembly files from the repository.
2. Open the files using SolidWorks 2020 or a newer version.
3. To view or modify individual parts, open the part (.SLDPRT) files.
4. To view the complete assembly, open the Assembly.SLDASM file.

Architectural layout encapsulating the three most essential components:

• A duckweed detection model embedded inside an OAK-D camera.
• A steering angle classification model and a temporal weight module, namely momentum. Both were embedded inside a jetson nano development card.

The database used for this model was built by scrapping images from Google Images and the Global Biodiversity Information Facility. The resulting 4,000 images were grouped into five categories according to the view-point of the duckweed colonies inside the samples: close, near-close, near-wide, wide, and empty. The final dataset is available here (you'll need to create a IEEE Dataport account to access the dataset).

What follows is the list of experiments and trained models to replicate the results obtained using Yolov5 and the augmented versions of the dataset:

Our data preprocessing approach, as well as our Yolov5 and Yolov8 implementations can be found here.

We tested two approaches for SAC: first, by collecting a syntethic dataset using a bespoke virtual environment and training DL classifiers. This approach, including object detection and moment, was named as M3. Then, we tried to classify steering angle using stereo vision along with object detection and momentum, which we later named M4.

The final dataset acquired for M3 can be shared upon request. The dataset weights 48.66 Gb in total, which prevented us to share it online. Yet, the results and pretrained models using general classifiers from TorchVision are available below:

The project RecyclingRush required a simulator developed in Unity with the primary goal of resembling an underwater environment where a virtual agent collects water duckweeds. The simulator incorporates advanced features to enhance the agent's perception of the environment, such as the integration of Python-based deep learning models and stereo vision.

• Unity: The simulator was created using version 2020.3.30f1, but is executable with newer versions.
• Python: We used python's version 3.11.3. It is suggestable to use conda or pyenv to install Python and its dependencies and avoid collisions.
• Windows: Note that the simulator was developed and tested with Windows 11.

1. Clone this repository to your local machine.

   git clone [email protected]:EdwinTSalcedo/RecyclingRush.git recyclingrush
   cd recyclingrush/vehicle_simulator

2. Install the python dependencies for the detection models:

   pip install -r servers/requirements.txt

3. Additionally, ensure that the following Python packages are installed:

    opencv-python-headless
    numpy
    matplotlib
    torch
    torchvision
    Pillow
    requests
    ultralytics

Note that the folders:

• RecyclingRush contains the pre-packaged simulator, which is runnable using the executable file UnityCrashHandler64.exe.

• Autonomous aquatic robot: contains the simulator's editable files.

• servers: contains python scripts that accept input data and returns the estimated angle as for M1, M2, M3 and M4 self-driving modalities.

4. Run the pre-packaged simulator (.exe) located in the "RecyclingRush" folder. This will create a folder named "!RecyclingRush" in your "Documents" directory.

5. Inside the newly created "!RecyclingRush" folder, copy the entire "servers" folder. This step is crucial for both the simulator and the program to utilize the Python models.

6. Load the folder RecyclingRush (inside recyclingrush/vehicle_simulator) with Unity.

Open Unity and load the Unity project by navigating to the "Robot_acuatico_autonomo" folder. If the project does not automatically initialize, follow these steps: - Click on "File" in the Unity editor. - Choose "Open Project." - Navigate to the "Robot_acuatico_autonomo" folder and open the project.

Three different modes are available in the main menu of the simulator, each of which serves a specific function in model training and evaluation.

Autonomous Driving: Used for data collection during autonomous driving model training. It captures image sequences, considering both the angle and speed of the vehicle.

Validation: Allows manual image capture, establishing a database for model validation. Useful for evaluating model performance in controlled and specific scenarios.

Evaluation: The activation of the four modes in this environment allows the evaluation of duckweed detection and steering models. It facilitates comprehensive evaluations and performance analysis under various conditions.

The self-driving environment is set in an aquatic setting with various elements simulating a lake, featuring aquatic plants and boats. The purpose of this environment is to capture a sequence of images, taking into account the boat's angle and speed at any given moment. The interface comprises two interactive elements and two informational elements.

• Menu: This button allows users to navigate back to the main menu.

• Record/Stop: Activates or deactivates the image capture process.

• Angle: Displays the current steering angle of the boat. This information is reflected in the title of the captured image.

• Velocity: Shows the speed of the boat at the specific moment. The velocity information is also included in the image title.

The validation environment is designed to simulate a port in the aquatic lake, resulting in a more organized layout of elements. The primary objective is to manually capture images chosen by the user. Additionally, users can adjust the camera to capture different types of images, both underwater and head-on. The interface features four buttons for user interaction.

• Menu: This button allows users to return to the main menu.

• Capture: This button is responsible for capturing and saving the selected image.

• Up: Adjusts the camera to face upwards.

• Down: Adjusts the camera to face downwards.

The evaluation environment is designed for model testing and stereo camera usage to optimize the collection of duckweed and avoid obstacles efficiently. It displays a well-organized section of the lake for data collection, particularly focusing on the duckweed gathered over specific time intervals. The environment allows adjusting robot modalities with different characteristics.

• Menu: Returns to the main menu.

• Restart: Resets the robot's journey to test different modalities or changes.

• Reduce: Slows down the environment's time to capture journey details more effectively.

• Increase: Speeds up the environment's time to accelerate the journey if necessary.

• Camera: Changes the user's perspective on the boat.

• Modality Selection: A dropdown menu presenting 5 options:

  • Manual: Default modality, allowing user manipulation of robot movement.

  • Modality 1: Utilizes only the duckweed detection model with a drastic angle change.

  • Modality 2: Utilizes only the duckweed detection model but with momentum for smoother angle changes.

  • Modality 3: Considers two models - duckweed detection and direction detection, resulting in smoothed angle changes.

  • Modality 4: Incorporates the duckweed detection model and uses a stereo camera to determine obstacles to avoid.

If you find our work useful in your project, please consider to cite the following paper. Give RecyclingRush a star ⭐ on GitHub and share it with your friends and colleagues. With your support, we can continue to innovate in the field of autonomous vehicles for water quality management.

@article{salcedo2024,
  title={Towards Continuous Floating Invasive Plant Removal Using Unmanned Surface Vehicles and Computer Vision},
  author={Salcedo, Edwin and Uchani, Yamil and Mamani, Misael and Fernandez, Mariel},
  journal={IEEE Access},
  year={2024},
  publisher={IEEE},
  doi={10.1109/ACCESS.2024.3351764},
  url={https://doi.org/10.1109/ACCESS.2024.3351764}
}