[Bug Report] Adding call for CV2 imshow will freeze entire Isaac-Sim #808

ahanjaya · 2024-08-09T03:24:52Z

The bug is about calling the OpenCV window to render the camera image from the robot. The entire simulation will stuck on startup.

def _debug_vis_callback(self, event):
     cv2.imshow("Robot Depth Frame", self.grid_depth_img)
     cv2.waitKey(1)

Please try to provide a minimal example to reproduce the bug. Error messages and stack traces are also helpful.

Describe the characteristics of your environment:

Previously this bug never occurred, I could easily stream the OpenCV result with the same code

I have checked that there is no similar issue in the repo (required)
I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

The text was updated successfully, but these errors were encountered:

jsmith-bdai · 2024-08-09T10:03:16Z

I'm not super familiar with cv2, but I think you can remove the cv2.waitKey(1) call to avoid blocking the simulation environment.

Let us know if this works for your issue!

ahanjaya · 2024-08-12T03:24:40Z

it still doesn't work, even by simply initializing the OpenCV window

def _set_debug_vis_impl(self, debug_vis: bool):
    if self.cfg.show_depth:
        cv2.namedWindow("Robot Depth Frame", cv2.WINDOW_NORMAL)

I did this before and had no problem the last 2 weeks, but after I rebased to the latest main branch this issue occured.

Provide feedback