Rerun

Rerun 0.17: defaults and overrides, streaming in notebooks, and better website embeddings The release brings a huge increase in explicit control over your visualizations. You can now use blueprints to both set default component values on a view, and to override components on specific entities in the view. You can do this both from code and in the UI. In the video, we use a default to bulk edit the size of all camera frustums and an override to edit just the front-facing frustum size. With the introduction of blueprint overrides and defaults, Rerun 0.17 gives you direct control over what visualizers are applied to what entities and makes all of this easy to inspect in the UI. Together, these features increase the amount of flexibility and control you have over exactly how your data is visualized in Rerun. Beyond defaults and overrides, this release also comes with: 🔸 Improved notebook and website embedding support 👉 You can now stream data from the notebook cell to the embedded viewer. 👉 There is improved support for having multiple viewers on the same web page. 👉 And you have more more configuration options to control the visibility of the Menu bar, time controls, etc. 🔸 Additional configurability from code, for example: 👉 ImagePlaneDistance(size of the Pinhole frustum visualization) 👉 AxisLength(axis length of the transform visualization) 👉 and all settings on TensorViews 🔸 New examples: 👉 PaddleOCR 👉 Vista, a generative driving world model 👉 Stereo Vision SLAM And much more 🚀🚀🚀 Check out the blog post on overrides and defaults, and the full change log in the links in the comments 👇

View Tensorboard log files in Rerun! The TFRecord data-loader plugin lets you view a TFRecord of Events (i.e., Tensorboard log files) in the Rerun Viewer. It uses the external data loader mechanism to add this capability to the viewer without modifying the viewer itself. Check out the example in the comments 👇

Recently there's been a ton of progress for depth estimation from a single image using a neural network, the problem oftentimes is having an easy way to compare all of the different networks. So Pablo Vela built a Hugging Face space using Rerun and Gradio! Why do this? By just looking at the 2D depth maps it's hard to see exactly what the values are, is the value returned disparity (as often is with scale-invariant models) or is it metric? How good is the depth outside of how sharp it looks in the 2D representation? The depth compare space makes it much easier, it provides a way to look at both depth maps simultaneously, along with the corresponding point cloud. This gives a much better representation of how well the monocular network performed. Lastly, both disparity and depth values are visualized. Rerun makes this all extremely easy to log and visualize, allowing you to see two different monocular depth networks performance simultaneously. Along with this it provides a separation between metric depth networks and scale invariant networks! Give it a whirl and check out the links in the comments:

Check out Robby Fischer, an autonomous robot arm that you can play chess against, created by Alexander Berntsson and Herman Lauenstein. To determine what move the human makes it only checks if a piece stands on a square and what color it is, you can see the result in bottom left corner in the video. To help check the squares for pieces it creates a mask for each square that determines what part of the image may only contain parts of a piece that stands on it. This is needed due to some pieces, like the king being, so tall that their heads appear above other squares. This mask is also shown in the bottom left corner. In the space view to the right we can see the pieces and their bounding boxes, the bounding boxes are used to figure out the height of the pieces so it knows how far up it has to move the pieces.  In the view we can also see it's planned trajectory before it starts moving. To decide what moves to make it uses the chess engine Stockfish. The arm consists of 3 steppers motors, 1 camera, 1 servo, 2 hall effect sensors, 1 limit switch and a raspeberry pico. There is also an additional camera that's shown in the middle but that's just used to show the robot, it's not used by the program. The pico is connected to a laptop that does the computer vision and planning. To generate the robot model in the video an URDF was created using the Fusion360 plugin fusion2urdf.

Fantastic to see Scaled Foundations integrating Rerun into their product! 👏 🦾

nuScenes in Rerun! 🚗🚙🛻 Fridays are for classics. Check out our example showing how to visualize the #nuScenes dataset by Motional in Rerun. The scenes in this dataset encompass data collected from a comprehensive suite of sensors on autonomous vehicles. These include 6 cameras, 1 LIDAR, 5 RADAR, GPS and IMU sensors. Consequently, the dataset provides information about the vehicle's pose, the images captured, the recorded sensor data and the results of object detection at any given moment. Link to the code and the example in the browser in the comments 👇

Streaming visualizations in a notebook ✨ With Rerun's improved notebook integration comes the ability to add to an existing recording within a notebook. This makes it possible to e.g. interactively try different learning rates, losses, and batch sizes, while continuously visualizing the training progress interactively. In this example notebook, we demonstrate this when fitting a simple neural field to a 2D image. The neural field is a simple multilayer perceptron with optional positional input encoding. The image is sampled uniformly, and the network is trained to predict the color given the pixel position. To visualize the progress of the training, we log the loss and regularly densely query the network to retrieve the image encoded in the network weights. Check out the notebook example, a Colab, and our how-to guide in the comments below 👇

Train a model on the #pusht dataset and visualize it in Rerun! 🦾 #LeRobot is a project by Hugging Face 🤗 that aims to provide models, datasets, and tools for real-world robotics in PyTorch. This example shows how to train a model on the pusht dataset and visualize its progress in rerun. Find the full example and a link to the data set in the comments 👇! Remi Cadene

📊 Image Histograms with Kornia in #rust -- and Rerun viz :) 💡An image histogram is a graphical representation of the number of pixels in an image as a function of their intensity, crucial for various image processing tasks because they provide insights into the image's contrast, brightness, and dynamic range. 🔥Checkout the example here: https://lnkd.in/dBBtE-bx What color is gonna win ? 🐉 ⚔️ #kornia #computerVision #rust #opensource #artificialintelligence #houseofdragons

Web embeddings are key to unifying your visualization stack 💡 Our goal at Rerun is to build a visualization tool kit that allows companies to unify their visualization stack. From early prototyping, to running things in production, to training jobs: you should be able to use the same tools. A key unlock for making that happen is the ability to embed visualizations everywhere you might want them. Prototyping in a notebook? Building a custom dashboard? Expose visualizations in your own product? All of that you can do with Rerun. The video is showing embedded Rerun in our docs. With the latest Rerun 0.17 we have made embedded Rerun, and especially Rerun in a notebook much better. Check out an embedded Rerun viewer in our docs, the How-to, and the full 0.17 changelog in the links in the comments 👇