Outlines via masking & postprocessing in `re_renderer` #1532

Wumpf · 2023-03-08T15:26:12Z

(*) Only supports meshes so far!

Screen.Recording.2023-03-08.at.12.14.55.mov

This test scene is a bit over the top but demonstrates key abilities:

arbitrary screen space outline width
two outline channels, in each objects may choose to share an outline or get their own
outlines extend inwards and outwards - this was not an original design goal, but more of a simplification
- It's worth noting that Blender does the same in their selection outlines, so I'm too concerned about it

How it works is detailed in renderer/outlines.rs, after reading the comment there I'd recommend first reviewing the changes in view_builder.rs, then the new draw methods in renderer/outlines.rs. If the process is not clear at that point better post questions, because from there on it only gets more complicated :)

Checklist

I have read and agree to Contributor Guide and the Code of Conduct
I've included a screenshot or gif (if applicable)

Part of gfx-rs/naga#889

… phase

…p left

as opposed to texel coordinates

crates/re_renderer/src/renderer/outlines.rs

teh-cmc · 2023-03-09T08:33:38Z

I've got two issues to keep you busy while I get through this 😄

Running the example in debug emits validation errors on my machine:
Cannot build the example in release it seems?

teh-cmc · 2023-03-09T09:15:32Z

Breaking the compositing shader crashes the entire program for some reason, which makes exploration-by-experimentation much harder 😞

23-03-09_10.03.51.patched.mp4

2023-03-09T09:03:55.687575Z ERROR re_renderer::error_tracker: WGPU error tick_nr=45182 description=Validation Error

Caused by:
    In a RenderPass
      note: encoder = `composite_encoder`
    In a set_pipeline command
      note: render pipeline = `<Invalid-RenderPipeline label=OutlineCompositor>`
    render pipeline (7, 1, Vulkan) is invalid

2023-03-09T09:03:55.688493Z ERROR wgpu_core::present: No work has been submitted for this frame
2023-03-09T09:03:55.689586Z ERROR wgpu_core::present: No work has been submitted for this frame
2023-03-09T09:03:55.690887Z ERROR wgpu_core::present: No work has been submitted for this frame
thread 'main' panicked at 'There was still a command encoder from the previous frame at the beginning of the current. Did you forget to call RenderContext::before_submit?', crates/re_renderer/src/context.rs:231:13
stack backtrace:
   0: rust_begin_unwind
             at /rustc/fc863f15669680fa70d255faec3ca3fb507c3405/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/fc863f15669680fa70d255faec3ca3fb507c3405/library/core/src/panicking.rs:64:14
   2: begin_frame
             at ./crates/re_renderer/src/context.rs:231:13
   3: {closure#0}<outlines::Outlines>
             at ./crates/re_renderer/examples/framework.rs:221:21
   4: sticky_exit_callback<(), outlines::framework::{impl#1}::run::{closure_env#0}<outlines::Outlines>>
             at /home/cmc/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.28.1/src/platform_impl/linux/mod.rs:884:9
   5: run_return<(), outlines::framework::{impl#1}::run::{closure_env#0}<outlines::Outlines>>
             at /home/cmc/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.28.1/src/platform_impl/linux/wayland/event_loop/mod.rs:548:21
   6: run<(), outlines::framework::{impl#1}::run::{closure_env#0}<outlines::Outlines>>
             at /home/cmc/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.28.1/src/platform_impl/linux/wayland/event_loop/mod.rs:222:25
   7: winit::platform_impl::platform::EventLoop<T>::run
             at /home/cmc/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.28.1/src/platform_impl/linux/mod.rs:792:56
   8: winit::event_loop::EventLoop<T>::run
             at /home/cmc/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.28.1/src/event_loop.rs:305:9
   9: outlines::framework::Application<E>::run
             at ./crates/re_renderer/examples/framework.rs:185:9
  10: {async_fn#0}<outlines::Outlines>
             at ./crates/re_renderer/examples/framework.rs:351:5
  11: block_on<outlines::framework::run::{async_fn_env#0}<outlines::Outlines>>
             at /home/cmc/.cargo/registry/src/github.com-1ecc6299db9ec823/pollster-0.3.0/src/lib.rs:128:15
  12: start<outlines::Outlines>
             at ./crates/re_renderer/examples/framework.rs:371:9
  13: call_once<fn(), ()>
             at /rustc/fc863f15669680fa70d255faec3ca3fb507c3405/library/core/src/ops/function.rs:507:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Wumpf · 2023-03-09T09:41:56Z

@teh-cmc ah odd just ran into the compile issue on a branch I based upon here and got surprised. cherry-picking the fix over.
The shader issue is clearly a bug in Naga it apparently is about @location(3) @interpolate(flat)on instanced_mesh.wgsl. Can you try removing the flat there?
~~The reason this is a hard crash for us is that this happens in the Vulkan validation layer and not in any shader compiler~~

Wumpf · 2023-03-09T09:45:01Z

The crash upon edit sounds like its on me - I added that check/error there not too long ago. Kinda surprised I didn't run into that so far. Would take that separately if you don't mind

Wumpf · 2023-03-09T11:09:03Z

We confirmed that validation issue is about outputting a uvec from the fragment shader. Opened an issue here gfx-rs/naga#2270

teh-cmc · 2023-03-09T11:11:26Z

We could confirm with @Wumpf that the validation error is coming from the use of a Rg8Uint texture for the mask, which in turns means returning a uvec in the fragment shader, which generates some Spir-V that vulkan doesn't like too much.
Hackishly modifying the code to use a Rg8Unorm instead does "fix" the issue (see patch below).

diff --git a/crates/re_renderer/shader/instanced_mesh.wgsl b/crates/re_renderer/shader/instanced_mesh.wgsl
index cc7a43be1..5254f730a 100644
@@ -62,6  61,6 @@ fn fs_main_shaded(in: VertexOut) -> @location(0) Vec4 {
 }
 
 @fragment
-fn fs_main_outline_mask(in: VertexOut) -> @location(0) UVec2 {
-    return in.outline_mask;
 fn fs_main_outline_mask(in: VertexOut) -> @location(0) Vec2 {
     return Vec2(in.outline_mask);
 }
diff --git a/crates/re_renderer/shader/outlines/jumpflooding_init.wgsl b/crates/re_renderer/shader/outlines/jumpflooding_init.wgsl
index 8b1b14d7c..85e1c3349 100644
--- a/crates/re_renderer/shader/outlines/jumpflooding_init.wgsl
    b/crates/re_renderer/shader/outlines/jumpflooding_init.wgsl
@@ -2,10  2,10 @@
 #import <../screen_triangle_vertex.wgsl>
 
 @group(0) @binding(0)
-var mask_texture: texture_multisampled_2d<u32>;
 var mask_texture: texture_multisampled_2d<f32>;
 
 fn has_edge(closest_center_sample: UVec2, sample_coord: IVec2, sample_idx: i32) -> Vec2 {
-    let mask_neighbor = textureLoad(mask_texture, sample_coord, sample_idx).xy;
     let mask_neighbor = UVec2(textureLoad(mask_texture, sample_coord, sample_idx).xy);
     return Vec2(closest_center_sample != mask_neighbor);
 }
 
@@ -54,17  54,17 @@ fn main(in: VertexOutput) -> @location(0) Vec4 {
     // TODO(andreas): Should we assert somehow on textureNumSamples here?
 
     let center_coord = IVec2(Vec2(resolution) * in.texcoord);
-    let mask_top_left = textureLoad(mask_texture, center_coord, 0).xy;
-    let mask_right_top = textureLoad(mask_texture, center_coord, 1).xy;
-    let mask_left_bottom = textureLoad(mask_texture, center_coord, 2).xy;
-    let mask_bottom_right = textureLoad(mask_texture, center_coord, 3).xy;
     let mask_top_left = UVec2(textureLoad(mask_texture, center_coord, 0).xy);
     let mask_right_top = UVec2(textureLoad(mask_texture, center_coord, 1).xy);
     let mask_left_bottom = UVec2(textureLoad(mask_texture, center_coord, 2).xy);
     let mask_bottom_right = UVec2(textureLoad(mask_texture, center_coord, 3).xy);
 
     var edge_pos_a_and_b = Vec4(0.0);
     var num_edges_and_b = Vec2(0.0);

diff --git a/crates/re_renderer/src/renderer/outlines.rs b/crates/re_renderer/src/renderer/outlines.rs
index 4f8eaa033..24a574cae 100644
--- a/crates/re_renderer/src/renderer/outlines.rs
    b/crates/re_renderer/src/renderer/outlines.rs
@@ -124,7  124,7 @@ impl OutlineMaskProcessor {
     /// Format of the outline mask target.
     ///
     /// Two channels with each 256 object ids.
-    pub const MASK_FORMAT: wgpu::TextureFormat = wgpu::TextureFormat::Rg8Uint;
     pub const MASK_FORMAT: wgpu::TextureFormat = wgpu::TextureFormat::Rg8Unorm;
 
     pub const MASK_MSAA_STATE: wgpu::MultisampleState = wgpu::MultisampleState {
         count: ViewBuilder::MAIN_TARGET_SAMPLE_COUNT,
@@ -398,7  398,7 @@ impl OutlineMaskProcessor {
                         binding: 0,
                         visibility: wgpu::ShaderStages::FRAGMENT,
                         ty: wgpu::BindingType::Texture {
-                            sample_type: wgpu::TextureSampleType::Uint,
                             sample_type: wgpu::TextureSampleType::Float { filterable: false },
                             view_dimension: wgpu::TextureViewDimension::D2,
                             multisampled: true,
                         },

The error seems to come down to naga mistakenly inserting a flat interpolation, see the diff below between the two Spir-V dumps:

1c1
< SPIR-V 1.0 module, <id> bound of 121
---
> SPIR-V 1.0 module, <id> bound of 122
74c74
< Output uint2* _115 : [[Location(0), Flat]];
---
> Output float2* _115 : [[Location(0)]];
135c135,136
<   *_115 = _120;
---
>   float2 _121 = ConvertUToF(_120);
>   *_115 = _121;

Wumpf · 2023-03-09T12:27:21Z

⚠️ I broke WebGL. Hitting gfx-rs/wgpu#4481

panicked at 'wgpu error: Validation Error

Caused by:
    In Device::create_render_pipeline
      note: label = `compositor`
    Internal error in FRAGMENT shader: ERROR: 0:25: 'invariant' : Cannot be qualified as invariant.

Fix incoming

See https://github.com/gfx-rs/naga/issues/2252

Wumpf · 2023-03-09T12:36:24Z

Fixed above, but there's more >.<

panicked at 'Tex storage 2D multisample is not supported', /Users/andreas/.cargo/registry/src/github.com-1ecc6299db9ec823/glow-0.12.0/src/web_sys.rs:3131:9``

I checked before I started this that sampling an MSAA target works fine, but there must be some fineprint on what exactly is possible. Digging deeper now

I checked before I started this that sampling a msaa texture should be supported. Need to dig a bit deeper

Wumpf · 2023-03-09T12:38:54Z

Turns out I was involved in this very issue (or a similar one) a couple of years ago. gfx-rs/wgpu#2149
Have no memory about this 😅

Wumpf · 2023-03-09T14:19:23Z

Seems actually that sampling MSAA texture is not a thing after all in WebGL? At least this ticket implies support won't come. gfx-rs/wgpu#2117
First time made use of our HardwareTier and provided a non-MSAA version. Tested it both on native and on web (native won't choose this unless forced to it)

… in there

teh-cmc

This took me way longer than I hoped to learn all the techniques involved and fully grasp everything that goes on in there... Really glad I did though, this is absolutely amazing on so many levels 🤯 .

Nothing blocking! 🚢

👏

crates/re_renderer/examples/outlines.rs

crates/re_renderer/src/renderer/mesh_renderer.rs

crates/re_renderer/shader/outlines/jumpflooding_init.wgsl

crates/re_renderer/shader/outlines/jumpflooding_init_msaa.wgsl

Wumpf added 23 commits March 8, 2023 16:14

outline example setup

c4c78b9

new outline mask phase, outline renderer skeleton

675bc5a

outline processor setup

03c69bd

basic mesh mask drawing

f06fd4f

outline mask has now its own depth buffer

29583e7

add outline mask to meshinstance, unused so far

a703a59

scaffolding for jumpflooding in outlines

d9af78c

potentially working jumpflooding init shader

40f2ba5

basically working jumpflooding, hardcoded red outlines for the moment

0f25979

wip

47428f1

rebase fixup

1abb07f

renames

3b0af4e

A lot better anti-aliasing handlign and documentation thereof

cd80950

don't render meshes that don't participate in outlines during outline…

1ce2ab7

… phase

configurable mesh outlines. demonstrate internal outlines in example

8eec43b

outline configurability, blend both outline a and b

70f2e95

Fix crash in write belt

b75e8a6

use texture sampler for jumpflooding to fix buggy outline point at to…

0bb0bd5

…p left

simplify shader a bit by using pixel coordinates all the way

e1c2183

as opposed to texel coordinates

use inf() to mark-up invalid positions

1dfd840

outline documentation

c0bf404

fix thickness/radius confusion

dad5cd2

cleanup code in outlines.rs

f619281

Wumpf added the 🔺 re_renderer affects re_renderer itself label Mar 8, 2023

fix typos

de51728

Wumpf force-pushed the andreas/re_renderer/outlines branch from 230fed6 to de51728 Compare March 8, 2023 15:32

teh-cmc self-requested a review March 8, 2023 18:32

Wumpf commented Mar 8, 2023

View reviewed changes

crates/re_renderer/src/renderer/outlines.rs Outdated Show resolved Hide resolved

Fix documentation in outlines.rs

654e362

fix release mode compilation issue

649ca07

Wumpf mentioned this pull request Mar 9, 2023

Outputting integral type from fragment shader causes Vulkan validation error gfx-rs/naga#2270

Closed

Workaround for WebGL shader issue

aac2ab5

See https://github.com/gfx-rs/naga/issues/2252

Provide non-msaa version of outline renderer for WebGL compatibility

7ec31a2

hackish pause/unpause to simplify visualizing everything that goes on…

c643bcf

… in there

teh-cmc force-pushed the andreas/re_renderer/outlines branch from 2eb5cc6 to c643bcf Compare March 9, 2023 16:34

teh-cmc approved these changes Mar 9, 2023

View reviewed changes

Wumpf added 2 commits March 9, 2023 18:15

pr feedback, fix demo timing

8acd9cb

bump ci?

1e75572

Wumpf merged commit 15d21af into main Mar 9, 2023

Wumpf deleted the andreas/re_renderer/outlines branch March 9, 2023 17:29

emilk mentioned this pull request Mar 27, 2023

Add a script that generates a changelog from recent PRs and their labels #1718

Merged

1 task

emilk changed the title ~~[re_renderer] Outlines via masking & postprocessing (*)~~ Outlines via masking & postprocessing in re_renderer Mar 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Outlines via masking & postprocessing in `re_renderer` #1532

Outlines via masking & postprocessing in `re_renderer` #1532

Wumpf commented Mar 8, 2023 •

edited

Loading

teh-cmc commented Mar 9, 2023

teh-cmc commented Mar 9, 2023 •

edited

Loading

Wumpf commented Mar 9, 2023 •

edited

Loading

Wumpf commented Mar 9, 2023

Wumpf commented Mar 9, 2023

teh-cmc commented Mar 9, 2023 •

edited

Loading

Wumpf commented Mar 9, 2023 •

edited

Loading

Wumpf commented Mar 9, 2023

Wumpf commented Mar 9, 2023

Wumpf commented Mar 9, 2023

teh-cmc left a comment •

edited

Loading

Outlines via masking & postprocessing in re_renderer #1532

Outlines via masking & postprocessing in re_renderer #1532

Conversation

Wumpf commented Mar 8, 2023 • edited Loading

Checklist

teh-cmc commented Mar 9, 2023

teh-cmc commented Mar 9, 2023 • edited Loading

Wumpf commented Mar 9, 2023 • edited Loading

Wumpf commented Mar 9, 2023

Wumpf commented Mar 9, 2023

teh-cmc commented Mar 9, 2023 • edited Loading

Wumpf commented Mar 9, 2023 • edited Loading

Wumpf commented Mar 9, 2023

Wumpf commented Mar 9, 2023

Wumpf commented Mar 9, 2023

teh-cmc left a comment • edited Loading

Choose a reason for hiding this comment

Outlines via masking & postprocessing in `re_renderer` #1532

Outlines via masking & postprocessing in `re_renderer` #1532

Wumpf commented Mar 8, 2023 •

edited

Loading

teh-cmc commented Mar 9, 2023 •

edited

Loading

Wumpf commented Mar 9, 2023 •

edited

Loading

teh-cmc commented Mar 9, 2023 •

edited

Loading

Wumpf commented Mar 9, 2023 •

edited

Loading

teh-cmc left a comment •

edited

Loading