Add Custom Shader support to CommonFilters.py #1643

raytopianprojects · 2024-04-15T18:31:07Z

Issue description

There have been many requests from community members to be able to add their own shaders to CommonFilters because it saves them a lot of time and effort of having to re-implement filters that are already supported.

Solution description

In short these changes allow users to add their own shaders to CustomFilters, have ported all built in shaders to use new system and additionally add Chromatic Aberration, Vignette, Tint, Distortion, Resolution, LUT, and Film Grain as built in filters because they are now common.

New API

delFilter(name) 
loadFilter(name, shader_string, uniforms=None, consts=None, shader_inputs=None, needed_textures=None, needed_coords=None, render_into=None, auxbits=None, is_filepath=False, order=None

Here is an example of how AmbientOcclusion is implemented using the new system.

load_filter("AmbientOcclusion",
                   shader_string="o_color *= tex2D(k_txssao2, l_texcoord_ssao2).r;\n", # This will code will be added to the shader used by finalquad.
                   needed_textures=["ssao0", "ssao1", "ssao2", "depth", "aux"], # Tells the system additional textures that need to be created
                   needed_coords=["ssao2"], # Some shaders need additional coords due to render passes
                   auxbits=[AuxBitplaneAttrib.ABOAuxNormal], # Enables use of AuxBits generated by the auto shader
                   # render into takes the format of a dictonary whose arguments will be passed into renderQuadInto
                   render_into={ 
                       "filter-ssao0": {
                           "colortex": "ssao0", # a string type means use a texture with this name
                           "shader_inputs": {
                               "depth": "depth",
                               "params1": (numsamples, -amount / numsamples, radius, 0),
                               "params2": (strength, falloff, 0, 0),
                               "normal": "aux",
                               "random": base.loader.loadTexture("maps/random.rgb"),
                           },
                           "shader": Shader.make(SSAO_BODY % numsamples, Shader.SL_Cg) # Can also be a string
                       },
                       "filter-ssao1": {
                           "colortex": "ssao1",
                           "shader_inputs": {
                               "src": "ssao0",
                           },
                           "div": 2,
                           "shader": Shader.make(BLUR_X, Shader.SL_Cg)
                       },
                       "filter-ssao2": {
                           "colortex": "ssao2",
                           "shader_inputs":
                               {
                                   "src": "ssao1",
                               },
                           "shader": Shader.make(BLUR_Y, Shader.SL_Cg)
                       },
                   }, order=2) # Order allows users to set what order a filter will be applied. If no order is set then the filter's order will be set equal to the total amount of filters that are being applied.

Due to changes to reconfigure there have been minor breaking changes (some filters don't take the same arguments due to not needing them), despite this though all filters have been tested and have the same visual output.

Checklist

I have done my best to ensure that…

…I have familiarized myself with the CONTRIBUTING.md file
…this change follows the coding style and design patterns of the codebase
…I own the intellectual property rights to this code
…the intent of this change is clearly explained
…existing uses of the Panda3D API are not broken
…the changed code is adequately covered by the test suite, where possible.

Allow users to add their own shaders to CustomFilters. Port all built in shaders to use new system. Additionally add Chromatic Aberration, Vignette, Tint, Distortion, Resolution, LUT, and Film Grain as built in filters.

WMOkiishi · 2024-04-15T19:09:33Z

direct/src/filter/CommonFilters.py


+vfs: VirtualFileSystem = VirtualFileSystem.get_global_ptr()


This type annotation is only useful if the type-checker doesn't have information about what VirtualFileSystem.get_global_ptr returns (i.e., through stubs). If that's the case, the type-checker almost certainly doesn't know anything else about a VirtualFileSystem, so it doesn't really do much in any case. I'd recommend removing it, unless I'm missing something.

Thanks for catching that.

rdb · 2024-05-14T08:31:16Z

This is pretty neat. I think we need a new filter system on the C end of Panda, but there's no way that'll make it in time for 1.11, and this is simple enough that I might consider accepting this in the interim.

I prefer splitting the PR into the part containing the new system and the part with the additional filters for individual review.

I'm on the fence about stuffing the entire definition into a single dictionary structure. It gets unwieldy compared to an object hierarchy, but on the other hand, it does make it easier to load filters from some json-like file structure, hmm.

How is the order of render targets maintained? Dictionary insertion order?

If the uniforms and shader inputs are supposed to be specified in the same load_filter call, should they not go in the same place, with a shader input dictionary containing name, type and (default) value? Or is there a reason it was done this way?

Use the word "sort" instead of "order" for consistency with other parts of the Panda API.

Having the order default to None for these filters is a breaking change. There's an order enforced currently, one which makes sense. It shouldn't suddenly make tone mapping go after sRGB encoding if someone happens to call them in a different order, with the default settings.

I will think a bit more about the other aspects.

raytopianprojects · 2024-05-14T20:55:20Z

This is pretty neat. I think we need a new filter system on the C end of Panda, but there's no way that'll make it in time for 1.11, and this is simple enough that I might consider accepting this in the interim.

Yeah I agree that this is more of a stop gap then the permanent system.

I prefer splitting the PR into the part containing the new system and the part with the additional filters for individual review.

I'll split the PR up and it'd be good anyways to get feedback on some new filters people would like.

I'm on the fence about stuffing the entire definition into a single dictionary structure. It gets unwieldy compared to an object hierarchy, but on the other hand, it does make it easier to load filters from some json-like file structure, hmm.

The dictionary structure was partially inspired by how webgpu's render pipelines work and in my opinion it's nice to use but also very powerful. I can also look into adding json loading/saving for the filters. I mean we can already save/load particles so why not filters. If I do add that though loadFilter should probably become setFilter instead.

How is the order of render targets maintained? Dictionary insertion order?

Currently yes, but I could add a way to set the sort.

If the uniforms and shader inputs are supposed to be specified in the same load_filter call, should they not go in the same place, with a shader input dictionary containing name, type and (default) value? Or is there a reason it was done this way?

Looking at it again, it's because you can load the shader string from a file but I'll experiment and see which is more ergonomic.

Use the word "sort" instead of "order" for consistency with other parts of the Panda API.

Will do!

Having the order default to None for these filters is a breaking change. There's an order enforced currently, one which makes sense. It shouldn't suddenly make tone mapping go after sRGB encoding if someone happens to call them in a different order, with the default settings.

I was on the fence if I should assign default orders to the or not but you're correct that it could cause a lot of issues if I break that. I'll fix that breaking change.

I will think a bit more about the other aspects.

Yeah if you have any other feedback or ideas please let me know (we can also discuss in the discord if you have time). I'd love having this get merged and I believe that it'd make writing filters a lot more accessible, which in turns brings more people into Panda3D.

Also I have been working on a similar api but at the node level, I can make a pull request if you'd be interested. https://github.com/raytopianprojects/Effects

rdb · 2024-05-14T20:58:37Z

I'm not proposing we add JSON loading to this system—it was just an observation that it would be easy for a user to do this with this structure.

Another thought I have is that people might want to write filters in GLSL going forward, but I'm not quite sure how to address that issue here.

raytopianprojects · 2024-05-14T21:02:34Z

Yeah I've been thinking about GLSL authoring too.

I could spend the time converting the CG shaders to GLSL and then add the add the ability to switch between the languages but then again I don't know how many users are creating new projects using CG. So it might be worth just having GLSL as an option.

Ensure that each filter's sort is no longer a breaking change, add examples/test if CommonFilters is ran by itself, add printShader for to easily see internal shader code (should be changed to better work with Panda3D's own debug system), remove newly added filters (will be in separate pr), change how uniforms work so you just need to pass in a uniform as a dict instead of 3 separate args, and add reconfiguring to cleanup so the internal uniforms and textures dicts don't get removed when adding new filters.

raytopianprojects · 2024-05-22T02:07:32Z

I've made changes based on your feedback rdb. I'll be creating a separate new filters pr soon.

While it no longer does anything this should fix the last breaking change.

Fix delHighDynamicRange

Change imports back to being relative

Merge tonemapping code from tonemapping pull request.

raytopianprojects added 2 commits April 15, 2024 11:07

Add Custom Shader support to CommonFilters.py

ec9cb13

Allow users to add their own shaders to CustomFilters. Port all built in shaders to use new system. Additionally add Chromatic Aberration, Vignette, Tint, Distortion, Resolution, LUT, and Film Grain as built in filters.

Check if render_into shader is a string and compile it

9d7f900

WMOkiishi reviewed Apr 15, 2024

View reviewed changes

Remove VirtualFileSysten Typing

41c252f

raytopianprojects marked this pull request as ready for review April 16, 2024 14:46

raytopianprojects mentioned this pull request May 21, 2024

Implement Khronos' reference PBR tone mapping operator #1659

Closed

raytopianprojects added 2 commits May 21, 2024 20:10

Add force back to setsrgb

ab1a0bb

While it no longer does anything this should fix the last breaking change.

Update CommonFilters.py

9d506b5

Fix delHighDynamicRange

raytopianprojects mentioned this pull request Jun 4, 2024

More Filters in Common Filters #1662

Open

raytopianprojects added 2 commits September 5, 2024 07:56

Update CommonFilters.py

2cff041

Change imports back to being relative

Add Tonemapping Support

898e29b

Merge tonemapping code from tonemapping pull request.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Custom Shader support to CommonFilters.py #1643

Add Custom Shader support to CommonFilters.py #1643

raytopianprojects commented Apr 15, 2024 •

edited

Loading

WMOkiishi Apr 15, 2024

raytopianprojects Apr 15, 2024

rdb commented May 14, 2024

raytopianprojects commented May 14, 2024 •

edited

Loading

rdb commented May 14, 2024

raytopianprojects commented May 14, 2024 •

edited

Loading

raytopianprojects commented May 22, 2024

Add Custom Shader support to CommonFilters.py #1643

Are you sure you want to change the base?

Add Custom Shader support to CommonFilters.py #1643

Conversation

raytopianprojects commented Apr 15, 2024 • edited Loading

Issue description

Solution description

Checklist

WMOkiishi Apr 15, 2024

Choose a reason for hiding this comment

raytopianprojects Apr 15, 2024

Choose a reason for hiding this comment

rdb commented May 14, 2024

raytopianprojects commented May 14, 2024 • edited Loading

rdb commented May 14, 2024

raytopianprojects commented May 14, 2024 • edited Loading

raytopianprojects commented May 22, 2024

raytopianprojects commented Apr 15, 2024 •

edited

Loading

raytopianprojects commented May 14, 2024 •

edited

Loading

raytopianprojects commented May 14, 2024 •

edited

Loading