-
Notifications
You must be signed in to change notification settings - Fork 363
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Transformer] The color of BitmapOverlay is a little off #1050
Comments
Hi @DeweyReed, thank you for reporting. If it is possible, could you share the input files? Or a screenshot with the input bitmap compared with the output overlay |
I have a somewhat related issue. The result of the shaders applied through the Transformer and applied to a GLSurfaceView is different: #1080 Maybe the texture colors are modified when applying a shader? No idea what is happening. |
You mention that the this may have to do with the color space. What color space is your input media using? The input colors expected by overlays is sRGB, per the code you linked. This is different from the color-space videos tend to use, of smpte 170m, so we convert in OverlayShaderProgram to get a consistent intermediate colorspace for mixing. Also, to make sure I'm understanding correctly, am I right that the main video's colors are correct, and the overlay is experiencing some color shift? |
@dway123 The color space of my overlays is sRGB. I get bitmaps from drawable resources and use them as bitmap overlays. The code I use is pretty simple: Transformer.Builder(this).build().start(
EditedMediaItem.Builder(
MediaItem.Builder()
.setUri(getResourceUri(R.raw.video))
.build()
).setEffects(
Effects(
emptyList(),
buildList {
this = OverlayEffect(
ImmutableList.of(
BitmapOverlay.createStaticBitmapOverlay(
drawable(R.drawable.lut).toBitmap(),
),
)
)
},
)
).build(),
File(filesDir, "output.mp4").canonicalPath
) I tried to adjust |
Oops sorry I missed this issue. Does this improve if you replace fragment_shader_transformation_sdr_oetf_es2, L71 here with I wonder if the issue is that we're using an incorrect color transfer for video display here (relevant internal link: http://b/310160336). |
@dway123 Thanks for your response. After changing the output color transfer to 10 using the debugging mode, there are some subtle color changes, but it's still different from the original image. The video is an SDR video. It's a low-resolution and clipped version of Big Buck Bunny. I can reproduce the situation with any video. Overlaying the image mentioned above on top of the resulting video makes the difference clearer. |
Hmm thanks, I think @tof-tof is more knowledgeable about overlays, so I'll defer to them here. |
hmmm... we are going to undergoing a project this quarter that effects how we will process colors in the pipeline. I'll check with this image after and see if it fixes the issue. An aside: we currently have |
Hi, I created a reproducible repo. https://github.com/DeweyReed/TransformerOverlay |
@DeweyReed could you let us know when you verify, so we can close the issue |
The second stage of the changes remove the conversion to linear colors in the SDR effects pipeline by default. also resolves Issue: #1050 PiperOrigin-RevId: 630108296
Thank you! @droid-girl It's better than before, but there is still a tiny difference. However, I'm unsure if it is a problem with my monitor or a lack of required modification to my code. |
I am closing this issue for now. Please reopen for any follow up work |
Recently, I finally found some time to continue the work on video editing. The result has been improved a lot after using Thank you very much for your patience and excellent work! Transformer.Builder(this)
.setVideoFrameProcessorFactory(
DefaultVideoFrameProcessor.Factory.Builder()
.setSdrWorkingColorSpace(
DefaultVideoFrameProcessor.WORKING_COLOR_SPACE_ORIGINAL
)
.build()
) |
Hi @DeweyReed! Has it improved colors of a BitmapOverlay for you? I tried the same settings for VideoFrameProcessorFactory and it changed nothing for my BitmapOverlays 😐
|
@yuriikonovalov I used a custom shader, but a BitmapOverlay made the issue more noticeable. You may want to open a new issue if the change of color spaces doesn't work. |
Hi, thank you for the excellent work!
I'm using media3(1.3.0-alpha01) to add a bitmap overlay to a video. However, the resulting overlay slightly differs from the original file. For example, the blue is a little dark, and the green gets lighter. Does this problem have anything to do with the color space? How to solve it?
The text was updated successfully, but these errors were encountered: