postprocessing: LUTEffect produces incorrect results

I’ve investigated this further and ended up implementing tetrahedral interpolation and custom input domains. I found out that the noisy errors we’re still seeing are indeed caused by hardware interpolation. This article by iq explains the details.

The tetrahedral interpolation algorithm performs all interpolations manually to avoid the quantization of fractional components. The following comparison shows the difference between the input colors and the output colors of the LUTEffect using a neutral 2³ LUT:

HW Trilinear 2D	HW Trilinear 3D	Manual Tetrahedral 3D

Tetrahedral interpolation eliminates all remaining errors.

@gkjohnson FYI: I learned that fract() actually uses floor() internally: “This is calculated as x - floor(x).” So the following is probably a bit more efficient for the 2D sampling algorithm:

float slice0 = floor(slice);
float interp = slice - slice0;

Good to know that looks like a useful site!

If the visual result looks good I think I’m content to leave it for now. I won’t rule out the sampling function still having some issues but with more calculations happening in the 2d sampling case I’d imagine you’d get some differences from floating rounding error, as well, so aiming for literally identical images may be an impossible goal.

And after taking a quick look at the TetrahedralUpscaler again I don’t think the sampling approach needs to be adjusted as I suggested in https://github.com/vanruesc/postprocessing/issues/250#issuecomment-755074282 and the visual result looks okay so I think that component is good.

while upscaling makes no difference whatsoever (as long as tetrahedral interpolation is on)

I wouldn’t expect tetrahedral upscaling on the CPU to make a difference if you’re using tetrahedral sampling on the GPU. Tetrahedral sampling in the shader is likely much more expensive than just performing built in trilinear sampling with the 3d texture so it’s a trade you can make.

Looks great! And thanks for the links and extra research.

@vanruesc My main concern was that the previous implementation was, frankly, pretty close to being unusable. When creating luts in separate software, there are certain expectations as to how the LUT will behave, and having banding (!!) or differences in colors was making it questionable whether trying to use this was worth the trouble.

Looking at the last demo, all of that has been fixed. My only use case is color grading, so I see no issues whatsoever. This is absolutely fantastic, and all the extras like upscaling make me so happy. For me, there’s nothing left to do, it is perfect.

That said, I do have some additional comments, but all of this is extra that won’t make my life any easier.

When it comes to working with video, there’s this idea of having generational stability. Some explanation:

The peak signal-to-noise ratio (PSNR) of the NDI codec exceeds 70dB for typical video content. Uniquely, and importantly, NDI is the first ever codec to provide multi-generational stability. This means that once a video signal is compressed, there is no further loss. As a practical example, generation 2 and generation 1000 of a decode-to-encode sequence would be identical.

This may sound too good to be true, and it is. NDI in obs-studio actually fails completely to do that, so there is more marketing fluff in that than actual substance. As you can imagine, it may be related to colorspace conversions, and maybe the actual issue is not in NDI itself. But their SDK is not open-source, so who freaking knows. (By the way, here’s a fun story about their SDK and GPL violations)

But this brings the question. If you take an image that has all possible sRGB colors (let’s say a PNG from here), and apply a neutral LUT on that, what happens? Does it produce exactly the same output on the first generation? If not, when does it stabilize? I could try this myself but I guess you were already writing a test for doing something like that.

My understanding is that it does not produce the same result on the first step. Testing this seems to be surprisingly easy in the demo itself. In the blend mode, choose DIFFERENCE (other options like SUBTRACT also seem to work?). You’ll see this:

So what’s the meaning of the DIFFERENCE blend mode in this case, and why are we seeing this?

As for comparing the images, the easiest tool is probably imagemagick’s compare:

compare a.png b.png diff.png

And here’s the output:

(3D texture is not selected on the screenshot, but the result seems to be the same regardless of that setting)

While this is completely unnoticeable to the eye, a lot of pixels seem to be off. Generational stability does matter in some use cases. Let’s say you were making a LUT-based node editor to create some sort of image processing tool in three.js. If there’s no generational stability, then just adding more nodes (even if they do nothing) will increasingly distort the final image.

Note, however, that even if that is fixed, more extreme LUTs can later reveal more issues. That is, the values in the LUT can be very peaky so to say, so that the interpolation between the values has a much harder time. The testing will need to be done in two steps though, so the first LUT should distort the colors to some precisely predicted values, and a second LUT will then need to restore them. This is actually a fascinating topic, it’d be very interesting to create a sort of LUT stress test suite and see how different software behaves. However, that is a completely different can of worms and I personally don’t need any of that, although I’m very curious.

This should now be fixed in postprocessing@6.19.1.

@AlexDaniel It would be great if you could take a look at the updated demo to check if you notice any remaining issues. I’ll keep this issue open until we’re certain that everything’s fixed.

@gkjohnson I’ve added a unit test for the TetrahedralUpscaler which confirms that it generates accurate data. The test scales a neutral LUT of size 4³ up to 32³ and compares it with a baseline neutral LUT of size 32³. The difference of individual values is below 1e-7 which is floating point inaccuracy territory.

@gkjohnson Thanks for the super fast fix and great explanation!

I’ve tested the new shader with a neutral LUT applied to linear input colors and found no issues.

Size	3D sampler	2D sampler
2³
32³

The artifacts are gone and the results match perfectly at both sizes. I’ve also tested with sRGB input and saw the same results.

When the neutral 2D/3D LUTs are applied to 0x151515, we get these results:

Size	3D sampler	2D sampler
2³	0x151615	0x161615
32³	0x151515	0x151515

The small difference at 2³ is interesting but shouldn’t be a problem.

One thing I had to change in the shader was this modf which is not supported in WebGL 1. I replaced it with this:

float interp = fract(slice);
float slice0 = slice - interp;