Some additional test results to previous post, but now for HDR mode.
When using RA in HDR mode and loading 2730-HDR preset, whatever setting I change, the mask output is incorrect, similar to SDR mode in DCI-P3, e.g. for Aperture Grille| 4K | 1000TVL:
Since HDR mode is ALWAYS doing the color gamut transformations (either 709 to 2020 space or “extended/vivid” to 2020 space) it seems that this colour gamut mapping is a source of messing up the mask output.
I think replacing the color transform matrix in the HDR shader with a unity matrix (i.e. it would pass native colors without doing any transform) could be good test case, to see whether then the mask output returns correct. That would confirm the transform between different gamuts is causing an issue.
Edit: So I’ve just tried it, replacing the transform matrix for 709 and expanded709 in include/hdr10.h
Note the matrices seem to already be the product of RGBtoXYZ and back out XYZtoRGB, i.e. 709(RGB)toXYZ * XYZto2020(RGB), both for 709_to_2020 and also this seems the case for the Expanded709_to_2020 matrix
/*
const mat3 k709_to_2020 = mat3 (
0.6274040f, 0.3292820f, 0.0433136f,
0.0690970f, 0.9195400f, 0.0113612f,
0.0163916f, 0.0880132f, 0.8955950f);
/* START Converted from (Copyright (c) Microsoft Corporation - Licensed under the MIT License.) https://github.com/microsoft/Xbox-ATG-Samples/tree/master/Kits/ATGTK/HDR */
/*
const mat3 kExpanded709_to_2020 = mat3 (
0.6274040f, 0.3292820f, 0.0433136f,
0.0457456f, 0.941777f, 0.0124772f,
-0.00121055f, 0.0176041f, 0.983607f);
*/
Replace by a “unity” matrix, such that there’s no transform and native colors get passed on:
const mat3 k709_to_2020 = mat3 (
1.0f, 0.0f, 0.0f,
0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 1.0f);
const mat3 kExpanded709_to_2020 = mat3 (
1.0f, 0.0f, 0.0f,
0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 1.0f);
And indeed the mask issue is solved, the shader subpixel output is clean RGB (RBG):
Now the million dollar question, can you think of any reason why color gamut mapping (e.g. 709 to DCI-P3 (in SDR mode) or 709 to 2020 (in HDR mode) in the shader is causing the mask output to get “messed up”?
It would be really nice if this could be resolved or at least understood why it is causing this result.
).
