Sony Megatron Colour Video Monitor

[MajorPainTheCactus]

Great work @rafan thanks for this investigation, this is really fantastic stuff! I did have a feeling its was to do with this bit of the code, as in the colour space mapping and looking at your snippet of code I think I can see the bug already:

const vec3 dcip3_colour = (scanline_colour * k709_to_XYZ) * kXYZ_to_DCIP3;
         gamma_out = LinearToDCIP3(dcip3_colour);

As in this kXYZ_to_DCIP3 transform is doing a LinearToDCIP3 conversion so we’re effectively doing it twice it looks to me. I’ll look at the code and confirm though. However this doesn’t explain the issue I was seeing as I wasn’t using DCIP3 (I think! hmm). I will look at this as soon as I can I’m just in the middle of a large software project.

As for Dogway and Guest code I certainly based my first attempt on Dogways (stipping out all the stuff I didn’t want/need) but I kind of ditched most of it when I came across the Kronos Groups transforms and various issues I had related to what you’ve said and other things. It may structurally bear some resemblance but the important stuff is all based on Kronos Group examples. I did look at Guest as well and have taken some ideas where they made sense along with other shaders Ive looked at. We all stand on the shoulders of giants at the end of the day and add our own twist.

As for grouping stuff in the same pass this is all down to performance - the first passes are at 240p (fast), the main pass at 4K (slow) and you just have to do things in certain orders rather than all up front. If you have any specific sugeestions I can certainly take a look at them though.

Thanks again for this and hopefully I’ll get some time to fix these issues in the next week or so.

[MajorPainTheCactus]

Amazing photo and now I think I can see where those incorrect sub pixels are coming from - they just look like the white sub pixels are being triggered in the whites of eyes so maybe there’s not an issue here! Woo hoo! Does it look alright?

[rafan]

Hi @MajorPainTheCactus,

Thanks again for investigating and being around here to answer questions. Now that I’ve been using your megatron a bit more and getting used to it, I’m really liking it. It’s a quite nifty approach to simulating our beloved CRTs on modern display devices. Your work on this is really appreciated.

With regards to the colour gamut stuff, the mapping from one color space into another is done via the CIE 1931 XYZ color space. I don’t see a gamma correction being done in kXYZ_to_DCIP3, while LinearToDCIP3 does it explictly, so I’m not sure something is done twice.

I found the following video a nice visualisation of how the colour gamuts are projected within the XYZ space as it also visualizes the spectral locus. Well worth a watch:

https://www.youtube.com/watch?v=x0-qoXOCOow

Since this colour mapping seems problematic in the shader it may have to do with the way gamma correction is integrated in the gamut mapping? From what I understand the matrix 709(RGB)toXYZ * XYZtoDCI(RGB) maps the 709 “cube” into the DCI-P3 “cube”, such that when you display your content on a DCI-P3 monitor your content is shown as rec709 saturated content.

But how is gamma correctly integrated into mapping one colour gamut into another one? Does the Kronos Group examples spell that out by any chance? There’s both a mismatch in gamma functions and values. I can imagine when using a 709 gamma value of 2.2 and a DCI-P3 gamma of 2.6 (as per spec), is there an implicit gamma correction of 0.4 in this colour gamut mapping or how does that work?

Edit: to complicate stuff my monitor is a wide gamut monitor, targeting DCI-P3, but its default “average” gamma is 2.35.

[MajorPainTheCactus]

Thanks for the video I’ll take a watch as for your questions I’d have to get my head back into the shader and those matrices and transfer functions again. In the mean time here’s the Kronos Groups low down on them chapter 13 is all the transfer functions and 14 are all the colour primaries conversions:

https://www.google.com/url?q=https://registry.khronos.org/DataFormat/specs/1.2/dataformat.1.2.pdf&sa=U&ved=2ahUKEwilsMeM6Jf6AhVtQEEAHZ2tCBsQFnoECAYQAg&usg=AOvVaw2B53QFDWWO_j2LvPFej4Nv

I vaguely remember the transfer functions working on the ‘intermediate’ XYZ colour space although I could be completely wrong and they work on the input/output RGB spaces - it’s definitely one of the two!

[rafan]

Thanks for the link to the Khronos document.

So the transfer functions work on the input and output RGB spaces.

As far as I can see it’s not very explicit about what gamma is applied on the input RGB (to make it the required linear input) and what gamma is applied on the output. On the Bruce Lindbloom site it mentions in both the RGB to XYZ section and XYZ to RGB section that “the operation depends on the companding function associated with the RGB color system.”

Taken literally, in the case of converting the 709 space/primaries to DCI-P3 space/primaries one would implicitly create a gamma correction, as input linearization is done with rec.709 gamma ~2.2, but output is done with DCI-P3 gamma of 2.6.

That seems a bit strange… If I look at guest.r method, he only uses the output space gamma value for both input and output (so 2.6 in case of DCI). Seems to make more sense, since you’re not implictly doing gamma correction, while remaining the transformation of the color primaries.

Also what I’m reading in the Bruce Lindbloom site is that the input and output space need to have the same whitepoint defined or otherwise the difference needs to be accounted for via chromatic adaptation (read: complicating things further?). I’m not sure how your whitepoint adaption in your shader ties in with the color space transform, but possibly it’s something to keep in mind?

Unfortunately most of the above doesn’t seem to help much with the shader issues we discussed, I think…

[MajorPainTheCactus]

So you need to convert into linear space to convert your colour primaries and then back out into the accompanying gamma space.

I might be able to look at this tomorrow and just see if I can spot on anything. Hopefully shouldn’t be too hard to figure out what’s wrong here.

[MajorPainTheCactus]

@rafan so found the issue! My gut instinct was right - it was a precision issue and a complete school boy error by having my SDR linear space buffer in low precision. Upped it to float 16 and bang fixed the weird issues.

So one thing you’ll notice now is that those bars in SDR are still very dark however they aren’t clipped now, they just contain very dark values. If you remove the sRGB gamma clip which is part of the sRGB standard and just replace it with a direct pow(X, 2.4) they pop back up but I’m keeping with the standard. I can elaborate on this if you’d like to try yourself.

HDR is unaffected by this as it uses the PQ transfer function. Although it would have been affected by the precision bug.

All in all this fix should be a large quality improvement so thanks rafan!

The pull request is Sony Megatron 3.5.

LCD Photos: OnePlus 8 Pro Camera: Pro Mode, ISO 400, WB 6500K, Aperture Speed 1/60, Auto Focus, 48MPixel JPEG.

[Cyber]

@MajorPainTheCactus

All in all this fix should be a large quality improvement so thanks rafan!

The pull request is Sony Megatron 3.5.

Quality improvements welcome! Thanks @rafan & @MajorPainTheCactus! Gorgeous screenshot by the way!

[GPDP1]

@MajorPainTheCactus Just thought I’d let you know it appears the BGR version of the 8K RYCBX mask just repeats the RGB version. It ought to output BCYRX, but ends up outputting RYCBX anyway. Fixing it is as simple as editing line 378 from

kRYCBX, kRMCGX, kRYCBX

into

kRYCBX, kRMCGX, kBCYRX

[MajorPainTheCactus]

Good catch! Thanks - I’m sure there are more of these lurking around what with all the tables!

[rafan]

That’s awesome @MajorPainTheCactus, gamma issues are fixed now! Thanks!

One issue remaining is the subpixel output when setting colour space to DCI-P3

"SDR: Display's Colour Space: r709 | sRGB | DCI-P3" to DCI-P3.

Then I’m getting a messed up subpixel output like below. It’s most visible when setting TVL to 1000.

whereas with Color Space set to 709 or sRGB I do get a clean subpixel output:

Could you test this by loading “crt-sony-megatron-sony-pvm-2730-sdr.slangp” preset, then -only- change “SDR: Display’s Colour Space” to DCI-P3 (setting 2.00), TVL to 1000 and check the subpixel output on a white screen?

Edit: I mentioned before that I’m testing this on 1440p monitor with native RGB subpixels.

Edit 2: a picture of the native monitor RGB subpixel layout

[rafan]

Some additional test results to previous post, but now for HDR mode.

When using RA in HDR mode and loading 2730-HDR preset, whatever setting I change, the mask output is incorrect, similar to SDR mode in DCI-P3, e.g. for Aperture Grille| 4K | 1000TVL:

Since HDR mode is ALWAYS doing the color gamut transformations (either 709 to 2020 space or “extended/vivid” to 2020 space) it seems that this colour gamut mapping is a source of messing up the mask output.

I think replacing the color transform matrix in the HDR shader with a unity matrix (i.e. it would pass native colors without doing any transform) could be good test case, to see whether then the mask output returns correct. That would confirm the transform between different gamuts is causing an issue.

Edit: So I’ve just tried it, replacing the transform matrix for 709 and expanded709 in include/hdr10.h

Note the matrices seem to already be the product of RGBtoXYZ and back out XYZtoRGB, i.e. 709(RGB)toXYZ * XYZto2020(RGB), both for 709_to_2020 and also this seems the case for the Expanded709_to_2020 matrix

/*
const mat3 k709_to_2020 = mat3 (
   0.6274040f, 0.3292820f, 0.0433136f,
   0.0690970f, 0.9195400f, 0.0113612f,
   0.0163916f, 0.0880132f, 0.8955950f);

/* START Converted from (Copyright (c) Microsoft Corporation - Licensed under the MIT License.)  https://github.com/microsoft/Xbox-ATG-Samples/tree/master/Kits/ATGTK/HDR */
/*
const mat3 kExpanded709_to_2020 = mat3 (
    0.6274040f,  0.3292820f, 0.0433136f,
    0.0457456f,  0.941777f,  0.0124772f,
   -0.00121055f, 0.0176041f, 0.983607f);
*/

Replace by a “unity” matrix, such that there’s no transform and native colors get passed on:

const mat3 k709_to_2020 = mat3 (
 1.0f, 0.0f, 0.0f,
 0.0f, 1.0f, 0.0f,
 0.0f, 0.0f, 1.0f);
 
const mat3 kExpanded709_to_2020 = mat3 (
 1.0f, 0.0f, 0.0f,
 0.0f, 1.0f, 0.0f,
 0.0f, 0.0f, 1.0f);

And indeed the mask issue is solved, the shader subpixel output is clean RGB (RBG):

Now the million dollar question, can you think of any reason why color gamut mapping (e.g. 709 to DCI-P3 (in SDR mode) or 709 to 2020 (in HDR mode) in the shader is causing the mask output to get “messed up”?

It would be really nice if this could be resolved or at least understood why it is causing this result.

[MajorPainTheCactus]

Hi rafan great stuff so we’re making progress. So aperture grille 4K 1000TVL should be selecting a Magenta Green mask but what you’re seeing is yellow and cyan in there too? This is odd. Maybe I haven’t taken into account the relative recent conversion into XYZ but then I’d expect that to be influencing every mask. I’ll take a look my end.

[MajorPainTheCactus]

Hi @rafan, so although my camera isn’t good enough to capture the sub pixels of the high density 1000TVL mask on my 27" 4K monitor I am able to take a screen grab of the shaders output and it looks like this:

which looks correct as in its a pure magenta green mask which is what the 4K 1000TVL options would select as a mask.

Also the maths all works out I think. The main shader (last pass) deals with rec. 709 colour primaries (see line 158 in colour_grade.h) not XYZ colour space and so all of the last colour gamut transforms should be from rec. 709 space (sRGB uses the same colour primaries as rec.709 it just uses a different gamma curve).

If you put those matrices to identity then you’ll still be in rec .709 space and so (I’d guess) your monitor is in that space and not DCI-P3 or rec.2020 and hence why it looks correct using identity.

So I say all that with the massive caveat that I’ve understood you correctly - I can misread things.

[MajorPainTheCactus]

Just released Sony Megatron 3.6 which fixes the 800TVL BGR aperture grille mask bug that @GPDP1 pointed out above. Thanks GPDP1 greatly appreciated! Also tweaked the shader parameters to be more intuitive with respect to gamma in/out and HDR/SDR (as gamma out isn’t a thing for HDR).

Also recalibrated the Sammy Atomiswave presets to try and get closer to the arcade screen with the new high precision gamma:

Much better reduction in detail that the arcade screen ‘suffers’ from - you can see the arcade version (and my previous attempt) right at the top of this thread. Its certainly better I think.

[rafan]

Hi, thanks again for looking into this.

I’m having the issue not only with 1000TVL, it’s with all the TVLs.

When the shader is in sRGB mask output is fine. If then the only change I do is put the shader setting colour space to DCI-P3 then the mask output goes awray.

Example: I load up your 2730-SDR preset and change TVL to 300, then this is my output (iphone protograph so not the best, but I verified with magnifying glass). It’s great!

Now I F1 and ONLY change colour space to DCI-P3 in the shader. It’s really the ONLY change I do, then output becomes this:

It’s quite bad as you can see, next to the red other subpixels light up, and next to green the same.

If I then F1 again and change colour space in the shader back to sRGB then output is good again (like the first picture).

It looks like mapping a color space into another in the shader affects the mask. As long as NO color space is set, everything looks OK.

Can you do the following on your end: load 2730-SDR preset on your end, change TVL to 300. Make a picture, then go into shader settings and change colour space to DCI-P3, make a picture again. You must see a difference, or not?

[MajorPainTheCactus]

I’ll try that specifically but I’ll be honest I’m not seeing that much of a change flipping between them so far.

One thing I would say though is that you should be setting your monitor to the corresponding colour space too - what happens if you select DCI-P3 in your monitor settings and flip between the two?

Its kind of a kin to selecting a HDR shader when your monitor is in SDR mode. It’ll look terrible.

I’m going to sleep right now but I’ll try and get some time tomorrow to do these tests for you.

[rafan]

@MajorPainTheCactus

I’ll try that specifically but I’ll be honest I’m not seeing that much of a change flipping between them so far.

Thanks, that’s appreciated. Possibly you could test with the 240p test suite “White & RGB Screen” test. If you press “W” on the keyboard you can cycle between White, Black, Red, Green, Blue screens.

I can clearly see unwanted subpixels light up on the Green subscreen when switching to DCI-P3 color in the shader.

Just out of interest, are you testing on a wide gamut monitor or sRGB monitor?

[rafan]

Hi @MajorPainTheCactus

I’ve got an additional more objective test related to the colour space settings of which the result seem strange: the luminousity of a pure white screen changes when cycling between the 3 colour spaces: the white screen shader output gets brighter with each change from colour space 709 to sRGB to DCI-P3

Would be great if you could do the following test:

First make the RA menu transparent such that when changing the shader colour space setting you see the result while in the menu, it makes the luminousity changes easier to spot when cycling through the colour spaces.

Go to Main Menu -> Settings -> User Interface -> Appearance -> Framebuffer Opacity and set this to a value of 0.00 (menu becomes fully transparent such that you can see the shader output when in the menu)

Start the SNES 240p test suite and go into test patterns “White & RGB Screens”, stay on the white screen.

Load the megatron 2730-sdr preset, go into the shader menu and cycle between the three colour spaces:

SDR: Display's Colour Space: r709 | sRGB | DCI-P3

You’ll see the luminousity/brightness of the 240p white screen increase with each step: r709 darkest, sRGB brighter, DCI-P3 brightest

Now since the white screen has no saturation I assume the luminousity of a full white screen should definitely not change when changing the colour space.

The only thing I could think of influencing the result is that the 709 and DCI-P3 matrix would be created with different white points, but as far as I can see they are both created with D65 white.

To be absolutely sure, Ive repeated above test on a completely different monitor (1080p sRGB), and the exact same happens, the luminousity of the 240p full white screen test increases with changing the shaders colour space settings…

Any idea what could be an explanantion or cause for this?

[MajorPainTheCactus]

Yes so this is expected and is kind of a problem with the limited interface we have for the shader params. Each of the colour spaces, rec.601, rec 709, sRGB and rec.2020 isn’t just colour primaries it’s also a transfer function too and from optical and electrical spectrums. With rec.709 and below this a gamma curve, with rec.2020 this is a PQ function.

When you cycle through the colour systems youre also changing the gamma curve which is why we see the change in luminosity. Rec.709 is ^2.22, sRGB is ^2.4 and DCI-P3 is ^2.6. Because I can’t change values in the menu based on other values in the menu behind the scenes I add 0.2 to gamma out for dci-p3 and minus 0.18 for rec.709 onto the menu value of 2.4. Ideally we’d just change the menu values for gamma when we change colour space.

However I do see your point in that on a pure white screen we shouldn’t see this as everything would be at 1 but is everything at 1? As these masks mask out channels to red, blue and green and then we have the drop off in intensity from the center of the scan line to the outer part. I’d guess at this point you will see a change in luminosity for certain.

So based on that there maybe a good argument for changing into the output colour/gamma space before we do the main shader. I’ll try that and see where it gets us in terms of white stability.

Thanks for these tests and thoughts it’s great appreciated. I will try this out hopefully today.