Dogway's grading shader (slang)

@Dogway

I’ve been thinking for a while that the default for vignette toggle should be off, and the default for phosphor should be 0.00. What do you think?

Also, a version of the shader without the LUT stuff would make it so much easier to combine with other shaders; just something to consider.

2 Likes

Everytime he’d do an update I’d gut the LUT code from the pass lol

Iirc I’d gut a few things each time.

2 Likes

Just wanted to mention that in the next version of the Mega Bezel the tweaked version of grade I use will have all the LUT stuff removed (Right now it’s turned off but there is still a reference to the textures)

So there is the possibility of using this version, but this probably won’t help the average user who doesn’t know about it.

6 Likes

If I stop being a giant potato, I’ll update my modified grade and send it to you @Nesguy, iirc I removed the LUT code, the vignette, and made some minor altercations to the saturation settings…

2 Likes

Trying to understand exactly how I’m supposed to use this shader. If I have HDR turned on in Retroarch (and thus on my tv), should I use this shader set to rec.2020?

When the shader is set to sRGB the different phosphor settings are only barely different but when set to rec.2020 the difference is more extreme where ntsc and pal (1-3) settings seem desaturated compared to the default 0 setting.

I updated Grade with what I’ve learned in the last 2 years. Following a summary.

After a long quest to find a trustful NTSC-J phosphor primaries I settled on a mix of various correlated ones, and comparing them to typical phosphor chemical chromaticities. It’s not a big departure from what was before but it should be more accurate now.

The main change in this version is the White Point or temperature adjustment. The function itself is not a big change, it’s simply more accurate after a custom modelled daytime locus and using CAT16 instead. But the implementation within the shader differs.

The other big change was getting rid of FCC and YIQ, they were barely used and only up to 1979. As I’m targeting NES, MS and above I don’t want to bulk up the code too much. YPbPr was also removed until I find or confirm it was indeed used in the analogue system.

And before I miss it a third big change was the gamma handling. I implemented the 1886a transfer function which is an accurate representation of a CRT gamma. In that regard with the recommended and default black level of 0.1 cd/m^2 it matches 1886 which is a simplified CRT like transfer (gamma of 2.4), but I added knobs to adjust the black level, brightness and contrast controls in 1886a in a CRT fashion. That means that adjusting the ‘black level’ to 0.0 leads to a gamma of 2.6, and same with the ‘brightness’ (aka black ‘lift’). ‘Contrast’ is a CRT gain adjustment.

Overall everything makes more sense and performance optimizations have been done all around.

It’s tagged as RC4 so please try it and give me your feedback. I also uploaded a version without the LUTs and all parameters set to noop (OFF) by default.

11 Likes

I did some comparisons with my old own presets and I dig it. The white point function now works correctly and actually shows a pleasant neutral look on the images so not as blue as before. In order to get the old “wrong” but nice looking electric blue temperature I added the parameter called “CRT Beam”, each for R,G and B. It made sense to add this as you can now control the power emmited by the elctron gun for each one. In the same sense it can work as a way to manipulate phosphor decay. As you know red phosphors were one of the first to decay so that’s why some CRTs displayed that electric blue.

Here’s a comparison with Sonic title screen (sorry I didn’t disable the sprite limit option).

The trees’ green is not so electric, the yellow ring is more pure (less green) and you can actually now see the shades in sonic’s blue (check the spikes). Aside from the “Blue-Green Tint” option I basically reduced the overall saturation as a side effect of the analogous system. In this way the clouds now pop out more, the red ribbon isn’t so glaring and the belly and mouth are more skin like toned.

#reference "shaders_slang/misc/grade_new.slangp"
g_space_out = "-1.000000"
g_Dim_to_Dark = "0.000000"
g_U_MUL = "0.950000"
g_V_MUL = "0.950000"
g_CRT_br = "0.750000"
g_lum_fix = "1.000000"
rg = "0.020000"
bg = "0.055000"

Now with SMW. In this case the overall look is not as pale or washed out, that is more neutral like, even so I could still raise saturation and “CRT Beam Red” to make it pop out a little more. The main thing to observe here is the cloud rim, see how now the two blue shades are distinguished when they make the transition from white to the sky’s blue, so they give a better illusion of blending with the background.

noop

Old preset

New preset

#reference "shaders_slang/misc/grade_new.slangp"
g_space_out = "-1.000000"
g_Dim_to_Dark = "0.000000"
g_U_MUL = "0.900000"
g_V_MUL = "0.950000"
g_CRT_br = "0.700000"
g_sat = "-0.050000"
g_vibr = "-1.000000"

Finally I googled up on the phosphors of the Toshiba A68KJU96X used in the Nanao CRT’s but couldn’t find anything relevant. What I found is that they seemed to use an illuminant of 0.285,0.285 (about 9700K) so I’d say that for arcade systems up to Naomi or so (rear projection CRTs) you can use the NTSC-J phosphors with a temperature of 9696K.

If there’s no inconvenience I will make RC3 final and update all my uploaded presets.

9 Likes

Hi, great to see you here, and I’m really excited about the update!!!

I have a question about gamma. There previously were a gamma in and electron gamma out parameters and it seems these are not used anymore and that black level of 0 is equivalent to an input gamma of 2.4 and an electron gun gamma of 2.6 and then the Contrast equates to a gamma adjustment.

I’m thinking about how to transition Mega Bezel users… So if someone wanted to have the equivalent of 2.4 gamma in and 2.4 electron gamma, would this be a contrast of -0.2 ?

4 Likes

With the default CRT black level of 0.1 you get a gamma of 2.4, which is the standard for Rec709/Rec2020 and the 1886 transfer function for dim surrounds, except if you enable the dim to dark option which takes it to 2.45 gamma for the mentioned spaces.

So now you control the system emulated gamma with the CRT black level option. 0.1 being 2.4, 0.0 being 2.6 and varying in between. I can make a function fit to remap the values so the effective gamma can be shown in the OSD for adjustment purposes (check at the bottom).

One thing I didn’t mention is that phosphor dynamics apply their own gamma if they are non-linear. Here is a summary.

So for example 2.4 converts to 2.75 with a phosphor gamma of 1/0.88. To get an effective gamma of 2.4 you would need to set the CRT gamma to 2.1 but you can’t reach that with CRT black level limit of 0.35. Only when phosphor dynamics are 1/0.9 you can set CRT black level to 0.35 and match the gamma of 2.55=2.3^(1/0.9)

(EDIT: or leave black level at 0.1 and raise brightness to 30 as per the below expression)

Scanline shaders should notify the gamma change when phosphor dynamics are applied, so you can make more sense of the whole system gamma.

To note “CRT black level” won’t make a big change on the black lift/pedestal, it’s mainly for calculating gamma, to actually raise the pedestal you use CRT brightness, but obviously any change in range will modify the gamma which is assumed.

Here is the expression for the effective gamma:

bl=0.1   # Black Level
br=0     # CRT Brightness
bl_gamma = 1.0/(-0.054348*pow(exp(bl)    ,-9.45461)+0.437886)
br_gamma = 1.0/( 2.057430*pow(exp(br/10.),0.028201)-1.060440)
gamma    = pow(bl_gamma,br_gamma)

Then you have to account for the phosphor dynamics gamma on top.

sys_gamma = pow(gamma,ph_gamma)

2 Likes

So in layman’s terms would it be possible to setup the new shader in a way that mimics the incumbent ones or provides a toggle to enable the old behavior so as to more or less act as a passthrough to allow existing Mega Bezel Reflection Shader presets to look the same as before in order to smooth/ease the transition?

This might be necessary for the newer implementation to not break any existing custom fine tuned black level, colour temperature or gamma adjustments which might have been made prior to these developments.

Other than that if integrated in the current form, the effects on the existing ecosystems of presets might be a bit unpredictable, as in might be better in some instances (best case) but might also be worse in other instances (worst case) and require a retuning/calibration of all existing presets.

1 Like

My recommendation is to use the CRT controls in detriment of the digital ones, so “CRT Brightness” as opposed to “Black Level”.

The changes are disruptive but you can approximate them as I showed with the examples above. That’s why I added the “CRT Beam” parameters.

In regards to gamma, this is something I wouldn’t obsess too much. The gamma would also change if you used the digital “Black Level” as per the listed equation. The only thing to remember here is that “CRT Black Level” (I wouldn’t touch it too much) is what drives the CRT gamma (the bl_gamma function) and that phosphor adds an additional gamma on top which is undocumented.

For example, with the default(?) phosphor beam gamma of 0.88 to get an output of 2.4 you’d need to set “CRT Brightness” to 30, and you are good to go.

If you want I can make it work backwards, reverse bl_gamma to solve for “bl”, so you input the gamma and you get the “CRT Black Level”:

bl = -(100000*log((72981-500000/(3*bl_gamma))/9058))/945461

I think this is more intuitive for people. I updated it here.

2 Likes

Thanks, when I was referring to “Black Levels” I wasn’t specifically referring to the Black Level control or parameter as that’s something I have never touched.

Rather, I was referring to the resultant effect on Black Levels and Shadow/Dark Detail levels after tweaking things like Gamma In, Gamma Out, Post CRT Brightness, Gamma C, Bright Boost Dark Pixels.

Some of what you said might be slightly above my pay grade so it might be better for @HyperspaceMadness to give an input as to whichever implementation or mitigation might be the preferred or more intuitive method if one had the goal of easing the transition.

I know he always strives for consistency. I will just work to suit with whatever I’m provided.

Thanks again for your prompt and detailed response.

1 Like

Man I’ve been away too long, all these new shader updates left and right I feel like I’m in a candy shop. @Dogway your shader has pretty much been the back bone to my shader presets and I love it so I’ll be trying this new updated grade shader out when I get a chance. Thank you for your amazing work.

2 Likes

Updated to RC5, running out of ideas so will probably update this to official repo tomorrow.

A few things I did here, defaulted gamma to 2.5 to don’t assume beam dynamics. Fixed “Dim to Dark” to “Dark to Dim” as the spec. And the main thing, fixed the ugly artifact of the vignette showing a hard cut where the vignette ends by using a smooth roll-off.

Something very funny that I encountered by chance is that when using the NTSC-J phoshpor gamut SMB’s sky turns blue.

I still have issues with the glsl versions so maybe someone might have a look if I’m missing something.

4 Likes

Updated to RC6, update to this version since I forgot OpenGL was column-major.

Now I double checked with my AviSynth benchtest and everything matches, including temperature.

I also updated the GLSL versions and now should work correctly.

Opinions on how to call each grade variant? grade.slang and grad-LUTless.slang?

4 Likes

Maybe grade-no-LUT.slang ?

3 Likes

This sounds great and is pretty clear.

1 Like

Thanks, straight forward.

I will update grade tomorrow probably due to inherent aspects of embedded color intents.

So probably it’s better to work backwards, say if they developed a game under a display with certain phosphor chromaticities and illuminant what the pixel color values end up is being a compensation of the display characteristics. So I will research on that front. For gamma this is currently how it behaves.

2 Likes

Updated to RC7.

Structural work mainly here. All color work was moved before transfer functions / vignette so it behaves more predictably.

The HUE vs HUE matrix was transposed to behave like Grade 2020.

And now NTSC-J uses D65 as reference white so all phosphors use a common illuminant and they are better correlated.

Also optimized and cleaned code a bit. The gamut compression was disabled since it clipped values instead of scaling them (tested with color bars). And ditched mixfix() for internal mix() as it was causing issues with blue (de)saturation.

I don’t think I will working more on this unless someone spots an issue, so I can move on to different projects.

6 Likes

Just updated Grade and Glass to official repo as PR, should be merged soon I guess.

Meanwhile this is my final comparison between RAW Sonic colors and final Grade adjusted colors. Settings are the same from the last time.

RAW

Grade 2023

6 Likes