Dogway's grading shader (slang)

Yes, updated yesterday in this post. Didn’t commit to libretro yet because I wanted you to test whether quantization wasn’t creating unwanted posterization. Also I’m researching on gamut compression and wanted to see if I can implement that before.

Grade should always be the first in the stack, it’s output shouldn’t conflict with anything down below because I don’t do any geometric distortion nor texture scaling.

2 Likes

So I likely need to stick this in the middle of my chain where I may have the passes in float format, will this work if I have Grade in Float format? Or will this cause problems?

3 Likes

I set the pass to float to reduce rounding errors but internally I do a quantization to 8-bit when using ‘composite’ signal type. Later float pipeline is restored for the digital grade controls.

3 Likes

Isn’t this accurate? Main difference is that it uses 9300k instead of 6500k, and blank/black are the same but I guess you know that already.

1 Like

Those are different topics, the paper simply defines the conversion from the RGB model to the YUV model while at the same time saturating the whole video range. I do the normalization separately and my model conversion is full->full. And here you find the range definitions for each region.

The temperatures also varies from spec to real world units as current TVs do. None were 6500K but more towards 7100K, and 9300K was only for master monitors in Japan, consumer units were more in the ~8000K, typically 8600K.

6 Likes

@Dogway seem to have done quite a bit of research into NTSC-J color temperatures. I was amazed.
Older CRTs don’t know the exact Kelvin, so for example Panasonic 4K LCD VIERA is 8830K as standard. Therefore, I think that 8600K on a cathode ray tube is almost close. kk32
6500K is often used as a cinema mode in Japan.

It’s wonderful that the white text doesn’t turn blue even though the sky blue of Super Mario Bros. (FC/NES) is light blue.

5 Likes

I decided that I should post solid source information, so I will post two articles.
The Panasonic 4K LCD VIERA and the SONY projector.
This is an article written by a Japan AV-related writer. Of course, it’s in Japanese, so if you’re interested, I recommend translating it into English.
Both references to color temperature are in the middle or bottom of the page.

https://av.watch.impress.co.jp/docs/topic/1318411.html

https://av.watch.impress.co.jp/docs/series/dg/351242.html

Sony projector is 8000K, but since the projector projects without directly emitting light, it is probably slightly darker, so I personally think it will be higher than 8000K.

1 Like

Thanks for the sources. It’s always interesting to see other POVs, but as far I could read (still can read some Japanese), it adheres too much on standards.

For instance the recommendation of D65 on a black box, where D60 would make more sense. Or the misconception to match white point with room luminary. There’s an optical effect called “Simultaneous Color Assimilation”, and it’s a non-linear surround visual leak into our perception of temperature. You can see a graph about it here. In other words, if you can’t control your surround illuminance and you’re stuck to 2700K or 3200K surround, the ideal white point for the display to be perceived white at standard 100nits is around 4100K. Given that eyes can’t white adapt to such temperature, it’s ok to set to D50, or simply turn off the lights and use D60. Translating that to a display with fixed D65 white point, surround illuminant should be around 6700K and 100nits.

2 Likes

yes. That’s right.
But I think it’s been proven that Japanese consumer CRT arent 9300K.
I remember that Professional CRT and PVM were 9300K.

Digression.
In Japan, it is standard to adjust the color temperature according to the lighting and ambient light.
I think this is wrong internationally. But I don’t know if this is unique to Japan.

1 Like

Thanks for the info, yes they aren’t despite what’s reported everywhere on the net. It was 9300K+27MPCD which translates to 8945K CCT for consumer units, that’s the standard while the practice is more spread on the 8000s. I read accounts of as low as 8200K and as high as 8800K.

I forgot what the typical lamp temperatures were in Japan, but in the cool mode it was quite cool, around D65 or maybe higher.

To expand a little on the Simultaneous Color effects:

  1. Simultaneous Color Assimilation: A color effect where color appearance is induced from the surround towards a central area when luminance contrast is high.
  2. Simultaneous Color Contrast: An color effect where color appearance of the complementary color is induced from the surround towards a central area when luminance contrast is low.
  • –Helson-Judd effect: Same as 2 but when additionally central luminance level is low.
  • –Kirschmann’s Third Law: Same as 2 but when additionally central luminance level is equivalent.
  • –Kirschmann’s Fourth Law: Same as 2 but when additionally surround saturation is high

It would be the case that Kirschmann’s Fourth Law counters a bit Simultaneous Color Assimilation, but to what degree I don’t know as the linked source already uses 3200K surround tests, I want to believe in typical bulb saturation values.

EDIT: Reading your second link I found it very interesting how they use as an alternative (custom) temperature tweaking by lowering the Blue channel output, that’s wrong of course because color temperature needs chromatic adaptation but good coincidence that that’s exactly what I chose to do in Grade with the CRT Beam Blue setting. Another thing is how they choose D60 as cinema mode, just as I posted above.

4 Likes

Since you seem to be proficient in the Japanese language or Japan affairs, I think you can decide based on your knowledge, judgment, or research.
I appreciate the results of your research. thank you! :star_struck:

2 Likes

Having first hand access to Japanese displays at this point is more useful than reading Japanese haha.

A good experiment to buy a bunch of old consumer CRTs and measure factory temperature and primaries. Not much info around, at least that I could find. People who buy CRTs for retro gaming in the west typically don’t have colorimeters or don’t care enough for color science.

2 Likes

If you want to know about Japan special circumstances, please feel free to send me a DM.
Or even the topic here, if other users will allow it.

I’m not an expert like you.
The whitest standard fluorescent lamp color is 7200K, but there are variations depending on the manufacturer, and I don’t have a professional measuring instrument.
So I may not be of any help.

2 Likes

Update. Added finally gamut compression to Grade so you never surpass your display’s gamut. Therefore this new setting (default off) applies only to Rec709/sRGB and Display-P3.

An observation, using 8600K surpasses the Rec709 gamut for blues in the Sonic title screen, when dialing back with gamut compression this saturation increase leads to some added brightness bringing a pleasant cyan-ish blue to out of gamut blues.

Please test, if everything is good I upload the last two updates to official repo.

6 Likes

Just a reminder that I uploaded zfast_crt_geo (and nogeo) to the glsl repo which aside from vignetting, screen curvature and round corners adds phosphor*temperature emulation for GPU deprived platforms like the Chromecast, Raspberry Pi-4B, Wii, Android tablets, etc. (~10GFlops and above)

The phosphor is hard-coded to mode 2 (NTSC-J) and temperature to 8504.

For details, this is the development thread, but you can also post here for any issues.

2 Likes

@Dogway so I’m finally able to play around with your new grade and right off the bat I notice in the Phosphor setting that setting it to 0 I’m still able to control the white point settings. In old grade setting that setting to 0 seemed to default the white point setting to 6500k and it wasn’table to be changed afterwards. I set that Phosphor setting to 0 in the old grade assuming that would give me a standard “neutral” look, that doesn’t seem to be the case with new grade especially since you defaulted the white point to 8500k now.

So my question is, should I leave the white point at default while turning the Phosphor setting to 0? I want a standard “warm” image while using the 0 Phosphor setting. I notice using those settings the image appears more cool instead of warm, bringing the white point down to about 7200k seems to give the warm look that old grade use to give at 6500k. Also just to be sure, what exactly IS the 0 Phosphor setting anyway in new grade? There isn’t any definition on it in your guide up above like the other setting numbers have, I also notice there isn’t one for the “-3” phosphor setting as well.

I may have a few more questions after this, grade is like the back bone to my presets and I’ve already been testing out new settings for my next preset pack update so I want to go into new grade with a clear understanding on some things.

1 Like

Yes, there wasn’t enough room in the param definition so I only noted the main ones. The full list is here.

So ‘-3’ is SMPTE-C or 170M standard although for old systems I still prefer to use the P22 primaries. ‘0’ is sRGB, and yes now you can change temperature in sRGB, why not. Setting temperature to 6504 as in the no-LUT versions should output the input colors and original temperature. Actually the no-LUT versions are no-op, so you can use those to start tweaking your way up with the different filters.

Don’t compare the new grade with the old one, I changed the temperature functions with different LMS and enhanced locus fit and a few more things, so current implementation is more accurate. Still you might want to make look things as the original grade (as I sometimes do too), so you need to play with the settings. I still haven’t updated my personal system presets to the new grade either.

2 Likes

Ah ok, I see it now.

Speaking on this, my next question has to do with P22. So after some research I see P22 is the US standard that consumer crts and arcade monitors used, my question has to do with Phosphor settings -1 (P22-80s) and 1 (P22-90s). What exactly does the “80s” and “90s” stand for? At first I thought it meant it represented a Phosphors used on a “1980’s” and “1990s” tv but something tells me that’s not what that means lol.

Also what would be a recommended combo to use to get a standard “warm” and “cool” image using srgb? For example I use Phosphor setting at 0 and white point at 7500k for a standard warm look (I think this white point looks the most balanced between 6500k and 8500k). For cool I was planning to use your default 8500k or maybe around 8800k.

I realized the more I played with the new settings that the difference is too great, so yes I do have to go forward into this without trying to compare this new grade to the old one. I use to use the digital settings on the old grade, now I use nothing but the new analog settings on new grade and man does it make things look sexy. Combined with that new base mask setting that guest.r just added into his shader I’ve been having fun in the last few days in the lab tweaking.

2 Likes

Well, there might be standards but back then it was more, not unlike today, each manufacturer improving the colorimetry by phosphor chemical improvements, first with Conrac Corp., then Sony, etc.

P22 used some known chemicals which were refined later with “tinted” variations to deliver more pure colors, these were around the 90s and so that’s where the naming came from. For some reason Japanese phosphors I think didn’t evolve much so P22-80s and P22-90s is for Europe and USA. There, as happens also today, displays weren’t never sold on D65 regardless of the impering standard. Real temperatures were more along ~7100K for USA and ~6900K for Europe.

Check here for the chemicals of each and my sources.

Also similar to how today music production is tested on AirPods, developers back then used consumer CRTs along their typical PC monitor to check the output, so for the Japanese case where temperature was so high that reflects on the colors of their games as well. You don’t need to be super accurate with the kelvins though, they could deviate ±500K, so 8800K is fine for NTSC-J, 7100K or 7500K is fine also for NTSC-U or PAL.

I can’t recommend you a warm or cool temperature because it depends on the embedded temperature of the content. D65 is considered neutral and D75 cool, but if the game was developed with a D75 CRT then D65 would look warm and D75 neutral.

As I found out though is that PVMs and BVMs mostly used EBU tech gamut phosphors (so PAL phosphors). So if you are shooting for that look go ahead. On temperature I’m not sure if PVMs were calibrated to D65.

Yes, also excited to test that “black” phosphors. It throws another myth out of the window with CRTs having pitch blacks.

4 Likes

That’s interesting information right there. I like probably many others always assumed crt accurate temperature was 6500k. I probably won’t aim for 100 percent accuracy with the temperatures but something close enough will do. I wish I could find a temperature list like the one from this post here but for the US and Pal temperatures, that would give a basic outline on crt temperatures to kind of go off.

Yeah I’ve still be experimenting, new grade really is a different kind of beast, old tricks I used in previous presets don’t translate the same on new grade so I’m taking my time just playing around finding a new but similar look. I see that when shooting for a “cool” image turning up the crt beam blue setting helps a bit. I’m still not getting the whites to look as “blueish” as I would like, maybe I have to dial down the other two crt beams or something, I’ll experiment again later.

Yeah I’m just going for the consumer crt look, not really a fan of the pvm/bvm look even though my aperture grille presets look close to a pvm look.

3 Likes