Dogway's grading shader (slang)

[Chthon]

I’d like to revisit the topic of color correction circuits for a bit.

To refresh the memory of anyone who’s forgotten (or didn’t read the earlier posts), CRT televisions with out-of-spec phosphors (basically all of them), generally included a “color correction circuit” to compensate for the discrepancy. So, if we really want to get the colors correct, we need to first emulate the behavior of the CCC, then do a gamut conversion from “NTSCJ white point and P22 phosphors” to sRGB/BT709 used for modern PC monitors. The behavior of these circuits is not well documented, and varied across time periods and manufacturers. 1970s era CCCs were abominations; 2000s era CCCs looked vaguely like a proper gamut conversion with some effort to deal with the clipping problem; what came in between was in between. Recap over.

I’m happy to say that I’ve found a couple patents that really do tell you everything you need to know to reproduce the circuit. In both cases, I was able to work out the math in a spreadsheet.

This patent was filed by Chunghwa Picture Tubes, Ltd. in 2003. While specifically intended for plasma television sets, its approach is applicable to the “phosphors don’t match the spec” problem in general. Adjusted for the “NTSCJ-spec to NTSCJ-white with P22 phosphors” case, it reduces to: multiply this matrix by the linear RGB input, clipping output if needed:

1	-0.042983396021096	-0.006989606552852
-0.374953732450873	1	-0.050527229116143
-0.107808598004971	0.069276873840935	1

A major shortcoming of this approach is that it moves white noticeably towards pink. (I’ve got an idea for a kludge to fix that, at the expense of not faithfully reproducing the circuit.) If you worry about not infringing patents even if no one likely cares, this patent expires in June 2025.

This patent was filed by Sanyo Electric Co Ltd in 2004. Like the other one, this one was specifically intended for very early OLED televisions, but involves another general solution for the “phosphors don’t match the spec” problem. It’s a more advanced take on the same principles as the other one, trying to correct at 7 points instead of 3. This one also suffers from moving white (though towards a more believable light teal). This patent was deemed abandoned due to Sanyo’s failure to respond to the patent office, probably because their facilities got flattened by an earthquake later in 2004 and the company basically collapsed after that.

Moving into the realm of speculation:

There was probably a generation of TVs that just multiplied by the correct gamut conversion matrix and didn’t even try to solve the clipping problem.

I’ve conceived a “generic shitty CCC simulation” that does a weighted interpolation between the correct gamut conversion and no correction based on saturation. This is based on the notions that: (a) The simple matrix approach is dead easy and gives correct answers for non-saturated colors where clipping isn’t a problem. So I speculate manufacturers used it. (b) Manufacturers probably didn’t want to give up the best saturated colors their phosphors could produce, nor accept local saturation inversions due to compression. The only way not to do at least one of those things was to have no correction all the way out at the primaries. © It looks “right” on one game where a proper gamut conversion looks notably wrong.

My next step is to work on C++ implementations of these so I can generate some LUTs from them.

What I still really, really hope for is to find documentation on a CCC that was definitely used in late 90s Japanese consumer CRT television sets.

[Azurfel]

@Dogway

In the process of porting the SMS Blue Lift and MD Palette functions to crt-guest-advanced-ntsc for more transparent use with Megatron, i noticed that your SMS Blue Lift function is missing the non-linear part of the behavior described in Notes & Measures: Nonlinear Blue on Sega Master System 1 & Other Findings.

That is, SMS Blue Lift’s blue push is effecting both the first and second blue and grey bars in SMSTestSuite’s Color bars test, while on real hardware the blue push has tapered off. Meaning that SMS Blue Lift is altering 32/64 of the Master System’s colors, rather than the correct 16/64 as described in the document.

[Azurfel]

@Dogway

I added a SMS Nonlinear Blue Fix to crt-guest-advanced-ntsc that fully implements the behavior described in Notes & Measures: Nonlinear Blue on Sega Master System 1 & Other Findings. I will attempt to port it to Grade for you at some point to save you the trouble if you don’t beat me to it.

Additionally, should the MD Palette be applied before the Sega Luma Fix?

[Cyber]

You move pretty fast @Azurfel. I don’t doubt your confidence and expertise but has @Dogway responded to what you’ve noticed with respect to the SMS Blue Lift fix?

Are we 100% sure his implementation contains an error or misinterpretation or is some of what he did intentional based on a particular context he had in mind?

[Chthon]

@Dogway,

I’ve been spending more time looking at patents for color correction circuits. I’m currently looking at the American adjunct to a 1992 Japanese patent filing by Toshiba (U.S. patent number 5,301,017), and it claims the “typical” CRT phosphor chromaticities are:

• red x=0.657, y=0.338
• green x=0.297, y=0.609
• blue x=0.148, y=0.054

These are waaaay off from P22. Any idea what’s going on here? Does that line up with any known phosphors in your reference materials?

Aside from that, what’s interesting about this patent is that it treats the naive gamut conversion “linearize, multiply by a RGB->XYZ->RGB matrix, gamma encode” as prior art. (Toshiba’s claimed invention is having a switch to select between multiple gamut conversion matrices depending on what spec your input signal is supposed to follow.) To the extent you can reliably read tone in a patent filing, it sounds like this method was “old hat” by 1992.

Another head scratcher is what they did about clipping inherent in this naive method. The patent doesn’t mention any solution. Not in the claimed invention, and not in the prior art. I’m beginning to think that maybe they didn’t address clipping because it wasn’t a problem for them, possibly due to some combination of (a) those phosphors listed were closer to spec than P22, (b) some out-of-bounds values would still be within the TV-range headroom/footroom, and thus displayable without clipping, © since the whole thing was analog, some even further out-of-bounds values might be displayable without clipping, and (d) live action content usually involves few colors saturated enough to clip. Any insights about that?

What’s really alluring about this speculation is that, if correct, it would mean that a one-step spec-to-sRGB gamut conversion would be equally correct as the painful two-step “simulate the color correction circuit, then do gamut conversion from phosphors to sRGB” approach I’ve been chasing after. (Well, at least with respect to CRT televisions employing this type of circuit.)

[DariusG]

That’s closer to a PC sRGB monitor, it could be some patent for early CRT PC monitors with minor differences to sRGB.

p22 should have some differences on manufacturers too, these are PC monitors.

[Chthon]

It’s from 1992, and it says these values are “the phosphor in a cathode ray tube(CRT) used in the typical television receiver.” Also, the filer is Toshiba, so it’s difficult to dismiss this as the filer not knowing what they’re talking about. Perhaps they’re not P22 at all…

And I think I figured it out as I was writing this. Those numbers look a hell of a lot like EBU. And a quick internet search finds two (unsourced) claims that Japanese high-end televisions used EBU phosphors. This and that.

[DariusG]

I remember i read somewhere that Japanese TVs used the EBU/PAL chromaticities with 9300k (probably a bit less). EBU is close to sRGB too if i remember well without searching. I had watched a video, some guy was measuring a PVM chromaticities with the proper equipment and they were almost identical to sRGB.

[PlainOldPants]

This is my first post on the Libretro Forums, so I apologize in advance if any of this is against this site’s norms. I’ve been teaching myself about this subject very slowly over the past several months, thanks to the NES palette situation. In fact, I’m new to retro gaming entirely, having only gotten my first real NES and Genesis in the past 6 months or so.

I believe I’ve gotten close to the information you all are missing. I have gotten much closer to the colors of my CRT TV by looking up the data sheets for real NTSC video decoders and implementing them as a shader. These data sheets have a section showing the relative amplitudes and phases of R-Y and G-Y compared to B-Y, and in two cases out of four, R-Y is about 112 degrees off from B-Y at 0.83 times the amplitude, unlike the typical YIQ matrix where it’s off by 90 degrees with 0.56 times blue’s amplitude. The specific data sheets I’ve looked at are the CXA2025AS (which already has an NES palette based on it), the TA8867AN, and the TA8867BN. As much as I’d like to assume that they work by just (in effect) adding three sine waves together, my gut feeling tells me there’s more to this. There’s also an optional “dynamic color” feature in the CXA2025AS that is mentioned but never documented, and I have no idea what this does.

It looks obvious to me that these drastically different settings are meant to increase the area of the YIQ color space that contains skin tones, but I can’t tell whether or not these also are trying to compensate for the FCC color primaries. Another thing to note is how the CXA’s JP axis still has the skin tone area widened, but it’s not widened as much as the US axis, so I guess that’s because Japanese TVs had a cleaner video signal. Obviously, PAL didn’t need this correction because its alternating phase already solves that problem, but it does have the color green slightly more saturated for some reason.

I’ve also looked up several Composite video ENcoders used in the Genesis–namely, the CXA1145, CXA1645, MB3514, and KA2195D–and seen that despite having different brands, they all share the same conversion matrix with R-Y off by 104 degrees at about 1.41 times B-Y’s amplitude, etc. Despite red being at 104 degrees instead of 90, it looks indistinguishable from the standard matrix with red at 90 degrees.

I know Libretro doesn’t welcome sudden gigantic poll requests from strangers, but please hear me out. For my own personal use, I have been writing my own simplified grade shader that implements the above color decoders simply as 3 sine waves and greatly reduces the amount of options that were in Dogway’s Grade. I couldn’t find correct saturations anywhere for these decoders, so I just made up my own saturations. I’ve uploaded it to MediaFire as a zip with several presets that combine it with ntsc-md-rainbows, gtu-famicom, and crt-guest-advanced, in case any lurkers want to try it easily: https://www.mediafire.com/file/3mmiifnyqt1a46t/crt-shaderpacks-for-retroarch-Jun-17-2024-ver2.zip/file

Screenshots: https://imgur.com/a/jFz3p7j

From Chthon

Along the way, in 1987, the Americans said “screw it,” and changed their broadcast spec to match the phosphors commonly in use, obviating the need for these circuits in American TVs.

Unless I misunderstood you, I doubt this is true, based on what I’ve seen from experimenting with these colors so far. I have no idea what I’m doing, but I just started searching and happened to see that the YIQ page on Wikipedia has a link to this source [ § 73.682 TV transmission standards (PDF). FCC. 2013. p. 210. ] which says that the USA and Japan both kept the FCC primaries, and there’s also a link to this source [ Rec. ITU-R BT.470-6 - CONVENTIONAL TELEVISION SYSTEMS (PDF). ITU-R. 1998. p. 9. ] describing USA TV transmission standards which still uses the old FCC primaries despite the most recent date mentioned in the document being “June 30, 1994”. It looks like the SMPTE standard was never accepted, except for maybe their new YIQ matrix. (I had to put the links in brackets because new users are only allowed to have 2 links.)

[PlainOldPants]

Apologies for double-posting instead of editing my first post. It looks like I have to wait for my post to be approved by a moderator before I can do that. I retract saying that “I’ve gotten close to the information that you all are missing” (Ugh, why!!!), about increasing the area with skin tones (which I’m not so sure about, but I’ve heard of), and about the standard having red and blue 90 degrees apart, which is only an approximation. I also think that the different JP axis might have to do with Japan’s different white reference.

I’ve reread the posts above, and I’m thinking now that these decoders’ different axes might just be the same thing as what Chthon found in those patents for gamut conversion circuits, just represented in a different way. The fact that the conversion in Toshiba and Sony’s chips is being done on R-Y and B-Y (and possibly G-Y too) without looking at Y is what confused me. The gamut conversion can be done without knowing Y (I think…).

I believe it’s likely that each one of these video decoders is meant to be used with different phosphors and, possibly, a different white reference. Although there is not any information in these data sheets describing what kind of TV, phosphors, and/or white reference they are meant for, we can use either mathematics or trial-and-error to infer them. Chthon has helped greatly by finally discovering those patents, and I believe that these real chips’ data sheets are the next place to look after that, in order to see what most brands were implementing in practice, not just in theory. I have already implemented some conversions quite loosely using this surface-level data, but I haven’t made any reverse-engineering efforts on these chips.

Last note about my previous post: My shader presets above are “my own” in the sense of “my own personal use”, not in the sense that they are my original work. The only original work I did was implementing the chips’ conversions based on their data sheets. There is a carefully written readme file and much more information inside that zipped folder which gives proper credits, allowing users to find the original free and open source work and its authors. I was completely unaware that a user named sonkun was already making superior presets, so I will be checking those out soon. You all are amazing on this site.

[Chthon]

@ PlainOldPants

Thanks for all that!

Perhaps I can help clear up a few things.

Yes, the “skin tones are off” problem is a symptom of the larger “the phosphors don’t match the spec” problem. Everything is off. It’s just that humans are evolved to pay more attention to human skin tones, so we notice the error there more. Some solutions are aimed at the full problem, and some are aimed just at skin tones.

The television manufacturing industry tried several different solutions to the “the phosphors don’t match the spec”/“skin tones are off” problem over the years. Also, it’s not the case that there was a clean progression from worse methods to better ones evenly across the field. As best I can tell, some manufacturers weren’t paying attention to what their rivals were doing, and some manufacturers may have been barred from using the current state of the art because someone else patented it. I can offer a (probably incomplete) taxonomy of methods used:

1. One of the earliest approaches was to bodge the Y’IQ demodulation axes to fix skin tones, at the expense of distorting everything else. Later iterations on this method tried to limit the area of affected colors. An example of this method is US Patent 4,695,875 (filed 1986) – colors very near the I axis are pinched onto the I axis, and colors sorta near the I axis are pinched towards the I axis.

2. Another early approach is the one you describe: Try to correct for the mismatched phosphors by demodulating Y’IQ using non-orthogonal axes and some color-specific gains. The seminal text on this method is “An Analysis of the Necessary Decoder Corrections for Color Receiver Operation with Non-Standard Receiver Primaries” published in 1966 (link). These corrections can be expressed in terms of an angle and a gain for each of red, green, and blue. I’ve not yet worked through the math to go from that nomenclature to the Y’IQ-to-R’G’B’ decoder matrix nomenclature and back again. Perhaps that’s something you’re closer to understanding than I am. Here’s a discussion by someone who does understand that conversion. This approach has three limitations: First, it doesn’t correct luminosity, only chromaticity. Second, because Y’IQ is gamma encoded, the output won’t be exactly correct everywhere. In fact, the whole derivation works by picking two IQ points where you want chromaticity to be exactly right (usually one is a flesh tone), and everything else will end up sorta right. Third, there’s no discussion about how out-of-bounds outputs are handled. Is C clipped? Are I and Q clipped? Are R’, G’, and B’ clipped? Do you just try to drive the electron gun at over 100% max voltage? I don’t know. (Aside: The Japanese demodulators were different because their white point was different, and so the luminosity weights used to convert R’G’B’ <–> Y’(B’-Y’)(R’-Y’) are different, and so the chromaticity error at any given IQ point was different.) A later iteration on this approach that tries to reduce the luminosity error can be found in US Patent 4,167,750 (filed 1977) – different gain values are applied above and below certain C thresholds. (So you now have an angle, two gain values, and a threshold value for each of red, green, and blue.)

3. If you’re willing to pay for a fancier IC, you can “Do It Right” ™. The above methods are trying to solve the problem on the cheap by futzing with Y’IQ. But if you’re willing to pay for an IC that can convert R’G’B’ to linear RGB and back, then you can do a proper gamut conversion. If the white points are the same, converting RGB to XYZ using the spec primaries, then XYZ to RGB using the phosphor primaries gives an exactly correct result. And it can be rolled into a single matrix multiplication. (Math explained in RP 177.) If you need to change white points, you need to do a chromatic adaptation, but that too can be rolled into the single matrix multiplication. (The “Bradford method” of chromatic adaptation was published in 1985. Prior to that, it would have been XYZ scaling or Von Kries (which dates to 1912, I believe). See Bruce Lindbroom’s site for how to do these chromatic adaptations. The current state of the art chromatic adaptation is CAT16, from 2016.) This method is described as being “prior art” in US Patent 5,301,017 (filed 1993). The “invention” claimed in this patent is having a switch to select between several different matrices depending on the format of the input. The only problem with this approach (aside from the cost of the IC), is that there is the possibility for outputs beyond the bounds of the device’s phosphors’ gamut. This patent handwaves that away by claiming it’s intended for a display with a wide gamut that encompasses all the gamuts it’s supposed to emulate, but then it also says that the same method produces “good color reproduction” on a plain old NTSC receiver, without any discussion of how out-of-bounds outputs are handled. I speculate that maybe they really didn’t do anything about them and just trusted in the headroom/footroom built into “TV range” and the analog nature of things being tolerant enough with out-of-bounds values that things just worked out somehow. The intriguing thing about this possibility is that, if it’s truly the case that out-of-bounds values were just tolerated, then the phosphor gamut drops out of the math and you can simulate televisions with this kind of color correction circuit by just converting straight from the spec to srgb (or rec2020).

4. LUT LUT LUT! I’m honestly not quite sure what’s going on in US patent 5,867,286 (filed 1996). The general procedure it describes is to cut up the (0-1 normalized) (B’-Y’)(R’-Y’) plane into a 32x32 (or whatever) grid, convert R’G’B’ to linear RGB, then multiply by a matrix selected according to which cell the color occupies on the grid, then convert back to R’G’B’. The patent is not clear about how the content of the matrices are arrived at. The vague description sounds the same in purpose and construction as the “RGB to XYZ using spec primaries, then XYZ to RGB using phosphor primaries” matrix described above. But then it’s not clear what the point is to having more than one matrix. Perhaps what’s going on here is that having a potentially different matrix for each cell allows them to implement some form of gamut compression in the outermost rows to avoid clipping. Except the patent never says anything about gamut compression. (Perhaps they wanted to bamboozle a patent without actually disclosing their secret sauce?) Ultimately I’m really not sure what’s going on here. I’d imagine that if we go forward some years we’ll hit the point where an IC with a full lookup table for every possible RGB input became practical. As I understand it, that’s how modern display devices handle gamut conversion.

5. Geometric headaches with no clear point. US Patent Application Publication 2006/01393.68 (filed 2004) and US Patent 7,193,587 describe what I’ll call “geometric” approaches to gamut conversion for early OLED and plasma TVs, respectively. They describe general-purpose solutions to the “the phosphors don’t match the spec” problem, but they look so bad when applied to the “NTSC-J spec vs typical NTSC-J phosphors” case that I don’t they were ever used for CRT televisions. Both of them distort white and suffer from a lot of saturation and brightness loss. 7,193,587 can also have outputs go out of bounds below zero. 2006/01393.68 appears to always stay in bounds, but the saturation and brightness loss is terrible. My best guess is that these approaches were pursued because the “phosphor” gamuts for these devices were so narrow that something had to be done to prevent clipping, and this was what they came up with. I’m kinda sad about this because I remember being really excited to find patents that actually described the invention with enough detail I could reproduce them in software (unlike 5,867,286), but their output looks like crap for our use case.

A few other interesting things:

• If a television set used a “futz with Y’IQ” method of color correction (types 1 and 2 in my taxonomy above), and the game console provided RGB input via a SCART or JP21 cable, then color correction would have been entirely bypassed.
• SMPTE-C was an odd duck. It specified new primaries, and proper encoding for broadcasts, but it allowed Y’IQ-to-R’G’B’ decoding using the old NTSC luma weights. This meant colors were decoded wrong across the board, but the error was not particularly noticeable. (The primary motivation for this was to be sorta-kinda backwards compatible with the millions of TV sets people already owned.) A few years later, the SMPTE 240M standard came out, which was “SMPTE-C with the correct luma weights.” But then Rec709 came out not long after and everyone forgot about SMPTE 240M. A TV set with SMPTE-C phosphors did not need any color correction to correctly display a U.S. broadcast from 1987 or later. (However, a TV set with P22 phosphors still needed color correction, but different than what it needed before.)

[Cyber]

@PlainOldPants

I was completely unaware that a user named sonkun was already making superior presets, so I will be checking those out soon.

How could you be have missed Sonkun’s Shader presets for so long? They are all over social media.

CyberLab Death To Pixels Shader Preset Packs

[CyberLab Mega Bezel Death To Pixels Preset Pack Title] [Screenshot_20220408-113722_Chrome] [Screenshot_20220408-114931_Chrome] [Screenshot_20220408-114949_Chrome] [Screenshot_20220408-115001_Chrome] [20220825_100739] Disclaimer: I present these to you without making any claims, representations, warranties nor acceptance of any liabilities including any arising out of the use of this software whatsoever. They are just my presets that I use and I’m sharing with you in the hope that y…

RetroGames4K's Mega Bezel screen pics, gameplays & custom settings

Welcome to RetroGames4K’s thread. Here I will post every screenshot, gameplays and custom settings from Mega Bezel and other Retroarch’s shaders. My main objective is to post screenshots from my custom settings and hope you enjoy it. Here are my custom settings. This setting is based on my 14" CRT Sanyo CE14AT08-C These images are real photos taken from my Sanyo CRT for comparison purposes: [image] [image] …

Hyllian shaders and presets

This is a collection of shaders and presets fine tuned by me for a plug 'n play experience. Latest releases are now available in this first post: New release: hspack-24-03-31-r1 Added nnedi3 and super-xbr presets in downsampling folder for 2x, 3x and 4x internal scalings; HiRes detection optimized with new algorithm reducing from 12 to just 2 samplings; Some curvature bugs fixed; LUTs changed again. Now a grade-composite with deeper blacks is available; slotmasks (from crt-hyllian) tweaked f…

[sonkun]

@PlainOldPants

I was completely unaware that a user named sonkun was already making superior presets, so I will be checking those out soon.

Hello and welcome to the forum. As Cyber stated and posted yes I have a pre configured shader pack, I wouldn’t say it’s “superior” though lol. I’m constantly working on it even now as of this posting. You can check out the main pack on my main page if you want but since I started working on a new look I pretty much deem that pack outdated and pretty much discarded. I’ll end up replacing it once I put together a new pack which will be soon.

If you want you can check out what I’m currently working on here.

You will also need guest.r’s latest shader release for what I’m working on which you can grab here.

To make things even simpler to install his shaders manually I put together a easy 3 step installation guide which you can use here:

[PlainOldPants]

(Cyber)

How could you be have missed Sonkun’s Shader presets for so long? They are all over social media.

That’s a really good question. I think I just haven’t been paying attention to social media lately.

Before posting, when I read through the more recent posts of this thread, I did see a lot of good information being posted, but it looked like this strategy of using a botched YIQ conversion was hardly discussed at all if not completely missing, despite it having been so common in consumer hardware (even into the 90s) and having caused so many consumer TVs to look different from each other. For that reason, I replied on here with my basic implementation of it, hoping it would help to improve Grade. (You might recall that the version of Grade currently in RetroArch has settings to change U and V’s offsets and responses, but that’s not the same thing as this.) I’m slightly sorry that by posting that, I made you (Chthon) spend your time explaining everything to me in detail like that, but then again, that’s light-years better than 10-15 years ago, when the average mod or program author would rant about “noobs” and answer every other question with a link to a disgusting shock image.

(Chthon)

In fact, the whole derivation works by picking two IQ points where you want chromaticity to be exactly right (usually one is a flesh tone), and everything else will end up sorta right.

I knew (or nearly knew) the rest of what you wrote in this paragraph, but this bit about picking two IQ points is completely new to me. In my spare time, I might code this derivation myself, and I might even see if I can reverse it on some decoders. As for what things get clamped, based on what I’ve seen, I think it’s safe to assume either U and V (if present) or R-Y and B-Y are clamped in most setups. The unknown is how much it’s clamped by; that’s not in the data sheets. Your whole post was very helpful and cleared up many things that I wanted to know.

(sonkun)

I wouldn’t say it’s “superior” though

Lol, that’s not what I meant. In the context of that paragraph, I just meant it’s superior to my personal presets that I’d just uploaded to MediaFire. The reason I had uploaded this was specifically for my implementations of the CXA2025AS and TA8867AN/BN color corrections. The presets I made with a rewritten, simple grading shader, a typical CRT Advanced setup, and the choice of USA NTSC-M, NTSC-J, and PAL were more of an afterthought for any random people (lurkers) who wanted to quickly see how the result looks. Now that I know about your presets, it would’ve been better if I just attached the color correction and directed people to your presets. I’m surely going to try those presets now that you’ve sent me instructions. Thank you all for your kind replies.

[Chthon]

@PlainOldPants

It took awhile, but I did eventually work through the last few pages of the Parker paper well enough to understand converting from “axis & gain” notation to a R’G’B’ to R’G’B’ correction matrix, or back the other way. Hopefully this will prove useful for you, or at least for someone:

In both cases we need a few constants.
The NTSC white balance factors (often truncated to 0.299, 0.586, 0.114):

• Wr = 0.298911657927057
• Wg = 0.586610718748869
• Wb = 0.114477623324074

The Y’UV scaling factors:

• Udownscale = 0.492111
• Vdownscale = 0.877283

The inverse Y’UV scaling factors:

• Uupscale = 1/Udownscale = 2.03206187221989
• Vupscale = 1/Vdownscale = 1.13988302520395

MatrixB (2x2):
(1-Wr)/Vupscale, -Wr/Uupscale
-Wg/Vupscale, -Wg/Uupscale

We also need MatrixBInverse, the inverse of MatrixB.

To Convert R’G’B’ to R’G’B’ Correction Matrix to “Axis & Gain”:

Assume we have a 3x3 R’G’B’ to R’G’B’ correction matrix we want to embody as a demodulator:
Krr, Krg, Krb
Kgr, Kgg, Kgb
Kbr, Kbg, Kbb

Define:

MatrixCr (1x2):
Krr – Wr
Krg – Wg

MatrixCg (1x2):
Kgr – Wr
Kgg – Wg

Matrix Cb (1x2):
Kbr – Wr
KBg – Wg

Compute:

• [[yr], [xr]] = MatrixBInverse * MatrixCr
• [[yg], [xg]] = MatrixBInverse * MatrixCg
• [[yb], [xb]] = MatrixBInverse * MatrixCb

Convert the coordinates xr, yr to polar form to get the angle and gain for red.
red_gain = sqrt(xr^2 + yr^2)
red_angle = atan(yr/xr)
You may need to correct the quadrant that atan() spits out. (Blue should be near 0 degrees, red near 90, and green near 235.) Also remember radians vs degrees.
Repeat for green and blue, and that’s it.

To Convert “Axis & Gain” to R’G’B’ to R’G’B’ Correction Matrix:

First, figure out if the gain values have been normalized. If the gain for blue is already around 2, you’re good to go. If the gain for blue is 1.0 and the angle for blue is 0 degrees, then multiply all the gains by Uupscale. If the gain for blue is 1.0 and the angle for blue is not 0 degrees, then you haven’t been given enough information and the data sheet author should be beaten.

Convert from polar to Cartesian coordinates:
xr = red_gain * cos(red_angle); yr = red_gain * sin(red_angle)
Repeat for green and blue.

Compute:

• MatrixCr = MatrixB * [[yr], [xr]]
• MatrixCg = MatrixB * [[yg], [xg]]
• MatrixCb = MatrixB * [[yb], [xb]]

Add [[Wr],[Wg]] to each of MatrixCr,g,b to recover Krr and Krg, Kgr and Kgg, and Kbr and Kbg, respectively. (See definitions of MatrixCr,g,b above.)

Finally recover:

• Krb = 1 - (Krr + Krg)
• Kgb = 1 - (Kgr + Kgg)
• Kbb = 1 - (Kbr + Kbg)

Which gets you to a 3x3 R’G’B’ to R’G’B’ correction matrix:
Krr, Krg, Krb
Kgr, Kgg, Kgb
Kbr, Kbg, Kbb

This is really neat for a couple reasons. First, converting the demodulator’s behavior into a R’G’B’ to R’G’B’ matrix means that we can have a really fast implementation that’s just multiplying by a hardcoded matrix. Second, the fact that our only operation is an invertible matrix means that we should be able to find gamut boundaries for the gamut of “all the colors (via a given trio of phosphors) that this demodulator can output given every valid input,” and we can use that for a gamut compression algorithm rather than the gamut of “all the colors this trio of phosphors can produce.” I’m really excited to try that out.

We’re still missing a few puzzle pieces though.

One open question is “What about Japan?” Japan used a different white point, so doing things “right” would have meant building a different series of modulators/demodulators with different constants for white balance and UV scaling. But, did they? It would have been cheaper and easier to use the same hardware across the board, and you can use any wrong constants you like and they’ll drop out so long you use them in both directions. Several things point towards the provisional conclusion that Japan just used the same modulators/demodulators domestically as US units:

• I have not, as yet, seen any Japan-only modulators/demodulators using the 9300K+27MPCD balance.
• The fact that chips like CXA2025AS exist, with US and Japan modes that seemingly differ only by axis and gain.
• Parker also does some examples for the Japanese white point, using the same white balance and UV scaling constants.
• US Patent 4,167,750 discusses axis and gain for the Japanese white point, without ever mentioning the use of different white balance and UV scaling constants.

Another big open question is “Which set of phosphors was paired with which demodulator chip?” I’ve made a small amount of headway here. It’s not hard to find Sony service manuals that tell you that a particular demodulator chip and tube were used in the same unit. The problem is finding the phosphor chromaticities for a specific model of trinitron tube. Thus far I’ve had no luck doing this.

I’ve a few thoughts on other topics, but no time to write them down now.

[Superdonkeyjack]

Man, I don’t think I can get that all downpat

[Chthon]

@PlainOldPants

Returning to the question of “Which set of phosphors was paired with which demodulator chip?” I might have one plausible answer. Please step this way down the rabbit hole:

In 1997, some visual science researchers decided to measure the properties of their GDM-17SE1 Trinitron monitor “for science!” (Link) They measured chromaticities of red: x=0.6233, y=0.3390; green: x=0.2838, y=0.5895; blue: x=0.1519, y=0.0669.

Color.org says that, according to “reference data… provided by the manufacturers,” the chromaticities of a Trinitron computer monitor are:
Sony Trinitron (all ± 0.03):
Red 0.621 0.340
Green 0.281 0.606
Blue 0.152 0.067
And, seeing as that’s really close to the researchers’ measurements on their one particular monitor, I think we can safely say these are the official values for the phosphors used in the GDM-17SE1’s tube.

Now, I’m going to make an assumption. I’m going to assume that Sony used the same phosphors that it used in the GDM-17SE1 Trinitron computer monitor in their Trinitron television sets that launched the same year. I’m not super happy about this assumption, but I don’t have anything else to go on. Musings about its validity are welcome. (Maybe…)

The GDM-17SE1 launched in 1994. According to CRT Database, the KV-20M10 and the KV-20S11 both also launched in 1994. Both are listed as using a CXA1465AS demodulator chip.

Looking at the datasheet for the CXA1465AS, it turns out to be the datasheet for both the CXA1464AS and CXA1465AS, which are the Japanese and U.S. models, respectively, and differ only in the demodulation axes. The Japanese axes are red 98 degrees, 0.78 gain; green 243 degrees, 0.31 gain; blue 0 degrees, 1.0 gain. And the American axes are red 114 degrees, 0.78 gain; green 255 degrees, 0.31 gain; blue 0 degrees, 1.0 gain.

One slightly puzzling thing about this datasheet is that only one set of gain values are listed, which look like the Japanese gain values on later “selectable axis” chips. Did the datasheet author just forget to write down the U.S. gain values, or did the chip designers just use the Japanese gain values for the U.S. axes? The latter is entirely plausible given Sony’s general attitude in the 90’s that everything outside of Japan was a secondary market full of philistines. In fact, I think the latter is the more likely answer. The KV-20V60 launched 2 years later in 1996 with a CXA1870S demodulator chip. The CXA1870S is a “selectable axis” chip (maybe the first "selectable axis chip?) and its data sheet clearly lists two sets of gain values, but the U.S. values are identical to the Japanese ones. So, I think the correct answer here is that Sony was just reusing Japanese gain values on U.S. demodulators chip models up through the mid 90s.

So, bottom line time: Subject to the validity of the big assumption above, I’m going to tentatively say that you could reproduce 1994-launch Trinitron television sets by pairing phosphor chromaticities red x=0.621, y=0.340; green x=0.281, y=0.606; blue x=0.152, y=0.067 with demodulation axes/gains of Japan: red 98 degrees, 0.78 gain; green 243 degrees, 0.31 gain; blue 0 degrees, 1.0 gain and U.S.: red 114 degrees, 0.78 gain; green 255 degrees, 0.31 gain; blue 0 degrees, 1.0 gain.

Again, I find I have more things to write about (I really want to talk about the CAX1145 modulator chip!), but not enough to do so.

[Chthon]

I guess my last post is now too old to edit? Anywho…

Maybe also worth noting, the same tube used in the 1994 models KV-20M10 and KV-20S11 mentioned in my last post showed up again in 1996 in the KV-20V60 paired with a CXA1870S demodulator chip, then again in 1997 in the KV-20M40 paired with the CXA2061S chip, and then again in 1999 in the KV-20M42 again paired with the CXA2061S. So, since it seems unlikely that Sony would change the phosphors without changing the part number on the tube, we can assume these probably had the same phosphors, and also assume that the CXA1870S and CXA2061S represent revised opinions about optimal color correction for those phosphors.

We could probably proceed further down this path by identifying other tubes that were paired with these 3 demodulators, and assume that they differed by form factor, not phosphors. And maybe eventually work our way towards finding a pool of models linked by overlapping tube or demodulators, and also the edges of that pool where a new model shares neither tube nor demodulator with anything that came before.

[PlainOldPants]

Here is my implementation of the GDM-17SE1’s colors, using yet another alternative grade shader implementation by me. My next goal is to change Grade’s analog controls to feature this optional fast R’G’B’ to R’G’B’ conversion, which I’ve procrastinated on because I’m so bad at understanding Grade’s code; although, I now know exactly how to edit Grade to do this. https://www.mediafire.com/file/lf9mpb25jzp9llm/trincolor-jul-6.tar.gz/file

My alternative shader also includes some alternative methods for gamut mapping / gamut compression, but they’re slower because they (for now) require multiplying together the entire rgbToXyz->wp_adjust->xyzToRgb sequence into a single matrix. To have those methods implemented in Grade, something would have to be done about the SAT + HUE vs SAT changes that are in IPT space (meaning it can’t become a matrix), but I haven’t taught myself about this yet. Besides, there’s likely a better way to implement this without needing to multiply the whole matrix together. I used the outdated Bradford method because I didn’t realize that Grade’s wp_adjust could be represented as a matrix, but this can be fixed very easily.

Notice how the blue area is more purple in Japan and more cyan in the US, while the yellow/brown area is more green in Japan and more orange/red in the US. (Top image is JP, bottom image is US)

[Chthon]

@Dogway

Would you mind explaining what’s going on in the last stanza of your EOTF function (“Black lift compensation”)?

The top half of the function looks like the function from Appendix1 to Rec. ITU-R BT.1886, modified to add brightness and contrast knobs.

But that last stanza is totally unclear to me. I don’t understand where these magic numbers are coming from. And, more fundamentally, I don’t understand conceptually what this code is supposed to do.

Judging from the comment “Undo Lift,” I initially assumed that you were chopping off the elevated black and scaling to get back to a 0-1 range, while keeping the relative effect of having fed everything through further up the curve. However, when I take a closer look, it doesn’t seem to be doing that. If you wanted to that, then bc would be sl*(Lb^a2). But it’s not. Instead bc is derived from magic numbers I don’t understand.

The “Restore Gamma from ‘Undo Lift’” line I understand even less.

Please kindly explain what this stanza is doing, and where the magic numbers came from?