PlainOldPants's Shader Presets

[kev4cards]

@PlainOldPants

The NES’s color palette can be decoded from its video output, but you can’t convert an NES palette back into the original video signal. Decoding the NES signal into a palette is not a reversible process for several reasons (for instance, because it involves clamping RGB values between 0 and 255), and even if it were reversible, each palette has decoded and color-corrected the NES signal using different methods from each other, so our shaders would have to be told exactly which palette is in use to be able to invert its process back to the original video signal, which would then limit the shader to only support a selection of palettes. Some palettes are just done by guesswork or by eye, so there’s no way to emulate video signal artifacts accurately from that regardless.

The only way is to use the raw palette to approximately create the NES signal from scratch on our own, and decode and color-correct that palette in our own way.

I’m used to emulation where the colors are generated from the voltages themselves. Also, before switching to patchy-ntsc, I generated a palette from gamutthingy and used that in addition to my LUT. That seemed to work acceptably. I don’t know if that palette is able to avoid the clamping or not. All that I am looking for is a way to apply the luma and hue skews on top since not all NES emulators have palette support because they already handle the color correctly with voltages.

@PlainOldPants

I forgot to directly address this. Unfortunately, I have nothing to equate the number to, or anywhere to get it. The solution probably is to look at the output of a real console using an oscilloscope and derive these numbers from that. Otherwise, it’s the same thing as with the Y/C filtering inside the console: The only solution in my shaders is to eyeball it, or you can use anikom15’s scanline classic shaders which I can only guess have this taken care of already.

Are you open to the idea of moving this conversation over to NesDev if I start a thread there? I have a good rapport with lidnariq, and I think a lot of the hardware pieces to the “shader equation” can be cleared up from the group over there who are well versed in baseband signals and video output from consoles.

[PlainOldPants]

@kev4cards

I’m used to emulation where the colors are generated from the voltages themselves. Also, before switching to patchy-ntsc, I generated a palette from gamutthingy and used that in addition to my LUT. That seemed to work acceptably. I don’t know if that palette is able to avoid the clamping or not. All that I am looking for is a way to apply the luma and hue skews on top since not all NES emulators have palette support because they already handle the color correctly with voltages.

Not your fault, but I’m not sure what you mean by this. Rereading question 3 from your last post, I’m confused now as to what exactly you were asking about or what you meant. The idea of incorporating the hue and luma skews without using the raw palette doesn’t make sense to me since that step is already completed and baked into the NES palettes, like the one you had output by gamutthingy, and it’s part of my video signal emulation process with the raw palette too.

I must’ve been very tired or distracted when I answered you last, too, because that tangent I went off on doesn’t really directly answer the question you asked. The raw palette is required to emulate the signal, and using measured voltages is a required part of that process too. Since the process of making an NES palette as RGB values is only based on decoding the signal in non-invertible ways, it’s not possible to take an NES palette (other than the raw palette) and construct a video signal based on it. That’s even if the emulator outputs the palette “correctly using voltages” which isn’t invertible and still has multiple different “correct” ways based on the same voltages.

My shaders are using lidnariq’s measured voltage levels to generate the NES video signal as a perfect square wave as outlined in the nesdev wiki. I did the row hue skew in a hacky way temporarily (which makes de-emphasized colors get skewed wrongly). (The slight luma increase on the de-emph columns is missing, however.) All we can do is try to make a new NES palette from this output. There’s a prependable “test pattern” shader with an option to show the whole NES raw palette as a grid of differently colored rectangles, so that might be useful for this. All I have left to do is make my shaders compatible with the gamutthingy LUT. (I believe this should be doable, unless I’m not thinking things through. Negative values clamping shouldn’t be a real problem if I make my shaders compatible with the postgammaunlimited mode properly.)

On the subject of the NES, I just realized there’s a small fix I need to do with the black pedestal crush too (though I believe that isn’t in the October version that you have.). So much cleaning up that I need to do with this shader code now that I’m coming back to it.

@kev4cards

Are you open to the idea of moving this conversation over to NesDev if I start a thread there?

I’m fine with that. This might hopefully help me get some direction on how to go about implementing the Genesis signal properly.

[kev4cards]

@PlainOldPants

Not your fault, but I’m not sure what you mean by this. Rereading question 3 from your last post, I’m confused now as to what exactly you were asking about or what you meant. The idea of incorporating the hue and luma skews without using the raw palette doesn’t make sense to me since that step is already completed and baked into the NES palettes, like the one you had output by gamutthingy, and it’s part of my video signal emulation process with the raw palette too.

So, both higan and ares generate their own palette based on hardware voltages (I’m pretty sure they are from lidnariq’s numbers). Anyway, I think what I’m looking at is finding out first if the emulator is not incorporating the hue and luma skews into its calculations. I think there is good reason for the emulator (and others with similar approach) to handle this. Then, I can make presets according to who uses the raw palette and who does not. So, basically, I’m trying to cover all of the scenarios that I can. I don’t want to neglect Nestopia or Mesen users when my daily driver is ares. I want to be fair to most of the commonly used active emulators.

@PlainOldPants

My shaders are using lidnariq’s measured voltage levels to generate the NES video signal as a perfect square wave as outlined in the nesdev wiki. I did the row hue skew in a hacky way temporarily (which makes de-emphasized colors get skewed wrongly). (The slight luma increase on the de-emph columns is missing, however.) All we can do is try to make a new NES palette from this output. There’s a prependable “test pattern” shader with an option to show the whole NES raw palette as a grid of differently colored rectangles, so that might be useful for this. All I have left to do is make my shaders compatible with the gamutthingy LUT. (I believe this should be doable, unless I’m not thinking things through. Negative values clamping shouldn’t be a real problem if I make my shaders compatible with the postgammaunlimited mode properly.)

I know postgammaunlimited mode (applied through grade) is a bit unusual for RetroArch shaders, but it works really well with koko-aio. I can reach a point of physiological nostalgia with how well the LUT and koko-aio work together. Adding the proper NTSC effects just further solidifies that feeling.

@PlainOldPants

I’m fine with that. This might hopefully help me get some direction on how to go about implementing the Genesis signal properly.

I opened up a topic under “Other Retro Dev”. It should appear once it has been approved. I’m hoping it will be beneficial to everyone in getting the puzzle completed.

Lastly, I’m hanging onto the “color initial offset” for Genesis. I want to get my preset pack out, and I can worry about corrections later. With the color initial offset in place, the lowpassWindowStdBaseSize came out to 0.525, and the lowpassSincStdBase Rate was 0.800. That’s where my unorthodox, nostalgia-guiding approach took me. I believe chroma bandwidth is normally 1.5 MHz. So, I’m not sure if those variable values align well with the bandwidth or not.

[Superdonkeyjack]

Pants, what shader can I use for the Genesis and 5th gen consoles from your presets?

[anikom15]

@kev4cards The problem I have found with Mega Drive is that if you ask five different people what it should look like you will get five different answers. Going by the schematic was the only reasonable choice I could make for it. I generally follow schematics for all the presets to the best I can find them. Most other consoles are more consistent.

The Famicom voltage outputs are fundamentally different from later consoles that output RGB and then convert to composite in the analog domain. The Famicom outputs direct composite voltages digitally. If the emulator wants to hold to the ‘output voltage directly’ ideal than it would have to output the dot clock pattern directly, not just the 256x240 framebuffer they do now.

Scanline Classic reconstructs the analog composite signal by first reconstructing the dot clock signal and then applying the appropriate cosine functions to chroma part of the signal. For Famicom/NES I would do the same thing but map chroma to a digital pulse pattern instead of multiplying chroma by a cosine. This is why there are no NES/Famicom presets yet (in fact there’s nothing earlier than the PC Engine yet).

[kev4cards]

I understand that the Mega Drive does fluctuate quite a bit due to wiring problems with the luma notch and the series of encoder changes from revision to revision, model to model. With that said, as I stated in Looking to combine an NTSC shader with koko-aio and a LUT, there does seem to be something else on for the shader to not be able to recreate the output seen in the YouTube video. I’m not sure if that is a bug in the shader or the schematic is incorrect or what. The only thing that appeared off to me was the pixel clock parameter within the gen.slangp preset is actually the master clock for the Genesis. I assume that’s intentional as the other preset files appear to follow that pattern, but I don’t know how you then adjust for the actual changing pixel clock based upon when H32 or H40 mode is used.

For Famicom, I realize I was thinking about it incorrectly. However, higan and its successor ares render at the pixel-level with cycle accuracy. So, with their lack of support for .pal files, I was seeking a way to apply the NTSC effects on top of the palettes those two emulators already produce natively. I like to accommodate emulators that lack .pal support as well as have presets for those that do.

Lastly, there is a topic over at NesDev to garner some more details from the folks over there who are well practiced in the space of signals: https://forums.nesdev.org/viewtopic.php?t=26521 . I hope that can become useful for technical information surrounding NTSC signals (and possibly PAL if discussion heads that way).

[anikom15]

@kev4cards

but I don’t know how you then adjust for the actual changing pixel clock based upon when H32 or H40 mode is used.

Fuzzy math is used. The Mega Drive uses a variable pixel clock, but thankfully the visible area portion of the screen (including borders) uses a fixed pixel clock, so the variable aspects can be ignored. The shader uses a function that goes through a number of steps to derive precise timing values, including the number of effective pixels per active line. This comes out to 420 for H40 mode. I forget what the number is for H32 but it’s exactly the same as the hardware.

You can find all the code for this in modulation.inc.

[PlainOldPants]

https://drive.google.com/drive/folders/1oGSvEAvGeJsXobgm9tOLhx0BtL3YvWui

I made a horribly laggy scanline shader. I’d appreciate any feedback, especially from coders.

I also made a quick shader that replaces blueMSX’s hardcoded MSX1 palette with two other palettes, and another shader that rescales pixel art with square pixels more evenly, with the edges smoothed slightly so that the pixels appear closer to the same size as each other.

Someone who is good at the economy please help me optimize this. My GPU is dying.

I’m hoping to add in an aperture grill effect too.

Here are some LUTs and NES palettes generated using https://github.com/ChthonVII/gamutthingy while I’m here: https://drive.google.com/file/d/1aXmnEsUdW4xDNqliGdhQC2xAsQ6wH8T5/view

(Game: Animal Land Satsujin Jiken, English patch alpha by furrykef on GitHub)

[guest.r]

Hehe, there are some hidden gems in there indeed:

float result = 1 - 1 / pow(1 + a1 * x + a2 * pow(x, 2) + a3 * pow(x, 3) + a4 * pow(x, 4), 4);

float level0 = 1.0 / pow(y - x, 1.0);

(y / sqrt(2) * approxErf(y / sqrt(2)) + exp(-pow(y / sqrt(2), 2)) / sqrt(pi))

This line is nested in a double loop:

vec3 inputColor = pow(clamp(texture(Source, vec2(pixelX, pixelY)).rgb, 0, 1), vec3(2.16));

I guess, beside small optimisations, more passes could bring significant speedups…

[PlainOldPants]

A month late posting this, but here’s an updated version of my NES and MSX1 palettes (but not LUTs) generated by an edited version of ChthonVII’s tool gamutthingy. Unfortunately, the new NES palettes have extremely saturated colors, but still, I insist that they are based in an accurate understanding of the hardware.

NES palettes: https://drive.google.com/file/d/1Ws6ExbM6a4Q-0wwMTLe-Dude27wAe04g/view To install in retroarch, copy an NES .pal file into the RetroArch “system” folder, and rename it to MesenPalette.pal for Mesen, nes.pal for FCEUmm, or custom.pal for Nestopia UE. Then, restart the core, and switch to the “Custom” palette in the core’s settings.

Shader for the bluemsx core that replaces bluemsx’s MSX1 palette with my palettes: https://drive.google.com/file/d/179964K4iNPGELhodMnB6sFX0Tsr97jas/view Make sure your machine type is set to “MSX”, and restart the entire core after changing the machine type. This shader is only made to be used with the bluemsx core with the “MSX” machine type. To use the shader: 1. Copy both files into RetroArch’s shaders/shaders_slang folder, 2. Check RetroArch’s settings and make sure that your video graphics driver is either Vulkan or Direct3D 12 (or 11? I forget whether it’s 11 or 12 that supports .slang shaders) and 3. After loading a game, open the quick menu and navigate to Shaders–>Load Preset–>shaders_slang–>bluemsx-palette-fixer.slangp. 4. In Shaders–>Shader Parameters, make sure the LUT setting is on, and use the setting directly under it to switch between the NTSC or PAL palettes.

The NES palettes are updated to no longer do any AGC on chroma based on the colorburst’s amplitude, since I’ve been told that NTSC CRTs generally don’t do that. As a result, these palettes are very, very saturated.

The MSX1 palettes are now generated using the chroma amplitude and phase columns from the TMS9118 datasheet screenshot that someone posted here. The previous MSX1 palettes were based on the TMS9928’s R-Y and B-Y values under the assumption that those were true R-Y and B-Y, but since it’s not clear whether those are true R-Y and B-Y or U and V, those MSX1 palettes may have been inaccurate. These updated MSX1 palettes don’t have to deal with that ambiguity, so they should be accurate, assuming that the amplitude and phase columns of that table are calculated correctly by the authors of the datasheet.

These updated palettes are generated using the Oklab colorspace. All my previous uploads have used the Jzazbz colorspace.

Other than switching from Jzazbz to Oklab, I have no reason to update the LUTs from my previous post. Feel free to continue using those.

Gamutthingy doesn’t have support the Oklab color space, MSX palette generation, or globally darkening the palette by a factor. I edited those features into the program myself to make these palettes.

[PlainOldPants]

Another new batch of NES and MSX palettes, now generated using my own new palette generator instead of ChthonVII’s. The palette generator’s source code is included. https://drive.google.com/file/d/1k_NqcCiRCF1UFDtItequ_6jJRMkRCrQv/view

The improvement in the NES palettes comes from the way that colors outside the sRGB gamut are mapped. Previously, the goal was to preserve hue and intensity perfectly at the cost of saturation. In these new palettes, intensity is preserved perfectly, and chromaticity is kept as close as possible to the original color, striking a perfect compromise between hue and saturation. That improvement is easiest to see in the blue colors in the Toshiba palette, where the saturation is preserved a lot better than in the previous version of that palette.

The MSX palettes turned out almost identical to the previous batch. You won’t notice a difference between the previous ones and the current ones unless you look at the precise numbers.

[PlainOldPants]

I’ve decided to try making my own LUTs too, although they’re for a different purpose than the ones from gamutthingy. These new LUTs that I’m posting today map the colors in the same way as my NES palette generator just did. As such, these new LUTs should only be used with consoles or games that have tough color palette limitations, like 16-bit consoles or earlier for example, but even on those consoles, some games work around those limitations well enough that the gamutthingy LUTs are the better choice. For everything else, or when in doubt, the gamutthingy LUTs are better because they use a proper gamut compression mapping algorithm that preserves the differences between colors that are close to each other. https://drive.google.com/file/d/1ikfE96E4GY4_CkBtXOY5A4HKp6aTA_5W/view

Edit: I’ll also point out that this is my first time posting 256x256x256 LUTs. To make a 256x256x256 LUT work in RetroArch, I used a resolution of 4096x4096 for the actual PNG image’s resolution. Typically, LUTs use a resolution of 65536x256, but the LibRetro slang shader system doesn’t allow you to set one axis’s resolution that high.

[guest.r]

I think it’s great to have accurate color representations and LUTs are a very efficient implementation. Just out of curiosity, can you make some 1024*32 or 4096x64 versions? (by chance, guest-advanced lut size upper parameter limit is 64 )

I think bilinear sampling is a very good approximation of the initial intent, Retroarch lut samplers could also make use of them, since 1024x32 is like the supported format here.

[PlainOldPants]

From the start of this thread to the present, I’ve been using a different LUT sampler shader from the one in RetroArch. The one I’m using is by ChthonVII, and here’s what the comments in it say:

// retroarch can't correctly sample a 1.0 coordinate
// so we are going to add a just-under-half-step offset to red and green, then increase their divisors by 1
// This should get us a slightly lower coordinate within the same pixel
// This is where reshade's LUT is slightly inaccurate because it's taking a linearly filtered sample from the wrong spot.

In their shader, I also added an extra line of code towards the beginning of the shader to clamp the input RGB values between 0.0 and 1.0. Values out of those bounds cause the result to become a (0, 0, 0) black.

For convenience, here’s a link to the most recent batch of 128x128x128 LUTs I posted that were generated by gamutthingy, which comes with a slightly modified version of Chthon’s trilinearLUT.slang. https://drive.google.com/file/d/1aXmnEsUdW4xDNqliGdhQC2xAsQ6wH8T5/view

@guest.r

LUTs are a very efficient implementation

…unless you include the two hours it took for my lazy implementation to complete the 8 LUTs in my last post. ChthonVII’s can complete a LUT in a few seconds. The culprit is a 100000-iteration for-loop to brute-force search the closest valid in-gamut color for out-of-gamut colors.

Because 32x32x32 is such a low resolution, it’s better for you to only apply the LUT in a way that converts from some linearized colors to some other linearized colors, or I guess, from one gamma to the same gamma might also give decent results. Since ChthonVII couldn’t figure out how to put a 256x256x256 LUT into RetroArch, they had to create a solution for this problem already, by adding the option --lutmode postgammaunlimited. It’s then the shader’s responsibility to perform the NTSC/PAL color corrective matrix, the CRT EOTF, and the sRGB OETF. The LUT is applied between the CRT EOTF and sRGB OETF steps.

For your convenience, though, I’ve just generated these LUTs to include the NTSC/PAL corrective matrix, the CRT EOTF, and the sRGB OETF for you, so that all you need to do is apply the LUT just like you’re used to doing. Ones that say “gamutthingy” are by ChthonVII’s program which I’ve linked in this thread half a million times. The other ones are by my LUT generator, which took about 24 minutes total to complete all eight LUTs. https://drive.google.com/file/d/11hCmww0f9YH_s-_xmhdxpHK-J3-eWZ3-/view

As I’ve stated in my previous post, my new non-gamutthingy LUTs should only be used in games that have sparse color palettes, since mine will cause some colors to become equal to each other when real hardware doesn’t make them equal to each other. If you need to preserve gradients or preserve the differences between colors that are closer to each other, you should use the LUTs from gamutthingy.

I’ll come back here later and post some that don’t include the NTSC/PAL color correction, since that’s skipped on YPbPr and RGB connections.

Click here for the commands I input into gamutthingy to generate the LUTs.

build/gamutthingy --lutgen true --lutsize 64 --crtemu front --crtdemod panasonic_CT36D30B --crt-saturation-knob 0.95 --crtclamphighenable false --crtblack 0.0023384888513 --crtwhite 1.34273107128 --source-primaries P22_panasonic_CT36D30B --source-whitepoint D65 --no-source-adapt true --outfile panasonic_CT36D30B_gamutthingy_res64_2026-10-31.png

build/gamutthingy --lutgen true --lutsize 64 --crtemu front --crtdemod TA7644BP --crt-saturation-knob 1.1 --crtclamphighenable false --crtblack 0.0023384888513 --crtwhite 1.34273107128 --source-primaries P22_toshiba_blackstripe_patchy68k --source-whitepoint 9300K27mpcd --no-source-adapt true --outfile toshiba_CF2005_gamutthingy_res64_2026-10-31.png

build/gamutthingy --lutgen true --lutsize 64 --crtemu front --crtdemod rca_colortrak --crt-saturation-knob 1.2 --crtclamphighenable false --crtblack 0.0023384888513 --crtwhite 1.34273107128 --source-primaries P22_rca_colortrak_patchy68k --source-whitepoint customcoord --source-whitepoint-custom-coords 0.294,0.309 --no-source-adapt true --outfile rca_e13169gm-f02_gamutthingy_res64_2026-10-31.png

build/gamutthingy --lutgen true --lutsize 64 --crtemu front --crtdemod CXA1870S_JP --crt-saturation-knob 1.2 --crtclamphighenable false --crtblack 0.0023384888513 --crtwhite 1.34273107128 --source-primaries P22_trinitron_bohnsack --source-whitepoint 9300K27mpcd --no-source-adapt true --outfile sony_CXA1870S_JP_gamutthingy_res64_2026-10-31.png

build/gamutthingy --lutgen true --lutsize 64 --crtemu front --crtdemod CXA1870S_US --crt-saturation-knob 1.2 --crtclamphighenable false --crtblack 0.0023384888513 --crtwhite 1.34273107128 --source-primaries P22_trinitron_bohnsack --source-whitepoint 9300K27mpcd --no-source-adapt true --outfile sony_CXA1870S_US_gamutthingy_res64_2026-10-31.png

build/gamutthingy --lutgen true --lutsize 64 --crtemu front --crtdemod CXA2060BS_US --crt-saturation-knob 1.0 --crtclamphighenable false --crtblack 0.0023384888513 --crtwhite 1.34273107128 --source-primaries P22_trinitron_bohnsack --source-whitepoint D65 --no-source-adapt true --outfile sony_CXA2060BS_US_gamutthingy_res64_2026-10-31.png

build/gamutthingy --lutgen true --lutsize 64 --crtemu front --crtdemod CXA2060BS_JP --crt-saturation-knob 1.0 --crtclamphighenable false --crtblack 0.0023384888513 --crtwhite 1.34273107128 --source-primaries P22_trinitron_bohnsack --source-whitepoint 9300K8mpcd --no-source-adapt true --outfile sony_CXA2060BS_JP_gamutthingy_res64_2026-10-31.png

build/gamutthingy --lutgen true --lutsize 64 --crtemu front --crtdemod CXA2060BS_PAL --crt-saturation-knob 1.0 --crtclamphighenable false --crtblack 0.0023384888513 --crtwhite 1.34273107128 --source-primaries P22_trinitron_bohnsack --source-whitepoint D65 --no-source-adapt true --outfile sony_CXA2060BS_PAL_gamutthingy_res64_2026-10-31.png

[guest.r]

Thanks for your dedicated work. I tested some of the 32size LUTs and the results were within expectations.

Personally i have always been a fan of such grading approaches, lut pre-calculations can include many features and even stuff, which is quite demanding even to organize as ‘on-the-fly’ shader code.

I also hope more of this will get into the repository at a point of time, there should be definitely more interest in accurate representations.

Also have an idea to test and compare 256 vs 32 dimension LUTs as they are applied on selected images and special tools are used to display color differences. Might be worth a try.