PlainOldPants's Shader Presets

[anikom15]

The missing dot is from the scanline with one less PPU dot every other frame. But you can’t actually know which frame is odd or even without guessing, right?

Let me put it this way: the NES video output can be represented as a 4096x240 monochrome image, right? If we take that and know the relative time for each subpixel t, we should be able to treat it as the composite signal directly and determine the right phase to demodulate at each subpixel. The raw palette gives us the information needed to reconstruct the monochrome image, but doesn’t give us the info to get time at that subpixel. We still need to guesstimate that, right? Is there any room bitwise in the raw palette to squeeze in more metadata? Like you would only need one bit somewhere to flag the current field, another to flag if the missing dot is present in the field, etc.

[PlainOldPants]

@anikom15

Is there any room bitwise in the raw palette to squeeze in more metadata?

There actually is, since the colors in the NES’s raw format are only 9 bits each, while the raw palette expands this into 24-bit color. Staying compatible with existing shader presets might be slightly tricky, but it’s doable, although it should also be simple to modify those shaders too, since there are only five presets total.

[anikom15]

Is the raw palette an actual palette like the other palettes? Or is it a different mode entirely? Like the does the emulator actually read off current the PPU state to write out the output color value?

[PlainOldPants]

It is just a color palette. Shaders currently have to make up the rest of the information from thin air, like with mine which has a manual switch for a 2-frame or 3-frame phase cycle, or the other shaders which only support a 2-frame cycle.

[Azurfel]

@Cyber

Scanline gaps do cause uneven wear over time though.

I remain unconvinced that this is the case for at least LG panels from 2020 on, so long as the Screen Move/pixel shift mitigations remain enabled, and the refresh cycles are allowed to run every 4-6ish hours of use.

[Cyber]

Well not everyone who has a WOLED TV has a 2020+ LG Display OLED panel eh?

I can only speak from my experience with my 2016 LG E6P which had Pixel Shift enabled and I never unplugged my TV so it was able to run all its panel refresh cycles on schedule.

The thing is once I noticed that there was an issue, Clear Panel Noise made no difference whatsoever.

Not sure if it was due to the fine pitch between the more worn and less worn areas.

Plus there’s no way to predict how every user is going to use their TV.

[Nesguy]

@anikom15

I remember trying one of the raw palette shaders and thought it was ugly.

“NES Color Decoder” looks very similar to Composite Direct FBX. I think NES Color Decoder + Raw looks a bit better.

@PlainOldPants

That is why using FirebrandX’s Composite Direct palette (or any normal NES palette at all) with a composite video emulation shader does not work

I came to the same conclusion- the NES needs its own composite video shader.

I’ve been doing a crude workaround where I just eyeball settings between an RGB and composite video preset (guest-advanced and guest-advanced-ntsc) until everything looks equal, and then applying the NES color decoder. Not ideal, obviously.

This translates to around +20% Saturation and +20% NTSC Saturation using guest-advanced-ntsc.

[anikom15]

@PlainOldPants The raw palette will send its RGB values to the shaders in normalized 0…1 values. This is what we have from NESDev:

Standard Video

Type	IRE level	Voltage (mV)
Peak white	120
White	100	714
Colorburst H	20	143
Black	0	0
Blanking	0	0
Colorburst L	-20	-143
Sync	-40	-286

NES Measurements

Signal	Potential	IRE
SYNC	48 mV	-37 IRE
CBL	148 mV	-23 IRE
0D	228 mV	-12 IRE
1D	312 mV	≡ 0 IRE
CBH	524 mV	30 IRE
2D	552 mV	34 IRE
00	616 mV	43 IRE
10	840 mV	74 IRE
3D	880 mV	80 IRE
20	1100 mV	110 IRE
0Dem	192 mV	-17 IRE
1Dem	256 mV	-8 IRE
2Dem	448 mV	19 IRE
00em	500 mV	26 IRE
10em	676 mV	51 IRE
3Dem	712 mV	56 IRE
20em	896 mV	82 IRE

I’m changing black to 0 because we are going to assume no setup on black. I believe blargg did his own measurements, but can’t find them. Do they line up with this chart? I’m concerned about the repeatability of this measurement and how the IRE values were derived (looks like assuming 1 V is 100 IRE and defining 0 IRE as 312 mV. We know 0D must be less than 0 IRE because of how it can be interpreted as a sync on some TVs).

Let’s simplify this to only the grayscale palette values and ignore the IRE:

Signal	Potential
0D	228 mV
00	616 mV
1D	312 mV
10	840 mV
2D	552 mV
20	1100 mV
3D	880 mV
30*	1100 mV

*Not measured, assuming same as $20.

R value from the raw palette represents a pair of voltages indexed from 0 to 3:

R-value (Normalized)	Potential Low	Potential High	Vpp
0	228 mV	616 mV	388 mV
1/3	312 mV	840 mV	262 mV
2/3	552 mV	1100 mV	548 mV
1	880 mV	1100 mV	220 mV

However, when there’s emphasis, it will modify this only when the emphasis attenuator is active:

R-value (Normalized)	Potential Low	Potential High	Vpp
0	192 mV	500 mV	208 mV
1/3	256 mV	676 mV	420 mV
2/3	448 mV	896 mV	448 mV
1	712 mV	896 mV	184 mV

We can turn that into an array:

[[0.228, 0.616],
 [0.312, 0.840],
 [0.552, 1.100],
 [0.880, 1.100],
 [0.192, 0.500],
 [0.256, 0.676],
 [0.448, 0.896],
 [0.712, 0.896]]

x is the low-level PPU clock output position which we calculate ourselves based on pixel position, current field, and if we’re playing Battletoads. In the active video portion there are 256 pixels corresponding to 2048 clock cycles. We can calculate Y:

Convert R-value into an appropriate integer, 0 to 3.

Check which attenuator bits (B) are set AND check if the attenuator color cycle is active for this x based on which bits are set. If it is, add 4 to R-index.

Check if the current cycle for the given hue (G) gives us voltage high or voltage low.

Y(x)= Array[R-index + 4 * Emphasis(B, x)][high or low from VHL(G, x)]

Emphasis(B, x) is a function that returns 0 or 1, VHL(G, x) is a function that returns 0 or 1.

Do I have that right? If so, I can figure out how to handle G and B values later, and then finally scaling the voltage levels to the appropriate (unenforced) range, 0 to 1 corresponding to 0 to 100 IRE.

[PlainOldPants]

@Nesguy

“NES Color Decoder” looks very similar to Composite Direct FBX. I think NES Color Decoder + Raw looks a bit better.

It may look good at first glance, but it’s actually the worst one. In my previous post, I didn’t explain this because I was trying to stay concise and on-topic.

For the heck of it, I’ll go through all five of those NES raw palette shader options and go over what’s wrong with each one. Information like this isn’t easy enough to find on the internet. This table is a bit of a rush job, being done largely from memory.

	cgwg-famicom-geom	gtu-famicom	ntsc-nes (or nes-color-decoder)	pal-r57shell-raw	patchy-mesen-raw-palette
Performance	Fastest, highly optimized.	Fast, simple.	Fast, simple	Idk.	Crap. Slow, poor quality code
Actually does the NES signal	Implemented.	Implemented	Maister SNES signal. All color filtering is done before encoding, which is wrong.	The only PAL option available. Horribly wrong colors.	Implemented
Battletoads 3-frame phase cycle	Unsupported	Unsupported	Unsupported	Not applicable	Supported via manual toggle in settings
Frequency Filters	Good “windowed sinc” FIR filters, but chroma is too sharp.	Just a raised-cosine integral lowpass filter. Luma still contains the subcarrier, and chroma is too sharp.	Precomputed in MATLAB.	Idk.	Unfinished “windowed sinc” FIR filters. Settings are just guessed by eye, looking at a CRT and video capture. Technically can fix with settings, but who has the time?
Comb Filter	Adaptive with notch. Not a good choice for NES.	Wrong! Mid-line comb filter.	Unsupported, trivially correct	It does something PAL-specific, but idk.	Off by default; trivially correct.
Row skew	Unsupported	Unsupported	Unsupported	Not applicable	Only supported for colors without de-emphasis.
NTSC color	Unsupported	Unsupported	CXA2025AS US axes, but wrong whitepoint, no Sony primaries, and wrong default tint/color settings. Defeats the purpose.	Not applicable	Various chips’ US and JP axes, but wrong whitepoint by default, and wrong default tint/color settings. Defeats the purpose. Fixable with settings, but who has the time?
Gamma/EOTF	Idk.	Unsupported, but appendable	Totally wrong!	Idk.	Wrong, but can technically fix with settings, but who has the time?
Over-brightened color clipping	Clamps at 255	Clamps at 255	Can clamp,darken, or desaturate. “Desaturate” is in R’G’B’ space, not great.	Idk, but the whole shader (at least the .slang version, idk about GLSL) looks absolutely disgusting.	Can darken the entire screen uniformly, and then clamp at 255.
Phosphor gamut	Not included, not appendable	Not included, appendable	Not included, appendable	It does something?	By ChthonVII’s program, gamutthingy: LUT for Sony P22 (and others) with Jzazbz-based gamut compression.

All of them are wrong. As of today, I recommend using my latest NTSC shader release, with p68k-fast-mp-nes-unfinished, which does all the above steps well, except that row-skew still is only supported for colors without de-emphasis (so games like Darkwing Duck (I think) and The Immortal will look off) and the BT.1886 EOTF is unused by default in favor of a straight 2.2 power law. (Edit: Now that I think of it, my shader here doesn’t support the Battletoads phase properly anymore, due to a bug that got introduced when adding interlacing support.)

@anikom15

I’m concerned about the repeatability of this measurement

While I don’t know where blargg’s measurements are, I do know there was at least one other post on the nesdev forums that showed different results. Those different results can be found in gtu-famicom. Notice how gtu-famicom only attenuates by multiplying by a constant factor, instead of switching to another pre-defined value. So, I am also concerned about repeatability. For now, this is the best I think we can do.

@anikom15

Do I have that right?

Just one thing: If hue is 14 or 15, you set Y(x) to the 1D voltage constantly, regardless of level or emphasis.

[Nesguy]

@PlainOldPants

All of them are wrong. As of today, I recommend using my latest NTSC shader release, with p68k-fast-mp-nes-unfinished

I’d really like to, but I’m pretty much locked into an HDR setup, now.

That, and there are some (gamma?) difficulties combining it with guest-advanced - I’m not sure what the fix is for that. Maybe a different CRT shader would work better.

[anikom15]

@Nesguy have you tried using float_framebuffer in your presets? That may help prevent inaccurate values when going from one shader to the next. If a shader outputs in linear space or outputs negative values or values past one, you must use a float_framebuffer.

@PlainOldPants Is row skew the measured phase distortion that increases at brighter values? I believe implementing this would require an NES specific composite demodulation shader in addition to the encoder to modify the phase applied to the mixing functions. A phase error in the output can be represented as an error in demodulation.

Also, what do you mean by gamma/EOTF in that chart?

[PlainOldPants]

@anikom15

Is row skew the measured phase distortion that increases at brighter values? I believe implementing this would require an NES specific composite demodulation shader in addition to the encoder to modify the phase applied to the mixing functions.

That is true. My implementation doesn’t correctly do row skew, but instead, it has a correction only in the decoder that detects certain Y values and fakes the hue rotation based on that. That is why mine does not support row skew with de-emphasis. The technically correct way would be to put row skew when encoding.

@anikom15

Also, what do you mean by gamma/EOTF in that chart?

I screwed up that entire row. It was partly about whether a power law or more complex EOTF (either way) is implemented correctly, but it also became about whether you’re able to adjust gamma, or whether the shader conflates the linear luminances with gamma corrected voltages (as in the colorimetry shader or Drag’s palette generator). I’ll need to fix that row of that table.

[anikom15]

Why would a gamma function need to be used at all in an NTSC shader? The output voltages from the device are already in gamma space. On the decoding, YPbPr matrix should output to gamma-corrected RGB values either for direct display or further processing by another shader.

Do the decoding chips on TVs involve linearization and gamma in their intermediary processing?

[PlainOldPants]

No, it’s not something that the decoding chips are doing. It’s done separately after simulating the signal, for simulating CRT colors. Some of these video signal shaders, including mine, include these color processing steps, in case you want to play without appending a CRT shader. Mine also let you turn off all this color processing so that a CRT shader can do these steps in its own way. Some of the shaders fail at this, such as ntsc-nes which does the entire NES palette and gamma correction before simulating the signal.

[anikom15]

That makes sense. I do color correction in the phosphor response stage, after YPbPr to RGB and before simulation of beam scanning (scanlines). The color correction is done as part of the phosphor shaders which involves linearizing the data, and transforms color to the light domain. The output remains linear up until the very end to allow for blending.

[Nesguy]

@anikom15

@Nesguy have you tried using float_framebuffer in your presets? That may help prevent inaccurate values when going from one shader to the next. If a shader outputs in linear space or outputs negative values or values past one, you must use a float_framebuffer.

I have no idea how to do this- does this require an additional shader pass?

[guest.r]

Just regarding the guest-advanced part, the shader chain mostly uses float framebuffers, it’s very well thought-through and tested with various testing environments. I tried @PlainOldPants presets with -hd version, looks regular, although there is some default brightness loss as one starts - due to mask and scanline effects.

To mention it, ntsc presets can also be followed by simple scanline and dotmask presets, but the adjustment options will be probably limited.

[Nesguy]

Quick sanity check. Does this look intended? (mask removed). Maybe it’s just much darker than I expect because I’m used to consumer-grade calibrations/settings?

Super-Mario-Bros-JU-PRG-0-251024-095736 hosted at ImgBB

[guest.r]

The output seems a bit darker on the ramp.

“Input”:

“Output”:

[anikom15]

You do it in the slangp, like this:

shader0 = "src/scanline-luma.slang"
filter_linear0 = "false"
float_framebuffer0 = "true"
scale_type0 = "source"
scale0 = "1.0"

That screenshot you posted doesn’t look right.