The 4.2 MHz lowpass is broken. All the presets are skipping it by default.
The problem is that my 4.2 MHz filter implementation reduces the chroma saturation. In other words, it is not actually behaving according to the graph, as it’s causing the 3.58 MHz frequency (as generated by the encoding pass) to get decreased too much.
I had implemented the chroma and luma filters first, and it looked like they were working alright. The RF lowpass was added later, so I didn’t realize something was off until then.
I need to check this later, but part of the culprit could be this bug in my hamming window. The original function from my June 12th version looked like this:
vec4 window(vec4 offsets, float hammingSize) {
const float a0 = 25.0 / 46.0;
return a0 + (1 - a0) * cos(2 * pi * clamp(offsets / hammingSize, -0.5, 0.5));
}
The fixed version is this. This is found in my latest version.
vec4 window(vec4 offsets, float hammingSize) {
const float a0 = 25.0 / 46.0;
vec4 res;
for(int i = 0; i < 4; i++) {
res[i] = offsets[i] >= -0.5 && offsets[i] <= 0.5 ? a0 + (1 - a0) * cos(2 * pi * clamp(offsets[i] / hammingSize, -0.5, 0.5)) : 0;
}
return res;
}
I only remembered to paste this fixed version in the encode and decode passes, but not in the RF lowpass pass. Still, I doubt that this is the entire problem. As you said, the simple discrete time FIR filter isn’t necessarily the best choice. Until I added the RF lowpass, I had thought that I was over-sampling high enough to where this wasn’t an issue, but clearly I was not.
Based on your previous message to me in your crt-beans thread, and based on what you’re saying here, I can clearly see that you are more experienced with signal processing/filtering than I am.
About the 4.2 MHz lowpass only being applicable to RF, that’s true according to the standard. I need to check the schematic diagrams to make sure that consoles are properly skipping that lowpass when connected only over composite.
I believe I remember the SNES having different filtering for the final RF output and for the final composite output–That’s two different lowpass filters (or whatever kind of filter), where one is for composite, and the other is RF. Again, I need to check to make sure I’m remembering correctly. I do not own a real SNES.
My NES has excessively blurry RF and excessively sharp composite. If I connect the NES’s composite through my VCR to modulate it into RF so that I can connect it to my 1989 RCA CRT, I still see the grainy, sandy look of the NES’s overly sharp composite, although it’s not as horrible as the straight composite signal.
I also believe the NES’s RF modulator may be causing the NES’s chroma to get more saturated. It may also be causing the luma to create more chroma artifacts. I need to check this later.
As for my Genesis, I don’t know. At least, everyone knows that the Genesis low-passes its audio, but I suspect that they were using the same audio low-pass filter for RF and for composite. That might suggest that video goes through the same low-pass for both RF and for composite. I need to check the schematic and see.
Speaking of filtering, I just noticed this past weekend that my 1989 RCA ColorTrak is being relatively generous with its chroma lowpass/bandpass. It looks like it is not lowpassing as low as the standard Q response, but it is definitely lower than the standard I response. I’ll need to investigate.
Long story short, the answers to most of our questions about filtering are in the schematic diagrams.
I have not gotten the capacitors replaced on my consoles. This might be affecting the video filtering. My Genesis has the well-known rainbow banding issue, and I’ve heard that you can reduce it a lot by replacing capacitors. The only CRT I have that’s been serviced is the 1989 RCA one, which still has its original whitepoint intact, but has its default brightness set based on 0 IRE and its default tint and color set to make the yellow area look acceptable without accounting for 1953 NTSC primaries. Everything else that I own may have capacitor-related problems with filtering.
That is correct. The number 2560 ensures that resolutions of 256, 320, 512, and 640 all divide each pixel into an integer number of samples. I don’t remember exactly how my code works and whether it properly handles an input size that doesn’t divide 2560 evenly. I know my previous release would either crop or add pillarboxes to ensure that the sample rate was divided evenly. As a bonus, that meant that if the user had enabled cropping overscan, or if the console (such as SNES or Genesis) supported multiple resolutions, the shader would automatically detect and compensate for that.
As far as I’m aware, never. I’m sure that consoles never ever did this.
The reason why I added that was so that I could try watching YouTube videos with the shader (either using ShaderGlass or the WindowCast core; I recommend ShaderGlass for better performance), and calibrate the NTSC brightness, tint, color, and sharpness manually by eye. This is because of how consumers could’ve done anything with their knobs, and how they may have set their TV’s settings manually instead of sticking to defaults.
The Genesis/MegaDrive is the only exception I know of. The RGB analog voltages are not linearly related to their internal digital values. Different Genesis/MegaDrive models had different voltages. According to the Genesis 240p test suite, the Genesis’s black level is about 6 IRE.
In Genesis Plux GX, you get the console’s raw digital RGB values. In BlastEm, you get the analog voltages. I assume everything except BlastEm also only has the raw digital values. https://github.com/ekeeke/Genesis-Plus-GX/issues/345#issuecomment-1675678054 “Literally all I did in BlastEm was took the voltages others had measured for each of the Mode 5 RGB levels, divided by the max measured voltage and multiplied by 255. Those values are in levels in vdp.c and get used for translating palette entries. Nothing fancy at all.”
Thank you. A lot of the detailed comments in there were just to compensate for my own crappy memory and difficulties with understanding code after it’s been written (and maybe because I’m too nerdy), but I’m happy to see that all my documentation and explanations are paying off, whether it’s in the code itself, or in my detailed posts that have hardly any views/downloads. Just beware that some comments were written before I actually wrote the code and are still unchanged.
I’ll look at this in more detail later, since I’m in a bit of a hurry. It took longer than expected for me to write this.
Edit, about 2 hours later:
I’m looking over the patent now, and this paragraph in particular on page 4 (or page 10 of the PDF) sticks out to me:
“In accordance with the present invention, an adaptive
comb filter for NTSC or PAL television standards has only
two horizontal scan lines of delay and effectively removes
the high frequency luma. The magnitude and phase of the
chroma signal from the comb filter are compared to the input
signal to the comb filter which has been subject to a
bandpass filtering process. Unless both the magnitude and
phase of the comb filter chroma signal are equal to or less
than that of the bandpass filtered signal, the comb filter
chroma signal is assumed to be incorrect (“illegal’) and a
different comb filter is adaptively selected to legalize the
chroma output signal.”
This is similar to what my latest NTSC shader is doing for its adaptive comb filter. What makes mine different is that, instead of checking whether a comb filter gives a “legal” result before falling back to another comb filter, mine compares all the comb filters to see which one is the most “legal”, even if all three are illegal. In other words, this paper’s invention puts the different comb filters in an order of priority, and always picks the highest-priority one that picks a legal result, whereas my shader assigns no priority order at all, and always picks the one that has the best result.
Another difference in my shader is that I perform a 1.3 MHz lowpass on each line, and I assume that this lower frequency range is almost entirely correct luma information. Then, when comb-filtering, the comb filter only is performed on the higher frequency information above that. This means that consecutive lines won’t get blurred together as badly. This patent instead claims that the entire luma signal needs to be comb filtered because even the lowest frequencies are allowed to contain chroma.
I was hoping to use it for some visual comparisons.