On CRTs, it works very well. You set 2.4 as the input and 2.2 as the output, and voilà , the colors pop beautifully!
However, when you try the same approach on non-CRT shaders, it messes everything up, creating an overblown, overly saturated, and bland image.
This has always intrigued me, so I decided to dig deeper and understand what’s going on under the hood.
One thing about CRTs is that the scanlines hide the gamma effect in the vertical dimension. It’s almost like the image is one-dimensional—horizontal only. Gamma correction is known for increasing brightness in lighter areas more than in darker ones. On CRTs, this effect is mostly noticeable in the horizontal dimension.
Take a look at the image below of crt-geom . See the dithering pattern on the US flag? The white squares become rectangles after gamma correction:
Everything is back to “normal”—at least what we’ve considered normal until now. We’ve avoided linear gamma on non-CRT shaders because it looks awful. So, we’ve stuck with uncorrected gamma. But look again at the last image and notice how the dithering pattern is isometric again—every dot is square. The overall image looks more regular, though bland compared to the CRT (with gamma correction).
I think you’ve already figured out the solution, right? Yes! In non-CRT shaders, we should only apply gamma correction when filtering horizontally ! We should skip linear gamma when filtering vertically. (We’ve been wrong for so long…)
So, I present to you the first Catmull-Rom filter with correct linear gamma (applied only horizontally):
It’s using the same input of 2.4 and output of 2.2 as CRT-geom. Now compare the flags and all the image details with the CRT version and see how it’s finally accurate! And the colors are just as beautiful as the CRT’s!
Here are some shaders I’ve converted to this approach, in case you’d like to test them on RetroArch:
An important caveat: this approach is specific to content made to run at 240p on CRTs. It’s not a general thing that can be applied to any image. It’s a particular characteristic of games made for CRTs!
I don’t know how well this could work with images made for modern devices or even 480i/p content made for CRTs.
Wow! This seems profound. I haven’t been able to follow the changes properly as I’m viewing on a small phone screen but I trust your judgement and theory and the example images look beautiful and as described.
I agree that CRT image have always popped in a beautiful way compared to what we have been doing here and I’ve tried very painstakingly to strike that balance between brightness, saturation, highlight and shadow detail and clipping/oversaturstion.
If this is as truly a game changer and other peers can verify your theory, I hope to see all shaders which have anything to do with colour and grading adopt these improvements across the board!
Not all shaders can benefit from this. Because some cannot separate filtering between horizontal and vertical dimensions. Maybe they can with some disadvantages…
@hunterk told me that there is another approach regarding gamma, which is ss-gamma-ramp (in the misc folder). I tested it and it can actually work as a gamma option in non-crt shaders. Although I think some of the benefits of linear gamma in just the horizontal dimension is that it brings the output closer to that of crt shaders. And this ss-gamma-ramp can’t reproduce.
Blending colors in a non-linear space is always wrong.
It makes sense to reduce vertical blending strength to account for the scanline gap. But there’s no reason that the amount of reduction should have anything to do with gamma.
I’d need to put more thought into it, but my hot take on the correct factor for vertical blending strength is G(H / (2*h)) / G(W / w), where G(x) is some Gaussian function that models phosphor glow at x distance from its center; H and W are the physical height and width of the CRT screen, and h and w are the height and width of the console output.
Did that sort of thing in shaders i wrote like crt-geom-mini, lottes-mini, fake-crt-geom-potato etc, after inspecting my CRT. White dots will expand horizontally only, vertically hidden by scanlines. Even a simple bilinear filter, horizontally only, is convincing this way like in fake-crt-geom-potato. That’s why they all have a linear pass first.
Maybe my execution isn’t the ideal answer. But I think it’s in the right direction. I know this because applying pure 2.4 gamma on these old games work beautifully when reproduced in a scanline environment and the same can’t be said when you apply the same 2.4 gamma and try to reproduce in a non-scanline display. Hence, something must be changed so that the reproduction is as good as with scanlines. Be either on the way gamma is applied, or in the blending process itself.
Talking about the blending itself, I’ve already did something by using the dilation shader horizontally. It outputs something good, though the effect is applied linearly instead non-linearly as the gamma.
In this case, instead applying gamma, you dilated horizontally the pixels. I saw this before in crt-easymode. Yes, this is some way to ‘mimic’ the effect of gamma on CRTs. The scanlines hide the gamma effect vertically someway, so you can just skip it and won’t notice much difference. That’s what I was trying to simulate on non-scanlines shaders.
Any shader you do that includes any filtering with a linear pass as source, filter and gamma out will expand pixels. Don’t ask me why lol. These shaders include gamma correction.
The logic of figuring out what to blend (e.g., edge detection or whatever) should probably operate as if you had a grid of square adjacent pixels. But the actual blending should use weights that take the geometric distance into account. Vertical neighbors are around 2 pixels away because of the scanline gap (though you also have to account for non-squareness).
Another way to look at it is that your gamma shennanigans are basically adding a faint scanline effect by darkening what the edges make us perceive as the “top” and “bottom” of certain pixels. (Though its backwards in that it affects brighter colors more strongly, when real scanline gaps as less prominent when colors are brighter.) So, why not make that explicit instead? Scale up bigger, apply a faint scanline effect using an implementation actually meant to model scanlines, then scale back down.
I can’t realize how the 2 pixels away scheme would work. It simply breaks the filtering.
There’s no correct way to apply gamma. In this particular case of games made for CRT, the gamma was defined as 2.4 because it had to take into account the effect of CRT scanlines. If scanlines didn’t exist, the artists would use another gamma, otherwise using 2.4 the picture would be washed out. Now, to “simulate” the CRT overall quality without the scanlines, you need to create a new way to apply gamma. That’s what I proposed. It’s not the final answer, though.
In my view, to get a CRT visual, you must deform the pixels from square to rectangles. No matter if using scanlines or not.
I have no technical background, but I want to share a thought: decades ago, no developer ever thought if their games would look good on a technology that would take many years to be implemented on consumer screens. They did it on CRTs and that’s it. For LCDs, it’s all uncharted waters, a new perspective, there’s no “right” or “wrong”. I suggest trying new approaches, testing and seeing which provides nice results and which doesn’t.
In the case of anything that looks like a Gaussian kernel, you just recompute the weights according to distance.
For stuff like xBR, it’s harder. You’d probably have to pick a single vertical blending factor (derived from the weights in the Gaussian case) and use that to reduce the blending strength whenever a pixel from another row is being blended into the central pixel.
In this particular case of games made for CRT, the gamma was defined as 2.4 because it had to take into account the effect of CRT scanlines. If scanlines didn’t exist, the artists would use another gamma…
No. CRT gamma is what is it because the only electron gun we know how to make has that particular nonlinear response to voltage. (And by happy accident, it happens to be roughly the inverse of human vision’s nonlinear response to luminance.) No one “picked” this gamma; physics picked it for us.
And developers during the CRT+console era were generally oblivious to CRT gamma. Since there was no alternative, no one ever thought about it. They absolutely were NOT working in linear space to make an “ideal” image and then applying the inverse gamma function. Rather they were just making something in raw R’G’B’ that looked good on their display and assuming/hoping it would look good on anyone’s display.
(Also, 2.4 is a bad approximation of CRT gamma. It’s really 2.5 with an offset and a linear(ish) toe near black. This is really worth reading. And the BT1886 Appendix 1 function is a pretty good approximation of actual CRT behavior. (The Appendix 1 function, and not the main Annex 1 function (which is crap).))
Now, returning to the problem at hand. You don’t like how your filters are blending vertical neighbors into each other more strongly than a CRT would. So why doesn’t a CRT blend vertical neighbors as much? Because they are farther apart, due to the blank scanline in between, so little of the phosphor glow reaches all the way across the gap into the neighbor. Note, now, that this has absolutely nothing to do with gamma.
(Also, I think part of what you don’t like is a consequence of treating CRT pixels as if they were a square area uniformly filled with a color. In reality, they’re closer to a Gaussian centered at a point (or, rather, 3 Gaussians centered at 3 points) that don’t completely fill the imaginary box around them, and aren’t constrained by it either. If the bright squares on that Street Fighter flag were circles instead, you’d probably be a lot happier with them.)
Then why does doing vertical blending in gamma space sort of give you the result you want? Two reasons: (1) It’s darkening a lot in the really salient case of a bright vertical neighbor blending into a dark one. E.g., that flag in Street Fighter. (Note, however that this is backwards from a real CRT where bright colors are the only ones that do bleed all the way across the scanline gap.) (2) Because it’s darkening in general, it’s basically applying a faint scanline effect. The cost, however, is that all of the vertical blending is wrong because it was done in gamma space. Also, this doesn’t really capture CRT vertical blending so much as hammer down that one salient case.
So what do I think would be a better solution? One of the following:
Modify how blending is done to reflect the physical distance rather than assuming vertical and horizontal neighbors are equally far away. Fairly easy for things that resemble a Gaussian kernel; harder for things that don’t.
Use something akin to anti-ringing to limit how much a pixel can change during vertical blending. Either across the board, or a special rule for half the rows corresponding to active scanlines in the original.
Do a faint scanline effect in a principled way at a higher resolution, then downscale.
I think I know what you mean. At first I interpreted as I had to consider alternating pixels from the center pixel. Now It looks like I have to shrink the filter kernel by half so that the weight applied at the first neighbor is very low. I’ll have to try it here (later I’ll post my results.)
This picture shows what I mean by shrinking the filter kernel. Red curve is shrinked by half:
About xBR, it’s more complicated because it uses filtering in diagonal directions and can’t be separated in horizontal and vertical dimensions. So for now I’ll concentrate only in separable filters (the ones inside interpolation folder).
Hey, thanks for the class! It’s always good to bring knowledge to the discussion. I didn’t know the actual reasons behind the CRT 2.35~2.5 gamma. Thanks for the link as well.
So, considering this fact, the artists at the time had no option (not even were aware) but to create games for these devices, which displayed content within the 2.35~2.5 gamma range. In that range, interactions between light and dark pixels result in uneven sizes, where the lighter ones appear larger than the darker ones.
On top of that, the scanlines (more precisely, the scanline gaps, another CRT characteristic) contributed to reduce the spread of lighter pixels in the vertical direction, which result in non-square pixels (it’s not the only reason for that, though, here I’m disregarding the horizontal blending).
All this reasoning is just to give us ideas about what we should try to apply on non-crt shaders so that using gamma 2.4 do not result in blown up pictures as shown in the first post I made.
I’m trying approach 1, shrinking the filter kernel vertically to half size. But, unfortunately, the filtering presents discontinuities, which appear as pixelization. (Using input Gamma 2.4 in both directions). Catmull-Rom below:
EDIT: Made some tests of your approach 1 on gaussian blur and the results are a bit better than what I’ve got with catmull-rom.
Using multipass shaders inside blur folder and applying linearizing gamma before it. First a picture of default (gamma 1.0, radius 5.0 and sigma 0.50 on both dimensions):
It indeed looks like the effect I’m expecting and approximates the results I was getting on my approach. Though the pixelization effect is more pronounced.
EDIT2: for comparison, the same gaussian blur (radius 5.0, sigma 0.50), now using my approach of gamma 1.0 vertical and linear gamma (2.4->2.2) horizontal:
Don’t change sigma. Rather, substitute ~2x in place of x when x is in the vertical direction.
[Edit: You can multiple sigma by some factor to get the same result as multiplying x by some other factor, but the calculation isn’t simple. Further edit: Actually, no. You can’t unless you renormalize the whole thing.]
Please note that I just made up those new outside rows as a quick example. I did NOT calculate the function. So don’t use this example as a real kernel. Also, I didn’t renormalize. Which you need to do. But not renormalizing shows better how the old 2-away row has become the new 1-away row (and a 4-away row becomes the new 2-away row). (I’ll try to edit in an example using real numbers tomorrow when I have time.)
I can’t change the “x” by “2x”, because it’s the same as skipping samples. In a Gaussian Blur, either you change the Sigma or the Radius. You can’t skip a sample. This is what happen if I change “x” by “2x” (skipping samples):
I’ll post here the presets and shaders so that you make the changes you’re thinking. It’s not easy for me to test what I’m trying to understand, so if you have some ideas, it’s better making directly the modifications and see if they work.
Just unzip inside shaders_slang folder and run on Retroarch. It has two presets, one for my approach and the other using half vertical sigma (linear gamma on both dimensions).
I’ve been meaning to reply to this thread for a while. To me, it just seems strange to interpolate/blur vertically in gamma-corrected space and horizontally in linear space. If anything, I would think of it the opposite way, since a composite video signal is encoded and decoded in gamma-corrected space in lines from left to right, and scanlines light up so briefly that any vertical blending could only possibly be the light itself, in linear space. (Besides, according to the above paper about gamma, the non-linear EOTF is caused by the electron gun, not the phosphors.)
This has led me to throw together this laggy tool to evaluate shaders’ handling of dithering. https://www.mediafire.com/file/cp48lj1rzxtfey8/graythingy.zip/file This is a tool that is both prepended and appended to any compatible shader preset. The prepended pass renders solid or dithered colors from 0 to 255 in sequence. The appended pass averages up an entire 384x216 area in the center with gamma taken into account and gradually builds up horizontal lines in pairs to compare the anticipated luminance against the actual result. This entire process assumes that all blending is meant to happen either exclusively in linear space or exclusively in gamma-corrected space, but not both.
The “bumpy” appearance of the lines is caused by floating point error when adding up over 80,000 pixels. The top line is the desired result, and the bottom line is the actual result. The order from top to bottom is: (1) solid color, (2) alternating scanlines, (3) vertical bars, (4) checkerboard, and the other four are are inverted versions of those.
Here’s with horizontal-linear-gamma-only. When alternating consecutive scanlines on and off, the result becomes darker than original hardware.
with crt-guest-advanced, with all RGBs with a checkerboard dithering pattern. The top color is x=0.444, y=0.399 – Notice how it becomes redder when it is dithered.
About gamma, personally, I don’t think debating over 2.4 or 2.5 gamma is entirely worth it, when every CRT has brightness and contrast settings that affect the image even more. Look on page 5 of “The Rehabilitation of Gamma”: The CRT’s true gamma response isn’t controllable, but you are able to control the amounts of current that correspond to the minimum black (brightness / black level) and to the maximum white (contrast / picture, roughly), while everything else is a linear function between those. Because of that, depending on what you pick for brightness and contrast, you’ll get the illusion of different gamma values. Not long ago, I used an X-Rite i1 Display 2 to sample a grayscale from my 13-inch CRT with my own preferred brightness and contrast settings and performed a naive power-regression as in page 5 of The Rehabilitation of Gamma, and the result was somewhere about 2.1. When I turned off the lights in the room, it was clear that my brightness was set too high, even though it always looked perfect when the room was lit. Perhaps its a good idea to be willing to use a different EOTF for each game, since each developer likely had a different EOTF from each other.
(More precisely, red, green, and blue each have their own separate minimum and maximum values which have to be kept in sync with each other, but it’s impossible to do that without some amount of error. On most CRT TVs in the wild, you can see the color temperature shift slightly as gray gets brighter.)
It’s strange to me too, I’d expect gamma would work the same way in both directions as in CRT shaders. But using only in horizontal works better than using linear gamma on both dimensions. Just compare the screenshots earlier.
And, if I only use linear gamma on vertical dimension, I’m sure it’ll look even worse (but I’ll spare you from this view, :P).
Not judging from a theoritical view, it’s just what I see in my practical tests.
Thank god this scheme of only horizontal linear gamma isn’t made to use with alternating scanlines!
linear-plain.slangp is a plain gaussian blur for comparison.
linear-vfactor.slangp reduces vertical blurring strength by computing the weights for vertical neighbors as if they were further away. This largely does what I think you want. Unfortunately, reducing the blur increases the “pixels have square corners” impression, which may not be desirable.
linear-vfactor-sl.slangp both reduces vertical blurring strength by computing the weights for vertical neighbors as if they were further away and blurs in a scanline gap. This does a great job of stopping bright stuff from “overflowing” vertically. The cost, like all scanline shaders (except mine), is that average brightness is reduced.
linear-vfactor-hey-scanlines.slangp swaps the order of horizontal and vertical blurs, leading to those scanline gaps being visible in the final result. (I think this is maybe an artifact of the default buffer type. Not sure.) Not really useful. Except maybe as a competitor to crt potato as a cheap way to create a scanline effect.
Anywho, the general idea here is not limited to Gaussian blurs. Any blending that applies different weights to neighboring pixels according to relative position could be adapted in a similar manner. For algorithms that don’t compute the weight from the position/distance like a Gaussian function, a hardcoded factor could be used based on the weight change for that position/distance for the Gaussian case. (E.g., if the Gaussian weight for some position goes from 0.5 to 0.3 with good parameters for reducing vertical blur strength, then you can use 0.6 as a factor to adjust the weight for that position for other blending algorithms where weights aren’t computed from the position/distance.)
