Dogway's grading shader (slang)

[Dogway]

Thanks! Yes I refined some settings so there’s more granularity or goes from negative to positive as for me that’s easier to grasp. EDIT: srgb_framebuffer to false, we are doing gamma inside the shader.

Dullnes/Vibrance is actually a mask for Saturation. It won’t do anything if that’s 0. In my opinion this is a better solution because it also allows you to desaturate low saturated areas (akin bleach bypass if used with contrast), or saturate even more saturated areas. More control in less settings, think of it as a 2D plot where Y is Saturation and X is the saturation mask.

The issue with the chroma banding (and partly the overexposition) is due to the Brightness setting, despite working in 16-bit half float. I integrated a rolled-off gain for Brightness so it works in a gradual manner in the high range. The code is dirty as it is but it works fine. I also implemented a Hotspot fix to bring down some overexposed highlights and recover some shading in those areas, but I’m not happy at all with the code. You can toggle that off though.

Another big issue I was having was posterization or banding in the low range. I thought this was a quantization side-effect of doing several gamma transforms in chain, and although I optimized how the transforms are taking place (check the presets below) the real culprit was crt-royale beam-min-sigma, by default 0.02 it was darkening the image, set to 0.10 and clipping goes away (EDIT: As I found out, only an issue with > 240px input sizes). Still, I really would like if the ntsc passes would work in linear space (seemingly they are designed with gamma encoded inputs), because it could save us 2 transformations that translates to better performance and less quantization.

After an advice from hunterk I also modified the variable names. As usual updates in the repo. I’m also sharing the full presets with recommended default settings because it’s not easy stuff, for both slang and glsl and 240 and 320 inputs.

Dogway/emulation-random

[Dogway]

OMG, it was the atan() function in include\compat_macros.inc, I completely overlooked that since it’s not in the crt folder and my diff comparison software skipping minor changes. Finally after all these years I can get rid of glsl at last, thanks for making me look twice.

[hunterk]

oh lol yep, I felt the same way when I figured out what was wrong… What a dumb problem to have caused us so much grief.

[Syh]

@Dogway

Man I feel dumb, I read your post about the dull/vibrance setting when I was half asleep, and was so confused. Re-read it today, looked at the graph and was fuuuuuu that’s what it does, lol. (I mean damn, you drew me a picture and everything, I’m not stupid I swear, rofl.)

You’re grade shader is great man, the white point lum preserve does alot for the image.

I need to update the copy you did without the LUTs to current, which I can most certainly do I’ll just cherry pick the update from the repo.

I did make two alterations to the shader though, I separated the saturation into separate RGB saturation(this may slightly be pointless as the dull/vibrance is a thing, but I like that grainual control), and added the X/Y Modifier’s from image-adjustment (mainly so I don’t need image-adjustment for it, lol.)

All in all this is my favorite video management shader.

[Syh]

@Dogway

Jesus, you’ve been busy over the last couple of days.

Decided I was going to start cherry picking that update for grade, opened up your repo and you went ham on updating grade. Also interesting update for the signal-bandwidth shader, will have to check it out.

[Dogway]

oh well, many are cosmetics. I mainly implemented corner size since crt-royale doesn’t have that one. Ideally this fits better into image-adjustment (flip, mirror, zoom, translate, etc) but no way I’m going into that territory haha. The other thing was slang versions for lut_x2, gdapt stripes and signal-bandwidth.

BTW, I forgot to say that signal bandwidth for the ntsc passes is emulated through hardware resizing.

Those:

scale_type_x1 = "source"
scale_x1 = "4.000000"
scale_type_x2 = "source"
scale_x2 = "0.500000"

I’m not sure if this is a requirement for ntsc to work or an option, in which case I would replace that “hack” with signal-bandwidth. I guess after the ntsc passes.

[Syh]

@Dogway

image-adjustment (flip, mirror, zoom, translate, etc)

That’s cake to do personally.

I haven’t updated grade since you posted the non-LUT version of it for me.

Could you elaborate more on this?

@Dogway

BTW, I forgot to say that signal bandwidth for the ntsc passes is emulated through hardware resizing.

Those:

scale_type_x1 = "source"
scale_x1 = "4.000000"
scale_type_x2 = "source"
scale_x2 = "0.500000"

I’m not sure if this is a requirement for ntsc to work or an option, in which case I would replace that “hack” with signal-bandwidth. I guess after the ntsc passes.

[Dogway]

Yes, well, I really don’t want to mess with my personal presets anymore. I personally don’t use image-adjustment but I guess people do.

On the ntsc bit, if you inspect ntsc-320px-svideo.glslp:

shaders = 2
shader0 = shaders/ntsc-pass1-svideo-2phase.glsl
shader1 = shaders/ntsc-pass2-2phase-gamma.glsl

filter_linear0 = false
filter_linear1 = false

scale_type_x0 = absolute 
scale_type_y0 = source
scale_x0 = 1280
scale_y0 = 1.0
frame_count_mod0 = 2
float_framebuffer0 = true

scale_type1 = source
scale_x1 = 0.5
scale_y1 = 1.0

You find that the image is upscaled to 1280 pixels wide before the ntsc (blur) pass. If you upscale more the blurring will be less prominent. So this is a non accurate way of simulating signal-bandwidth (my interpretation).

[Syh]

You can use this method for the first pass of ntsc, I’m pretty as @ProfessorBraun was discussing it in the CRT shader show-off thread. (It’s about 100-200 posts back.)

scale_type_x1 = "source"
scale_x1 = "4.000000"
scale_type_x2 = "source"
scale_x2 = "0.500000"

Ohh I was saying you can pull that stuff out of image-adjustment and place it into grade. (Already have done it myself and then removed the stuff I didn’t need.) Most of it happens in an earlier part of the shader before the void main()

Not sure if you were saying you didn’t want to add image-adjustment as a pass (to the shader preset chain), or that you didn’t want to add any code from image-adjustment.

Personally the only thing I kept from image-adjustment that I ported over into grade was the X/Y Modifiers.

Regardless, look forward to messing with the new grade update, and checking out your updates to the signal bandwidth shader.

[Nesguy]

I’m a bit late to the party. Is there a write-up or summary describing the new shaders and how they improve on existing shaders? Looks like good stuff from what I’ve seen but info seems a bit scattered.

[Dogway]

Basically it’s on the shader header, it’s an ubershader that comprises a set of existant shaders to work better in conjunction and ease of use. But I also included new code.

It’s all color/tone related that’s why “grade”. I implemented color-mangler, vignette, lut_x2 and white_point, and added vibrance, black level, corner size, rolled gain, sigmoidal contrast and proper gamma transforms.

This should be you first shader on the stack related to all color work, and next you can do ntsc emulation, blurring, scanlines…

If you can test and give some feedback that would be very welcome since I consider it pretty much done. The only thing I don’t like (and hence default to false) is the HotSpot fix. I need to come up with something better once I have time.

[Syh]

Could you elaborate on the sigmoidal contrast and proper gamma transformation?

Like what is the difference with sigmoidal contrast, and what is different with how you’re doing gamma vs how it was originally handled.

[Dogway]

TV sets (and therefore many shaders here) consider “contrast” as black lift (in color grading terms).

For me that’s Lift (called Black Levels in grade.glsl). In the pic you can also see Gain, people commonly call that luminance or brightness. I kept the name Brightness but instead of a straight line I applied a soft curve (roll-off) towards the end, so highlights are not harshly clipped but gently.

In comparison, sigmoidal contrast is a non destructive way of increasing contrast by drawing an S curve shape on the function transform. What you get is brighter brights and darker darks (or the inverse) without clipping.

On gamma, after some research I read that while CRTs employed a pure power gamma LCDs abid to sRGB gamma, they are different, sRGB gamma has a linear part on the low end which are better at avoiding crushing the blacks. This was my first attempt but the blacks kept being crushed none-the-less due to the initial pure power linearization. I decided against literature to use sRGB gamma also for the CRT part, after some adjustement on the values what you get is the same image but with better dynamic range (brighter if you like) low range. You can check this on a dark frame real time, toggle Gamma Type on and off and decide which one do you prefer. Most shaders on retroarch use pure power gamma.

[Syh]

@Dogway

Hey, I don’t know if it’s my setup specifically or what, but the black_rgb tint settings seem pointless. If any of them go to 0.01 it immediately makes black that color, I know that is sorta it’s purpose but it’s fairly drastic(like too strong too fast to be of use imo).

You move them in tandem and you get a way stronger version of the black level setting, that’s not anywhere as nice imo.

I mean it’s your decision, but you may want to review the black tint settings and see if you want to phase them out, black lift (level) is great though, the tints are just weird.

I don’t want you to think I’m complaining, as this is an issue with color-mangler as well. I just thought you’d be interested in checking it out.

Shaders great regardless, lol.

[Dogway]

It’s color-mangler behavior untouched. I agree stepping is too high so I refined it, thanks for the observation.

[Syh]

Yeahhh, I knew it was a color mangler issue. Just thought you didn’t really use that setting (possibly being unaware of this), and this was something you’d care about (the ability to fine tune the color profile).

Black RGB tint is no longer useless, lol.

Thanks for the fix!

[Syh]

Alright after going over this thread again I have a question, not really pressing or anything.

The hotspot fix, you said it was to fix overexposed highlights, is it basically a clipping fix for highlights? (Because that’s what it sounds like to me, I’m not super informed about color management so that’s how my brain is interpreting overexposed highlights fix.)

I mean I also feel my explanation is an oversimplification of things as overexposure doesn’t necessarily mean clipping is involved, it just means those colors are kind of getting less contrast between them, idfk. (Also feel this isn’t right either, the contrast thing, not the oversimplification bit.)

Hope you can clarify for me when have the time.

[Dogway]

I don’t know the real reason but comparing my old default presets against my new presets (which should be more correct) I get more hotspots or overblown highlights. I don’t know why, it’s not grade as it is as transparent as it can be (as default), it happens at the scanline stage.

I think my old presets lowered the brightness in pre-scanline shaders but I have to dig deeper on where it happens. Anyway at worst you can use my Brightness setting which does a rolled gain (good) even for negative values, but a better solution is to don’t touch whites and recover shading in highlights so I need to design a curve specifically for that. Negative contrast is a good start but I have to limit it to highlights only, I think a luma mask can cut it.

EDIT: CRTs weren’t perfect machines as power is lost through the signal. I shouldn’t be surprised if lowered gain is a more correct approach (which later scanlines and bloom would correct).

[Dogway]

So I run some tests, I grabbed an old shot as reference, then the new preset with grade, and following the fix (color adjustments) I did just now.

What I did mostly was lower saturation, lower gain, enable hotspot fix and lower white-green. I also added some green to reds to remove that blown pure red. As usual I also add green to blues, this makes skies look better (Azure blue), or Sonic color more natural.

This is something I would think ntsc emulation would suffix but it doesn’t, so there’s something else going on. I tried using the Triniton LUT but it’s not working fine on all systems and I don’t know why. It’s the “Sony Trinitron Std D50 - Edit 1 - Gamma Unadjusted -32px”. I interpret that as a linear 32 points LUT, it changes the white point to D50 (from D65) I guess.

Reference

crt-royale+grade

crt-royale+grade (color fix)

*Screenshots have been converted from Display profile to sRGB so it’s not 100% accurate on how I see colors here, but better than if I didn’t convert.

gamma_out = "2.200000"
gamma_in = "2.400000"
gamma_type = "1.000000"
g_vignette = "1.000000"
g_vstr = "40.000000"
g_vpower = "0.250000"
g_csize = "0.000000"
g_bsize = "600.000000"
LUT_Size1 = "32.000000"
LUT1_toggle = "0.000000"
LUT_Size2 = "64.000000"
LUT2_toggle = "0.000000"
wp_temperature = "9311.000000"
g_sat = "-0.020000"
g_vibr = "0.000000"
g_hpfix = "1.000000"
g_lum = "-0.050000"
g_cntrst = "0.000000"
g_mid = "0.500000"
g_lift = "0.040000"
blr = "0.000000"
blg = "0.000000"
blb = "0.000000"
wlr = "1.000000"
wlg = "0.850000"
wlb = "1.000000"
rg = "0.020000"
rb = "0.000000"
gr = "0.000000"
gb = "0.000000"
br = "0.000000"
bg = "0.050000"

[Nesguy]

@Dogway

Is there somewhere I can grab SLANG versions of your shaders (Grade, NTSC, signal bandwidth)? They seem kind of scattered across multiple forum posts and I want to make sure I’ve got the most current versions. Do you still need to do additional testing before adding these to the repo? In case you can’t tell, I’m eager to add these to my shader presets.