Objective image quality when using shaders

[Nesguy]

These are the best settings for dynamic range, which is probably the most important factor for objective image quality. If anyone is able to increase the dynamic range further, please let me know. This is the result of a lot of rigorous testing of every single CRT shader available.

These settings are intended to be used with the LCD backlight maxed out. This gives a big boost to contrast and results in the greatest possible dynamic range. This is the only way to compensate for the lost brightness from scanlines and mask effects without a significant alteration of the original color values and/or introducing clipping. Also, take note that these settings are intended for 1080p and a 5x vertical scale. Other scales/resolutions haven’t been thoroughly tested and may lead to sub-optimal results.

As an alternative to the Lottes shadow mask, try the guest-dr-venom slotmask. Set “CRT mask” to -1.00, “slot mask width” to 3.00 and “slot mask strength” as high as you can while still maintaining adequate brightness and vibrancy. “Slot mask width” should be set to higher values for resolutions higher than 1080p, but you should use whole integer values to keep the phosphors a uniform size.

There are a lot of things to adjust within Dogway’s grade shader that will depend on display conditions, preferences and whether you’re going for a specific CRT gamut.

Be aware that some settings in Dogway’s grade shader need to be adjusted on a per-game basis for optimum results. Check out the shader notes within the shader file for Dogway’s suggested settings for accuracy. Of course, these are just suggestions and you can adjust everything to personal preference and display conditions. Some games will require additional adjustments even after applying the suggesting settings. For example, some of the fine details in Outrun look better with slightly lowered brightness (-0.02). There are a lot of ways to achieve the same or similar results, just play around with the settings until you’re familiar with what they do and tweak away

shaders = "10"
shader0 = "shaders_slang/stock.slang"
filter_linear0 = "false"
wrap_mode0 = "clamp_to_border"
mipmap_input0 = "false"
alias0 = ""
float_framebuffer0 = "false"
srgb_framebuffer0 = "false"
scale_type_x0 = "source"
scale_x0 = "1.000000"
scale_type_y0 = "source"
scale_y0 = "1.000000"
shader1 = "shaders_slang/stock.slang"
filter_linear1 = "true"
wrap_mode1 = "clamp_to_border"
mipmap_input1 = "false"
alias1 = ""
float_framebuffer1 = "false"
srgb_framebuffer1 = "false"
scale_type_x1 = "source"
scale_x1 = "1.000000"
scale_type_y1 = "source"
scale_y1 = "1.000000"
shader2 = "shaders_slang/misc/grade.slang"
filter_linear2 = "true"
wrap_mode2 = "clamp_to_border"
mipmap_input2 = "false"
alias2 = "WhitePointPass"
float_framebuffer2 = "false"
srgb_framebuffer2 = "false"
scale_type_x2 = "source"
scale_x2 = "1.000000"
scale_type_y2 = "source"
scale_y2 = "1.000000"
shader3 = "shaders_slang/crt/shaders/guest/afterglow.slang"
filter_linear3 = "true"
wrap_mode3 = "clamp_to_border"
mipmap_input3 = "false"
alias3 = "AfterglowPass"
float_framebuffer3 = "false"
srgb_framebuffer3 = "false"
scale_type_x3 = "source"
scale_x3 = "1.000000"
scale_type_y3 = "source"
scale_y3 = "1.000000"
shader4 = "shaders_slang/crt/shaders/guest/avg-lum.slang"
filter_linear4 = "true"
wrap_mode4 = "clamp_to_border"
mipmap_input4 = "true"
alias4 = "AvgLumPass"
float_framebuffer4 = "true"
srgb_framebuffer4 = "false"
scale_type_x4 = "source"
scale_x4 = "1.000000"
scale_type_y4 = "source"
scale_y4 = "1.000000"
shader5 = "shaders_slang/crt/shaders/guest/linearize.slang"
filter_linear5 = "true"
wrap_mode5 = "clamp_to_border"
mipmap_input5 = "false"
alias5 = "LinearizePass"
float_framebuffer5 = "true"
srgb_framebuffer5 = "false"
scale_type_x5 = "source"
scale_x5 = "1.000000"
scale_type_y5 = "source"
scale_y5 = "1.000000"
shader6 = "shaders_slang/crt/shaders/guest/blur_horiz.slang"
filter_linear6 = "true"
wrap_mode6 = "clamp_to_border"
mipmap_input6 = "false"
alias6 = ""
float_framebuffer6 = "true"
srgb_framebuffer6 = "false"
scale_type_x6 = "source"
scale_x6 = "1.000000"
scale_type_y6 = "source"
scale_y6 = "1.000000"
shader7 = "shaders_slang/crt/shaders/guest/blur_vert.slang"
filter_linear7 = "true"
wrap_mode7 = "clamp_to_border"
mipmap_input7 = "false"
alias7 = "GlowPass"
float_framebuffer7 = "true"
srgb_framebuffer7 = "false"
scale_type_x7 = "source"
scale_x7 = "1.000000"
scale_type_y7 = "source"
scale_y7 = "1.000000"
shader8 = "shaders_slang/crt/shaders/guest/linearize_scanlines.slang"
filter_linear8 = "true"
wrap_mode8 = "clamp_to_border"
mipmap_input8 = "false"
alias8 = ""
float_framebuffer8 = "true"
srgb_framebuffer8 = "false"
scale_type_x8 = "source"
scale_x8 = "1.000000"
scale_type_y8 = "source"
scale_y8 = "1.000000"
shader9 = "shaders_slang/crt/shaders/guest/crt-guest-dr-venom.slang"
filter_linear9 = "true"
wrap_mode9 = "clamp_to_border"
mipmap_input9 = "false"
alias9 = ""
float_framebuffer9 = "false"
srgb_framebuffer9 = "false"
scale_type_x9 = "viewport"
scale_x9 = "1.000000"
scale_type_y9 = "viewport"
scale_y9 = "1.000000"
parameters = "g_gamma_in;g_signal_type;g_gamma_type;g_crtgamut;g_space_out;g_hue_degrees;g_I_SHIFT;g_Q_SHIFT;g_I_MUL;g_Q_MUL;g_lum_fix;g_vignette;g_vstr;g_vpower;g_lum;g_cntrst;g_mid;wp_temperature;g_sat;g_satr;g_satg;g_satb;g_vibr;g_lift;blr;blg;blb;wlr;wlg;wlb;rg;rb;gr;gb;br;bg;LUT_Size1;LUT1_toggle;LUT_Size2;LUT2_toggle;SW;AR;PR;AG;PG;AB;PB;sat;lsmooth;GAMMA_INPUT;TAPSH;GLOW_FALLOFF_H;TAPSV;GLOW_FALLOFF_V;TATE;IOS;OS;BLOOM;brightboost;brightboost1;gsl;scanline1;scanline2;beam_min;beam_max;beam_size;h_sharp;s_sharp;csize;bsize;warpX;warpY;glow;shadowMask;masksize;vertmask;slotmask;slotwidth;double_slot;slotms;mcut;maskDark;maskLight;CGWG;gamma_out;spike;inter;interm;bloom;scans"
g_gamma_in = "2.500000"
g_signal_type = "0.000000"
g_gamma_type = "0.000000"
g_crtgamut = "0.000000"
g_space_out = "0.000000"
g_hue_degrees = "0.000000"
g_I_SHIFT = "0.000000"
g_Q_SHIFT = "0.000000"
g_I_MUL = "1.000000"
g_Q_MUL = "1.000000"
g_lum_fix = "0.000000"
g_vignette = "0.000000"
g_vstr = "40.000000"
g_vpower = "0.200000"
g_lum = "0.000000"
g_cntrst = "0.000000"
g_mid = "0.500000"
wp_temperature = "5505.000000"
g_sat = "0.000000"
g_satr = "0.000000"
g_satg = "0.000000"
g_satb = "0.000000"
g_vibr = "0.000000"
g_lift = "0.000000"
blr = "0.000000"
blg = "0.000000"
blb = "0.000000"
wlr = "1.000000"
wlg = "1.000000"
wlb = "1.000000"
rg = "0.000000"
rb = "0.000000"
gr = "0.000000"
gb = "0.000000"
br = "0.000000"
bg = "-0.005000"
LUT_Size1 = "16.000000"
LUT1_toggle = "0.000000"
LUT_Size2 = "64.000000"
LUT2_toggle = "0.000000"
SW = "1.000000"
AR = "0.070000"
PR = "0.050000"
AG = "0.070000"
PG = "0.050000"
AB = "0.070000"
PB = "0.050000"
sat = "0.100000"
lsmooth = "0.900000"
GAMMA_INPUT = "2.400000"
TAPSH = "4.000000"
GLOW_FALLOFF_H = "0.300000"
TAPSV = "4.000000"
GLOW_FALLOFF_V = "0.300000"
TATE = "0.000000"
IOS = "0.000000"
OS = "1.000000"
BLOOM = "0.000000"
brightboost = "0.500000"
brightboost1 = "1.500000"
gsl = "2.000000"
scanline1 = "15.000000"
scanline2 = "5.000000"
beam_min = "1.000000"
beam_max = "0.850000"
beam_size = "0.000000"
h_sharp = "3.000000"
s_sharp = "1.000000"
csize = "0.000000"
bsize = "600.000000"
warpX = "0.000000"
warpY = "0.000000"
glow = "0.000000"
shadowMask = "3.000000"
masksize = "1.000000"
vertmask = "0.000000"
slotmask = "0.000000"
slotwidth = "2.000000"
double_slot = "1.000000"
slotms = "1.000000"
mcut = "0.250000"
maskDark = "0.500000"
maskLight = "1.500000"
CGWG = "0.300000"
gamma_out = "2.400000"
spike = "1.000000"
inter = "400.000000"
interm = "1.000000"
bloom = "0.000000"
scans = "1.000000"
textures = "SamplerLUT1;SamplerLUT2;SamplerLUT3"
SamplerLUT1 = "shaders_slang/crt/shaders/guest/lut/sony_trinitron1.png"
SamplerLUT1_linear = "true"
SamplerLUT1_wrap_mode = "clamp_to_border"
SamplerLUT1_mipmap = "false"
SamplerLUT2 = "shaders_slang/crt/shaders/guest/lut/sony_trinitron2.png"
SamplerLUT2_linear = "true"
SamplerLUT2_wrap_mode = "clamp_to_border"
SamplerLUT2_mipmap = "false"
SamplerLUT3 = "shaders_slang/crt/shaders/guest/lut/other1.png"
SamplerLUT3_linear = "true"
SamplerLUT3_wrap_mode = "clamp_to_border"
SamplerLUT3_mipmap = "false"

[rafan]

@NesGuy I think we got a new challenge to add to our “test suite”.

This is about contrast. Check out the clouds in Arcade Out Run, they should be “real” clouds, with depth and bulges to the clouds. Check the wikipedia Out Run picture below and you see the grey areas in the clouds providing the depth. If you look closely there are A LOT of subtle grey gradients that provide the depth. I imagine on a CRT this depth is much more pronounced even.

Now fire up Out Run in MAME RA, using your shader preset, and… the depth to the clouds is gone.

Only a few subtle grey patches, but for the remainder it’s quite flat looking. Tweaking the parameters does not help.

Now press “p” (it’s the pause button in MAME, works in RA), which halves the brightness, and voila, the depth to the clouds is back!(?!)

I didn’t have luck by tweaking parameters, so this may be a shader thing. Now I noticed two things:

1. This issue of “flatness” in the clouds is worse when using the grade shader included. The vanilla guest-venom shader has more contrast/depth in the clouds.

2. Interesting enough if you enable the “sega luma fix” the clouds become completely flat, no contrast at all anymore. Just plain white without depth.Which makes you think we would actually need an “inverse sega luma fix” to have the clouds show proper depth.

Would be nice to see what’s the underlying issue, as this is definitely a deterioration of objective image quality when using (grade included) shaders.

Tagging @Dogway too, because the correlation with using the sega luma fix making the issue much worse may provide a clue to what’s happening under the shader hood…

[Nesguy]

Hey, good stuff! Haven’t had time to read all of this yet, but can you please provide a link to the image source?

[rafan]

You can do right mouse button over the picture and copy the image location URL (browser feature).

If that doesn’t work (remove spaces from h t t p s, other wise forum automagically inserts the picture):

h t t p s://upload.wikimedia.org/wikipedia/en/e/e1/Out_Run_screenshot.png

[Dogway]

Sega Luma Fix is for a few systems but not MAME, namely: Genesis, 32x, CD and Saturn 2D. And the actual purpose is to maximize the dynamic range in the whites since in source it’s compressed range (value of 239 white).

I had once some type of hotspot fix that I removed, but it would be like an inverse curve for the brights, it needs some crafting so I didn’t find the time.

[rafan]

I understand, I don’t use Sega Luma Fix on Arcade. I was just trying all the shader parameters (randomly) to see if they change anything to the better, and then found the sega luma fix actually having an impact to the opposite, so I thought it might provide a clue. I.e. doing the “opposite” to what the sega luma fix is doing, could possibly improve the clouds in Out Run…

@Dogway

I had once some type of hotspot fix that I removed, but it would be like an inverse curve for the brights, it needs some crafting so I didn’t find the time.

That would be nice. Didn’t think much of the clouds in Out Run previously, but now that I noticed it seems like a thing that could improve objective image quality.

[Nesguy]

@rafan

It’ll be a while before I can look at this, need to download Mame roms.

[rafan]

No worries, things look great as they are with the current shader preset.

Still using your guest-venom preset with slotmask and grade shader included and for the first time in years I’ve actually been playing the games more than tweaking shader settings

[Nesguy]

Glad you like it! I feel the same way, I can finally stop tweaking settings and actually play some games, lol.

I’ve been playing around with adding some glow (glow strength 0.02), but I’m kind of on the fence about it. If your eyes aren’t playing tricks on you with the scanlines (like mine are) then I would leave it at 0.00 unless you like the effect.

CRT gamma is another thing to play around with, of course, being a per-game thing.

I still need to put CRT Royale, guest-sm and CRT Lottes through the gauntlet, but I don’t think Lottes is going to hold up since it’s a fairly old shader and not quite as versatile.

On MAME, have you examined the core options? I have very little experience with MAME, just wondering if maybe there’s something going on there.

[rafan]

@Nesguy

I still need to put CRT Royale, guest-sm and CRT Lottes through the gauntlet, but I don’t think Lottes is going to hold up since it’s a fairly old shader and not quite as versatile.

On MAME, have you examined the core options? I have very little experience with MAME, just wondering if maybe there’s something going on there.

Will be interesting to see your findings with the other shaders. I went trough the core options in RA’s MAME core, but there’s nothing of interest there. When you disable the shader you see more of the subtle grey coloring appearing in the clouds, so I’m guessing this is a shader thing. With the shader enabled the lightest shades of grey seem to disappear. As said, I’m noticing a difference between vanilla guest-venom and the one with grade shader included, where the vanilla one seems less (or not?) affected.

[Nesguy]

Hmm, that would point to something going on with grade, then. Maybe there’s some kind of MAME-specific fix that’s needed (like the Sega luma fix for Sega systems). Could also be something to do with Sega’s arcade hardware. Have you tried Outrun on other systems? Unfortunately this MAME download failed so I’m still waiting on that

[hunterk]

Cannonball may be a good one to check, though I would guess it acts the same as MAME.

[rafan]

@Nesguy

Unfortunately this MAME download failed so I’m still waiting on that

What’s failing on you, it’s less than 1 meg? If I may suggest you should look to elgoog piz.nurtuo . Cheer up

[Syh]

I’m pretty sure he’s referring to the romset.

[rafan]

He doesn’t need the full romset, right?

[rafan]

@hunterk

Cannonball may be a good one to check, though I would guess it acts the same as MAME.

I tried Cannonball and the shader effects are the same as with MAME outrun, clouds are flat when using grade.

Never tried Cannonball before, but it’s nice! Thx for pointing it out. Played a few levels, it’s a bit easier though than MAME outrun for some reason. Normally I would come out of level 1 with 12s, but on first race with CannonbalI engine I entered second stage with 16s left… Plays very nice though…

[hunterk]

The big draw is that it can do true 60 fps (and some stuff up to 120 fps) along with native widescreen

So, you’re getting the cloud-flattening just with ‘grade’ or with bare CRT presets, as well?

[Syh]

You should pm him about it , lol.

But yeah if he’s just testing outrun I don’t see the need for the whole thing. It’s like 60-ish GB last time I checked. (Don’t have it myself, use fbneo with only the stuff I actually want for it, for my arcade needs lol.)

That being said there’s a few Mame only games I’d love to have though. (If they happened to find their way to my PM’s I wouldn’t be mad about it the slightest, .)

[Nesguy]

Oh, I’m just an obsessive completionist and expected this download to be done this morning. I have full romsets for everything else so I wanted to do the same for MAME. I’ll DL Outrun when I’m back at my computer if the romset isn’t done by then.

[BendBombBoom]

If I put the scanline bright way down and the scanline dark way up (With scanlines type at 2) I get shaded clouds. I’m quite sure this would cause problems in other games. The first shot is from venom and the second is venom with the grading shader on top. If you adjust the contrast in the grading shader you can exaggerate the amount of depth in the clouds.

We need a shader that is more dynamic with the color and luminosity range. Either I make many custom shaders for lots of scenarios or I have one shader and deal with losing things at the ends of darks and lights.