What does it mean when adding a shader does nothing?

RetroArch Additions Shaders
#1

Having some difficulty sorting this one out. First, my scenario: I’m trying to work out a good solution to the dithered nature of certain MS-DOS games. I eventually determined that tvout.cgp did the best job of masking the dither effects, at the cost of a decent amount of image blur. (Gdept tries to correct dithering without adding blur, but it doesn’t do an adequate job of choosing what and what not to un-dither.) Anyway, from the soft result provided by tvout, I seek to return to an image like what Sabr can generate, so I plug in sabr-v3.0.cg as a third shader pass. And I make sure to Apply Changes.

Now here’s the part that I don’t get. In the case of Sabr, and the majority of the other available shaders (though not all), my application of the shader as an additional pass has no effect. Sabr should be providing quite a dramatic visual modification, even if it’s not necessarily what I hope for. But I get nothing.

The easy assumption is that I have overlooked something important about the way these shaders interact with one another. But I’m not finding it. Sabr is a straightforward shader with no parameters. The .cgp only instructs that the filter be “nearest”, and I make sure to have it so.

Another day, another point of confusion. ;p

#2

SABR and other interpolation shaders, like hqx and xBR, use very specific pixel comparison strategies that can be thrown off by blur. For de-dithering plus smoothing/interpolation, try using one of the existing presets from the dithering directory, like g/mdapt+xbr-hybrid+whatever.

#3

I gave the dithering shaders a shot before. The problem is that they leave too much dithering on the table. In a more idealized scenario with low image complexity, they might work, but not in this case.

shader_comparison_1.jpg589×928 304 KB

Gdept, the better of the several options, can be seen getting confused by the details on the left, and seemingly ignoring the dithering on the man’s face. Correspondingly, Xbr does what it does in the next image and it’s a mess. TVout, as noted before, does a good job masking the dithering but softens (and somewhat darkens) the image, which is why I aim to find a way to process that result to give the false-contouring look that Xbr/Sabr are good at.

Maybe the solution is to find a variant of TVout that kills the dithering but doesn’t actually try to make things look analog?

#4

It looks like the key is setting an explicit 1x scale on the tvout-tweaks pass. After that, the interpolation shaders are okay, though it seems you have to get it pretty damned blurry before it’ll smooth out the dithering.

#5

When I load the TVout preset, tvout-tweaks is already set to 1x. The preset designates the next shader as image-adjustment.cg with a lot of variables. If I eliminate that pass outright, some of the dithering comes back. In any event, Sabr is ignored.

#6

Hmm, yeah, I’m getting some weird, inconsistent behavior using that tvout.cgp preset. If I load the shaders alone, it works. If I use the preset, I can’t manually load other passes and get the effects to combine.

Try working from this preset instead (just save it in your shaders_cg directory as whatever.cgp):

shaders = "2" shader0 = "crt\shaders\tvout-tweaks.cg" filter_linear0 = "false" wrap_mode0 = "clamp_to_border" mipmap_input0 = "false" alias0 = "" float_framebuffer0 = "false" srgb_framebuffer0 = "false" scale_type_x0 = "source" scale_x0 = "1.000000" scale_type_y0 = "source" scale_y0 = "1.000000" shader1 = "xbrz\shaders\4xbrz.cg" filter_linear1 = "false" wrap_mode1 = "clamp_to_border" mipmap_input1 = "false" alias1 = "" float_framebuffer1 = "false" srgb_framebuffer1 = "false" parameters = "TVOUT_RESOLUTION;TVOUT_COMPOSITE_CONNECTION;TVOUT_TV_COLOR_LEVELS;TVOUT_RESOLUTION_Y;TVOUT_RESOLUTION_I;TVOUT_RESOLUTION_Q" TVOUT_RESOLUTION = "256.000000" TVOUT_COMPOSITE_CONNECTION = "0.000000" TVOUT_TV_COLOR_LEVELS = "0.000000" TVOUT_RESOLUTION_Y = "256.000000" TVOUT_RESOLUTION_I = "83.199997" TVOUT_RESOLUTION_Q = "25.600000"

#7

Better, yes. The missing image-adjustment.cg still results in some dithering being present but I see that Xbr (and presumably Sabr) is being permitted to function in this case. I think you’re onto something. The inconsistency of the results is what has been throwing me for a loop. I’ll follow the outline in your provided text and try to re-inject the image-adjustment and see how it goes.

#8

Success. Also with Sabr.

Subsequent attempts to edit the sequence of shader passes has led me to conclude that the inconsistency of results arises from the phenomenon whereby adding a new shader most often breaks the preceding one, unless done strictly via editing a text file. The breaking of a preceding pass can still happen if any part of a given pass fails for whatever reason; it breaks that shader plus the one before it.

Currently seeing what I can do to sharpen the image after the two TVout passes, before reaching Sabr. It’s too soft. ;p

#9

Here’s the finalized result.

antidither_comparison_1.gif1440×1080 901 KB

I will probably try to return to this later, to see if I can reduce the blurring that’s going on in tvout-tweaks. Still not satisfied with how much there is. There’s also some inconsistency in the roof tiles on the left that I’ll want to sort out. But overall it’s a much better result than any of the filter presets by themselves.

#10

that looks good. You could try putting a pass of ‘adaptive-sharpen’ in between tvout-tweaks and sabr to see if that sharpens it up any.

whenever you get it all worked out, post your preset here and I’ll add it to the repos.

#11

You could try putting a pass of ‘adaptive-sharpen’ in between tvout-tweaks and sabr to see if that sharpens it up any.

Exactly what I did. And it did help. But any real progress would need to start with tvout-tweaks. I also noticed that the hues of solid colors are being arbitrarily adjusted (like the solid blue water in the upper left), so at the very least I intend to go back and prevent that from happening.