Mdapt & gdapt - dithering treatment [Updated 06/06/14]

RetroArch Additions Shaders
#201

Yes, I started using bsnes-balanced since accuracy uses double res or something (not sure why but…) I tried to stack a scanline filter after this but renders bad whatever the res I set it to. I tried to stack a stock to x4 but didn’t make the trick either. Let’s see, I’m trying to decide what filter to use prior to scanlines to smooth things better.

edit: Got it, used stock at x1, then scanline at x4:

shaders = 8

shader0 = ../../stock.cg
filter_linear0 = true
scale_type_x0 = source
scale_x0 = 0.5

shader1 = passes/mdapt-pass0.cg
filter_linear1 = false
scale_type1 = source
scale1 = 1.0

shader2 = passes/mdapt-pass1.cg
filter_linear2 = false
scale_type2 = source
scale2 = 1.0

shader3 = passes/mdapt-pass2.cg
filter_linear3 = false
scale_type3 = source
scale3 = 1.0

shader4 = passes/mdapt-pass3.cg
filter_linear4 = false
scale_type4 = source
scale4 = 1.0

shader5 = passes/mdapt-pass4.cg
filter_linear5 = false
scale_type5 = source
scale5 = 1.0

shader6 = ../../stock.cg
filter_linear6 = true
scale_type6 = source
scale5 = 1.0

shader7 = "03. crt-hyllian.v16 (noring).cg"
filter_linear7 = false
scale_type7 = source
scale7 = 4.0
#202

I don’t think that’s necessary. you probably made a mistake in your cgp. at least in the one you posted you have the line “scale5 = 1.0” two times.

#203

I think that if I set scale_type7 to viewport I don’t need the stock nor the x4 resolution.

#204

I ended up using the snes-hires-blend.cg shader (because I found others too aggressive), for this game and some more, at the moment I only applied it to Jurassic Park, Kirby Dreamland 3 and Secret of Mana 2 and 3. Do you know of any more snes games with hires effect?

I also applied to all the mega drive, mega CD and 32x games. Do you think this is ok, or the use of hires effect in the sega consoles is marginal? For example Comix Zone is heavy on hires effect. I don’t care that much for the dithering at this point since with scanlines, movement and the own LCD motion blur do the effect.

#205

These shaders are really amazing with Genesis games, restoring the transparency effects or smooth gradations without blurring the image. Are they supposed to work with Super Nintendo stuff at all?

While dithering is a lot less common, there are still some games which use it. gdapt/mdapt don’t seem to detect anything.

Secret of Mana:

Blurred to show transparency:

Super Metroid:

Blurred to show the smooth gradients on text:

#206

if it’s a strict checkerboard pattern it will be detected (which is not the case for secret of mana). with the SNES system you must pay attention to which core you use for that. a few games are using the high resolution mode of the snes system for things like hires fonts. different cores are handling it in different ways. normal pixels are doubled this way which breaks the detection of the algorithm. see the above discussion with me and dogway for that and the solution.

here I zoomed in your pictures and added a grid to make it more clear:

Secret of Mana (no checkerboard): https://i.imgur.com/QkDLP8y.png

Super Metroid (checkarboard): https://i.imgur.com/NFX1b26.png

if the algorithm doesn’t detect the pattern in Super Metroid, you probably need to lower the threshold values inside of the shader options menu of retroarch.

#207

Well I wondered if that 1x2 checkerboard would be something you could detect, rather than only 1x1 checkerboards. Perhaps as a third type?

I tried adjusting all the thresholds and there didn’t seem to be any effect, which is why I wondered if it applied to SNES games at all. (using the bSNES Accuracy core) Edit: reading above, apparently it does not work with the accuracy core. :frowning:

I was also wondering if it might be possible to have the shader detect whether something is supposed to be a “box”? I have no idea if this is an impossible thing to ask.

There seems to be some difficulty with Elevator Action Returns’ Caution box for example. (fuzzy edges and missing areas)

I’ve noticed similar things on health bars in other games. I tried changing the thresholds but didn’t really get anywhere. It’s difficult in RetroArch because the menu seems to cover up most of the image.

#208

there are many different variants of dithering patterns. mdapt is really specialized on this one type. if one would add more to this algorithm it would lead to too many false detections. that’s the problem with the whole dithering detection. locally you can’t decide if it’s supposed to be details / parts of a structure or dithering.

regarding snes. it does work with the bsnes accuracy core, see this post for the correct shader combination: viewtopic.php?f=6&t=493&start=175#p14828

the fuzzy border is expected behavior. I could easily modify it so that a pixel is regarded as dithering if a minimum number of neighbors is marked. but again that would make the parts where the shader already makes errors even more apparent. so I decided against it in my endless test runs with different cases. the missing areas are simply a case of a too high threshold, but this is a tradeoff. lower the value and you get more false detections, increase it and some dithering parts are left unmerged. there is no perfect value.

also make sure you’re using the latest version from the shader repository on github. the shader versions which come with retroarch are not always up to date. I can say at least that a new version of mdapt is already up my sleeve :wink: but don’t expect this specialized dithering detection at any time to be error free. it lies in the nature of this approach. in general it’s recommend to hide the errors by using linear filtering or adding other scaling shaders on top of it. more general techniques to merge dithering are CRT/NTSC-shaders due to color bleeding and scanline emulation, check’em out!

https://github.com/libretro/common-shaders

#209

I forgot to mention before, but I another elegant way to get rid of dithering is using jinc2-sharp.cg (not “sharper”) as a first shader at 1x. Then you can put after that any other shader to upscale the image. The algorithm of jinc2-sharp has this side effect of cancel any hash graphics without any rules as It’s a linear filter. Maybe adding some bilateral filtering idea to this jinc shader can bring good results…

#210

I have one issue concerning the dithering shaders. Independent if I use gdapt or mdapt, in some games the letters become hard to read. I would like to use a dithering shader to get games like “Kirbys Dreamland 3” and “Comix Zone” working. However the gdapt as well as the mdapt shader also affects “A Link to the Past”.

flat,800x800,070,f.u1.jpg800×676

Especially the “W” is always detected as dithering pattern, even with error correction. I have to increase the error correction until “Kirby” and “Comix Zone” also loses the dithering effect.

Can anyone suggest a settings or shader, which is universally usable with SNES / Genesis games?

#211

Nope. The shaders work through pattern detection and there’s no way to avoid all of the false positives without also missing a bunch of actual dithering (i.e., false negatives). Your only option is to use different settings for SNES and Genesis, with more aggressive detective on the Genesis, since dithering is more common there.

#212

Yes, but the pattern could be optimized. I mean in the case of the “W” you could do an “if…then” detection. Means if you have at least a patter of 4 repeats with the same lenght. means -> white black white black white black white black. Therefore the w is not detected but all remaining dithering patterns (which i currently saw) could be covered.

Edit: is snes9x using a dithering pattern? because kirby looks perfect and zelda as well. So maybe a port of this one could solve the issue.

Edit 2: seems snes9x relies on open gl dither.

Edit 3: the dithering algorithm seems a little bit different in OpenGL https://www.opengl.org/documentation/specs/version1.1/glspec1.1/node100.html but i dunno if the gl_dither can be replicated as retroach shader. Maybe a detection via a matrix could also be a good alternative approach.

#213

this is what gdapt does with the different error levels. but again like hunterk said, if you set it to just the right amount like in the case with the letter “W” to 4 so that those letters arend smeared anymore, all dithering patterns which are smaller than 4 are no longer detected anymore. with this approach you’ll always have this problem, since you cannot clearly differentiate between dithering and regular structures. the only thing I could imagine would be to train a neural network on this problem, but of course this isn’t trivial either.

regarding your edits… there are some snes games that have dithering by using the snes high resolution mode (Jurassic Park, Kirby’s Dream Land 3, …). in these cases you have a very easy solution. just put this as the first pass of your shader preset (cgp file):

shader0 = stock.cg
filter_linear0 = true
scale_type_x0 = source
scale_x0 = 0.5

and you’re done (you probably need to correct the path to the stock.cg file). this cuts the horizontal resolution in half by blending horizontal pixels pairwise together which results in a picture with the standard snes resolution of 256x224 instead of 512x224 (hires).

#214

Thanks a lot for the hint. What would happen if I add this shader to Genesis Games and other SNES games than Kirby, Jurassic Part?

#215

Would it be possible to update this shader to support linear light blending, similar to how the pixellate shader was recently updated?

  1. Source image
  2. Blurred in linear light
  3. Blurred in gamma light
  4. MDAPT (results similar to the blur in gamma light)

As you can hopefully tell, blending in gamma light darkens the image compared to the source while blending in linear light matches the original brightness.

#216

yeah, this is actually pretty easy to do, since Sp00ky Fox used a handy macro for all of the texture sampling, which we can wrap in a pow in the first pass and then use float framebuffers for the rest of the passes. Seems to work as intended: gdapt original: gdapt linearized: If Sp00ky’s okay with it, I can push linearized versions of mdapt and gdapt to the repo.

#217

I don’t think so. problem is that the metric I use is for gamma light as far as I understand (check it out http://www.compuphase.com/cmetric.htm). so transforming the colors values to linear light in pass0 would give you messed up metric values. this is why you got almost no blending there in your screenshot, the big difference is not a result from the now gamma-aware blending. the correct way should be to leave pass0 alone. in pass1 only apply the linearization on the xyz-component of the sampled colors (w-component is strength) and do gamma correction on the return line.

#218

This is exactly why I wanted to hear your input before pushing anything up :smiley:

I’ll try your suggestion and see if that helps and run those results by you, as well, before making any changes.

#219

Played around with this filter for weeks. But honestly, the filter causes to many failure. gdapt is to agressive and mdapt is missing options to avoid failure. Most common issues 3 dots (e.g. in a text), the letter W and M, spirte patterns.

An optimisation would be to a) select the repeat level of a pattern (means how many times, must a dot, a vertical or horizontal line appear to be a dither pattern). b) select the length of lines when a pattern should be applied (I guess the VO and HI Option in mdapt can provide this, but I am not sure - Moreover I was not able to manage that some pattern are ignored)

#220

@Sp00ky Fox Like this?

/*
   Genesis Dithering and Pseudo Transparency Shader v1.3 - Pass 1
   by Sp00kyFox, 2014

   Blends pixels based on detected dithering patterns.

*/


#pragma parameter STEPS "GDAPT Error Prevention LVL"	1.0 0.0 5.0 1.0
#pragma parameter DEBUG "GDAPT Adjust View"		0.0 0.0 1.0 1.0

#ifdef PARAMETER_UNIFORM
	uniform float STEPS, DEBUG;
#else
	#define STEPS 1.0
	#define DEBUG 0.0
#endif


#define TEX(dx,dy) tex2D(decal, VAR.texCoord+float2((dx),(dy))*VAR.t1)

struct input
{
	float2 video_size;
	float2 texture_size;
	float2 output_size;
};

struct out_vertex {
        float4 position : POSITION;
        float2 texCoord : TEXCOORD0;
        float2 t1;
};


/*    VERTEX_SHADER    */
out_vertex main_vertex
(
	float4 position	: POSITION,
	float2 texCoord : TEXCOORD0,

   	uniform float4x4 modelViewProj,
	uniform input IN
)
{
        out_vertex OUT;

        OUT.position = mul(modelViewProj, position);

        OUT.texCoord = texCoord;
	OUT.t1       = 1.0/IN.texture_size;

        return OUT;
}

/*    FRAGMENT SHADER    */
float3 main_fragment(in out_vertex VAR, uniform sampler2D decal : TEXUNIT0, uniform input IN) : COLOR
{

	float4 C = TEX( 0, 0);
	float4 L = TEX(-1, 0);
	float4 R = TEX( 1, 0);

	C.xyz = pow(TEX( 0, 0), 2.2).xyz;
	L.xyz = pow(TEX(-1, 0), 2.2).xyz;
	R.xyz = pow(TEX( 1, 0), 2.2).xyz;
	

	float str = 0.0;

	if(STEPS == 0.0){
		str = C.w;
	}
	else if(STEPS == 1.0){
		str = min(max(L.w, R.w), C.w);
	}
	else if(STEPS == 2.0){
		str = min(max(min(max(TEX(-2,0).w, R.w), L.w), min(R.w, TEX(2,0).w)), C.w);				
	}
	else if(STEPS == 3.0){
		float tmp = min(R.w, TEX(2,0).w);
		str = min(max(min(max(min(max(TEX(-3,0).w, R.w), TEX(-2,0).w), tmp), L.w), min(tmp, TEX(3,0).w)), C.w);
	}
	else if(STEPS == 4.0){
		float tmp1 = min(R.w, TEX(2,0).w);
		float tmp2 = min(tmp1, TEX(3,0).w);
		str = min(max(min(max(min(max(min(max(TEX(-4,0).w, R.w), TEX(-3,0).w), tmp1), TEX(-2,0).w), tmp2), L.w), min(tmp2, TEX(4,0).w)), C.w);
	}
	else{
		float tmp1 = min(R.w, TEX(2,0).w);
		float tmp2 = min(tmp1, TEX(3,0).w);
		float tmp3 = min(tmp2, TEX(4,0).w);
		str = min(max(min(max(min(max(min(max(min(max(TEX(-5,0).w, R.w), TEX(-4,0).w), tmp1), TEX(-3,0).w), tmp2), TEX(-2,0).w), tmp3), L.w), min(tmp3, TEX(5,0).w)), C.w);
	}


	if(DEBUG)
		return float3(str);

	float sum  = L.w + R.w;
	float wght = max(L.w, R.w);
	      wght = (wght == 0.0) ? 1.0 : sum/wght;

	return pow(lerp(C.xyz, (wght*C.xyz + L.w*L.xyz + R.w*R.xyz)/(wght + sum), str), 1.0 / 2.2);	
}

Here’s the result: