Mdapt & gdapt - dithering treatment [Updated 06/06/14]

RetroArch Additions Shaders
#141

Hi Sp00ky,

Thanks for looking into it. Here is what I get with pass0 (still different colors from what you get) http://www.denki-den.com/tmp/mdapt-ecco-pass0.png

And with pass0 and pass1 together (green screen, start flashing bright green as demo starts) http://www.denki-den.com/tmp/mdapt-ecco-pass0and1.png

Same results confirmed both on Windows and Linux. Don’t lose too much sleep over it, I understand shaders can be really hair pulling stuff between nVidia, AMD, Cg, d3d9 Glsl etc… I just wanted to know if other people have the same problem. Your earlier mdapt-4p from the repo seems to work alright so I’ll use that instead =)

#142

seems like there is something weird going on with the color channels. the way pass0 is intended is that the red channel marks a horizontal dithering pixel, green one the vertical direction and blue is just the min of the red and green intensity (which together with red and green gives the white areas). ok it seems that for example many pixels which should be green are instead kinda cyan’ish (which is rgb(1,1,0)). I took it for granted that values outside the [0,1] boundaries are automatically clamped in the pass output, but I could be wrong. so my guess would be that in this case for the pixels the red channel is negative and has a very small absolute value which on your system gets interpreted as a high positive value. same argument for the magenta pixels. let me see what it does if we clamp the output. here is another modification for you to test.

it’s weird though that the third channel (minimum of red and green) doesn’t seem to be affected from this. if red or green is a negative value, the minimum should also be and all these cyan and magenta pixel should be white instead. anyways, check out the test version while I’m still thinking about the problem ^^

#143

It seems to be working well here on my AMD GPU, FWIW.

#144

well no wonder here I guess. as the dev of the shader my GPU is also from AMD. well let’s wait tanuki is saying to the new test version. would be nice if you could look into the issue with cg2gsl.

#145

I’ll certainly take a look at it. It’s maister’s script, and I don’t have much experience with python, but maybe I can at least pinpoint what’s going wrong.

#146

I tried the new test file, but the problem seems to remain both in Linux and Windows: http://denki-den.com/tmp/mdapt-test2-ogl-pass0-ecco.png

The problem is with nVidia OpenGL somewhere since the shader works as intended with d3d9 on Windows: http://denki-den.com/tmp/mdapt-24-d3d-pass0-ecco.png

Talking about which, what driver do people recommend on Windows? d3d9 or gl?

#147

yeah, I don’t really get it. even if you don’t understand what I’m doing there, in the end I’m calculating two values r and g and I’m outputting R=r, G=g and B=min(r,g). so magent’ish (rgb(1,0,1)) or cyan’ish (rgb(0,1,1)) pixels shouldn’t be even possible, since it contradicts with the definition of the blue channel. maybe we’ll figure it out in time.

#148

the new fuzzy logic which I implemented in the last version led to some little holes in the checkerboard handling. so I added a few new rules. and another thing, the output pixel is now a linear combination of the original one and the merged sum, weighted by a probability value that this pixel is indeed part of a dithering pattern. this fits to the new smooth non-aprupt behavior of the shader.

mdapt v2.5

if anyone is interested, personally I like this shader combination.

shaders = 7

...

shader5 = 2xBR-Hybrid-v5-gamma.cg
filter_linear5 = false
scale_type5 = source
scale5 = 2.0

shader6 = advanced-aa.cg
filter_linear6 = true
scale_type6 = source
scale6 = 2.0

you can swap out shader6 though with ddt-extended.cg if you like it a little sharper.

#149

awesome man, stil not working in glsl format though. I am dying to use this as i recently figured out how to use 12 pass shaders in RA instead of just 8 and this would be perfect with a little combo I have been wanting to share but I was waiting for a less false positive prone version of this shader. Just have to wait a little longer I suppose. Thanks yet again for this kick ass shader though.

#150

sounds interesting, I’m looking forward to your shader combo. too bad I can’t help with the cg2gsl problem. if someone figure it out I’ll modify the shader accordingly.

#151

It might make sense to maintain a list of games that have been tested with it, and which work without any false natives/positives. I believe it would be impossible for it to work perfectly with every game possible since there’s so many thousands of different possible dithering patterns. But it could work perfectly for a more finite amount of games.

#152

I don’t know if a list makes too much sense since you can easily tweak the shader settings so that in the specific game the number of false positives is minor or they’re hard to notice. it’s also very dependent on the specific scene. maybe the game looks good at the start and then there is one scene in the midgame where mdapt screws up. overall the standard settings were cautiously chosen.

so a list with settings would probably make more sense. on the other hand, such a list could easily be outdated with a new version (not that I have new ideas atm) and do you really wanna compile a list with hundreds or thousands of games? :smiley:

yeah it’s convenient to throw mdapt on every game but it’s probably better to only use it on games where you know that it uses dithering constantly.

#153

I tweaked mdapt a little further. no new features here but overall a better behavior of the algorithm. the most important change is a new rule for the vertical line pattern. the waterfall in sonic looks flawless again like before the introduction of fuzzy logic.

here is the download and a detailed changelog:

mdapt v2.6

Update 12/06/13:

  • added a new vertical line rule in pass1 to fix some gaps
  • replaced euclidean distance with a fast color metric
  • strict option in pass0 automatically changes behavior in pass1
  • replaced sigmoid function in pass2 with the more suitable tanh (approximation)
  • removed a too aggressive checkerboard rule
#154

I’ve gone through every pass to check for more improvements and funny enough I seem to have overlooked something very basic. mdapt actually never cross checks the horizontal and vertical colors in the CB detection. this is of course a source for false detections. for example this pattern

  R
G B G
  R

in v2.6 gets the strongest CB value of 1.0, in v2.6a the value is now significantly lower corresponding to the color metric value of R-G. apart from this there is nothing new. I think this will be it for 2013 :wink:

mdapt v2.6a

#155

Awesome job! Is this newest shader set batch included with the newest version of PS3 retroarch or do I need to download it separately? I ask because I tried your link and it says the archive is corrupted, etc.

#156

I don’t know if it’s included in the PS3 build or not. I would assume so, but if not, you can get it from https://github.com/libretro/common-shaders

#157

Hey Spookyfox, or anyone for that matter,

I’ve tried this new shader set. the 5-pass 2.6 version. Not sure it is working. :-/

My real hope with this is that I can use it WITH some xBR filters. Am I able to with these? Doesn’t seem like it when I test them out. Just reverts to the stock pixeled look.

Im running 0.9.9 retroarch.

#158

Maybe you have the same issue like me? Doesn’t look like someone is in the mood to fix it. viewtopic.php?p=8848#p8848

Of course you still have “pixel look” with only mdapt.

#159

Touche. Yeah, what i meant to say is:

Can this wonderful mdapt 2.6a multipass be used with any xBR? In fact, in the new retroarch can filters be added after loading the presets? Doesn’t seem to let me… :-/

I want to use the newest 2.6a filters and then independently add an xBR filter and maybe a CRT scanline look.

For now would be satisfied with the 2.6a + the newest lv3XBR multipass.

Can these two multipasses be combined in the newest version of retroarch?

#160

I believe you would have to make a combined cgp that merges the two presets with mdapt’s passes first at a scale factor of 1.0 and then xBR’s passes after that. If a multipass xBR doesn’t work for you, you should be able to use one of the older single-pass varieties instead tacked onto the end.