Mdapt & gdapt - dithering treatment [Updated 06/06/14]

RetroArch Additions Shaders
#1

mdapt - merge dithering and pseudo transparency

0. Introduction

well it’s time for a solid start post. so let me start by explaining what it’s all about. in many old arcade or console games are dithering effects which are there to compensate platform weaknesses or generate specific effects like transparency by using an alternating pattern every frame. this works so well since back then the monitors (CRTs) had scanline bleeding and other certain features which merged the dithering through the display technology. but nowadays every pixel will displayed perfectly so dithering won’t look like it should be.

1. Algorithm

there are a few other shaders who are trying to simulate how a CRT displays an image. mdapt goes a different way and tries to detect specific patterns by analyizing the relation between the pixels. this way only those specific parts of the image are merged, which is a good base for further scaling e.g. with xBR.

mdapt can detect two basic patterns. the so called checkerboard pattern and vertical lines (see the screenshot section). it actually doesn’t matter of how many colors the pattern consists and mdapt doesn’t even use difference thresholds to determinie similarity. the algorithm just looks for a regular “up and down” between the pixels. there will always be errors since the dithering process itself is lossy and not invertible. but mdapt tries to balance it by checking if there are enough detections in one local area. if that’s the case it’s considered as dithering and will be merged. there are a few settings you can manipulate in the cg-files though, see section 3.

2. Download

common-shaders repository

Update 06/06/14: gdapt and mdapt support the new shader parameter feature which allows you to modify them using the Retroarch GUI.

  • MDAPT: sigmoid function in pass2 uses smoothstep instead of tanh
  • MDAPT: checkerboard merging is sharper by using only the orthogonal neighbors
  • GDAPT: added a custom error prevention level. tradeoff between number of detections and false ones.

Update 02/20/14: You now can find some preconfigured .cgp-files I like to use on the shader repo along with official readme’s. mdapt updated to v2.7:

  • reworked vertical line handling
  • eliminated the need to check colors in pass1
  • improved checkerboard ruleset in pass1
  • fixed a bug in pass2 regarding calculation overflow with tangent hyperbolic
  • removed some unneccessary parts in pass4

Update 02/05/14: Added a dumped down but more aggressive version called “gdapt”. It should replace the mdapt-2p version on the shader repo.

Update 01/21/14:

  • better detection of border pixels in dithering areas

Update 12/11/13:

  • added vertical-horizontal coherence in CB detection (reduces false detections)

Update 12/06/13:

  • added a new vertical line rule in pass1 to fix some gaps
  • replaced euclidean distance with a fast color metric
  • strict option in pass0 automatically changes behavior in pass1
  • replaced sigmoid function in pass2 with the more suitable tanh (approximation)
  • removed a too aggressive checkerboard rule

Update 11/24/2013:

  • added some new checkerboard rules to fix the gaps due to the new fuzzy logic
  • the output pixel is now a weighted linear interpolation between the original pixel and the merged one

Update 11/21/2013:

  • replaced binary logic with fuzzy one, fixes many false detections

Update 11/09/2013:

  • fixed: holes in vertical line patterns
  • fixed: checkerboard with two or three lines weren’t detected
  • added a precalculation pass, no performance impact
  • removed many false detections especially at text
  • combined normal and strict version (you can switch the mode in pass0)
  • moved pattern settings to pass4

Update 10/29/2013:

  • fixed: 45° edges were interpreted as checkerboard dithering

Update 10/28/2013:

  • eliminated all branches: performance improvement about 25%

3. Usage

mdapt consists of 5 passes (precalculation, pattern detection, isolation termination, pattern completion, blending). every pass works on scale factor 1 and should be considerably fast even for old systems. just use the cgp I wrote as a base and add additional shaders if you want.

to the settings: mdapt comes with two modes. the unrestricted one which is flexible in detecting patterns with any number of colors. and a strict one which only accepts patterns consisting of two different colors. the later one may reduce errors in graphically simple games (think gameboy, nes etc.) but don’t expect it to look good on games with pseudo transparency effects. you can switch between them via the strict def in pass0.

in pass4 you can deactivate the detection of checkerboard or vertical lines by commenting out the corresponding def line. pass2 is important for the error detection balance. there are two minimum values (int2 thresh) which determines how many detections must be in a local area to be considered as dithering. the standard values are working well for the most part.

4. Screenshots

complete gallery

768×448
768×448

my original post which started it all:

I played a few genesis games lately where I noticed that the dithering effect is often used to compensate the low number of possible colors in comparison to the snes. Yeah there are already many crt and ntsc filters available but if used with some sharp scaling filter like xBR or scaleX it looks like garbage of course.

So I thought about a pre-shader which keeps the resolution intact but blends the pixels only in those areas where dithering or pseudo transparency is present. This way you could use that image as input for a sharp high resolution shader. Too bad I have no experience with shader programming. I would need some time to read me into the subject but I thought people can help me out here with their knowledge.

Well just let me explain my firs naive idea. Consider a pixel C and his left and right neibors L and R. If the difference between L and R is under a given threshold they’re assumed equal and the output pixel C_new is a blending (with equal weights) of all three original pixels C,L,R. Otherwise we just keep the original pixel. That would merge dither with horizontal direction. questions is what to do if one would append the rules in the same way to vertical direction (with neigbor pixels U and D) and both directions are detected as dither. Merge all 5 pixels? Another problem would be fine lines or details like eyes where the neighbors are equal but it’s not supposed to be dither. Maybe check one addtional pixel in every direction to detect a pattern?

what do you think about it? feedback and help would be appreciated, thanks

ps: here are some example images of dithering in genesis games. http://retro-sanctuary.com/comparisons%20-%20differing.html

1 Like
#2

It’s been discussed, but the false-positive problem is pretty overwhelming. What might work better is to use aliaspider’s GTU shader at a low scale, like ~2 (scale of 1 seems to get weird) with the HQnx/xBR/whatever shader doing the rest of the scaling.

Similarly, for SNES at least, you could use the hires blend shader at a scale of 1 and then follow it with xBR or whatever.

1 Like
#3

could you post me a link to the discussion? I’d be interested in the results even if they’re not working like it’s supposed to be. well to your idea, I don’t think that’ll work. most scaling filters of this category need the pixel art structure in their original resolution. if one pixel is now a 2x2 area it’ll detected as a block and not as part of a line for example.

#4

Some of it is covered here: http://board.byuu.org/viewtopic.php?f=10&t=3712

Smoothing out dithering without going for heavy phosphor emulation, scanlines, etc. was one of aliaspider’s chief goals in designing that GTU shader.

I think Hyllian has talked about it some as it relates to xBR, as well, since the pattern recognition aspect would likely be very similar to what xBR does. If it’s a thing you would like to explore yourself, I would think that would be a good place to start. His 4.0 version looks beyond the immediately surrounding pixels, so that might help avoid some of the false-positives, as you mentioned.

1 Like
#5

ok, got a little bit into it. made a first attempt at trying to implement my first idea.

here is an example (original, shader):

edit: never mind. fixed the error by myself :wink:

1 Like
#6

Oh yeah, that looks great, so far! gj, dude :slight_smile:

Here’s how the snes-hires-blend shader looks (the image is split diagonally, with the original image on the left and the blended one on the right): I think the result is pretty similar to what you’re working on, so you can have something to compare with later on, once you start combining with the scaling shaders.

#7

Hi guys, I am new to RetroArch, this custom shaders feature is amazing thing! I am start to make some experiments with multi pass shaders, so here is example of current wip: At the moment Genesis games looks more like old DOS versions, a bit blurry, but still i like it more then original ones.

#8

@coastkid yeah, a bit blurry for my taste, but it definitely smoothed out the dithering :slight_smile:

How does it work?

#9

Right now I’m using 3 passes with 4 pixel blur and different threshold value, i am not sure if this is the best solution, so i will try some other ways to increase sharpness.

Another example, affection on colors:

#10

nice to see that someone else is working on it. well I’ve finished my first attempt so far. I’m calling the shader mdapt (merge dithering and pseudo transparency). works fine so far but I had to lower my wishes regarding enhanced effekts like those damn waterfalls in sonic. with threshold values I had too many false detections so I just concentrated on clear dithering patterns.

here are 2 examples of the shader in action:

and the download: https://anonfiles.com/file/b36189ae6849b943ad3bfb9c63b6c0ba

btw a friend of mine can’t run it in opengl on his nvidia card. can someone confirm it and have a look on the shader if there is something incompatible? any other suggestions or enhancements to the algorithim are also appreciated.

1 Like
#11

@Sp00kyFox Nice! I’d say that compares favorably to the snes-hires-blend shader, since your mdapt keeps the edges of non-dithered pixels sharp. The Lion King title screen is a pretty striking improvement.

@coastkid Have you looked at Hyllian’s Reverse AA shader(s)? Adding a pass of that at the end might sharpen it up a bit without sacrificing the blending.

1 Like
#12

Cool shader, I uploaded it to common-shaders and gave you credit for it.

BTW - in case people didn’t know - another good ‘use case’ for this is Street Fighter Alpha 3 (arcade, FBA) with the ‘FIght’ splash screens at the start of every fight and the ‘You Win’ splash screens.

#13

This is seriously cool. Anyone tested it with Streets of Rage 2’s ceiling lights?

#14

Hey, Sp00kyFox,

Nice to see you here.

Dithering is something very hard to detect without break something else. It seems you’re finding some great rules for it. Do you recommend use it at 1x scale and then use another shader to upscale the image?

#15

also tried it with street fighter alpha 3. yeah the pseudo transparency effect are not detected perfectly but it’s ok I think. here is a comparison (original, mdapt, mdapt + bilinear filtering):

[Album] Imgur

@Squarepusher thanks. but the cgp file is kinda important to make sure that the scaling factor of both passes is 1x. btw I made some small changes to my code, added some comments and modified the neighbor rule in pass2. so here is the latest version: https://anonfiles.com/file/7fae14d5e0e1d6ef9cdfa7f07dfa11d8

@GPDP I don’t know the game too well but I took a fast look at the first stage. well the neon signs aren’t detected but they aren’t supposed to since the blocks are too big.

@Hyllian hi! nice to see you again and that you’re interested in this thread. (thanks again for your DOOM related help) I’m kinda wondering now about all the praise and amazement since I thought my approach is pretty basic. if there is interest though I could post a commented pseudo code to explain it.

and yeah, mdapt is supposed to work at scale factor 1x. feel free to use any shader you think appropriate on the result (use the cgp-file for that). for games with many pseudo transparency effects I’d recommend something that blurs the little flaws of my shader like crt, phosphor or gtu maybe. games with only “normal” dithering on the other hand should also look pretty good with mdapt and your xbr(h) scaler.

#16

thanks. but the cgp file is kinda important to make sure that the scaling factor of both passes is 1x. btw I made some small changes to my code, added some comments and modified the neighbor rule in pass2. so here is the latest version: https://anonfiles.com/file/7fae14d5e0e1d6ef9cdfa7f07dfa11d8

Updated it and added the cgp as well.

#17

@Sp00kyFox Your shader is really good, very accurate dither selection! Do you have any plans to add some kind of vertical dither processing?

Thanks for the advice, i’ve tried, but it looks noisy with my output. I’ve made my own low-res sharpness so here is the results (accuracy is still far away from Sp00kyFox’s shader, but…) : https://dl.dropboxusercontent.com/u/8913813/retroarch/coastkid_filter_test_04.png https://dl.dropboxusercontent.com/u/8913813/retroarch/coastkid_filter_test_03.png

#18

could you post some examples where it would matter? how I understand it, the dithering effect is based on the color bleeding of crt scanlines which are projected horizontally. but I guess you could just use the same approach for vertical lines. of course it would be interesting to use surrounding pixels of neighbor lines to detect difficult patterns.

made another small improvement. so far mdapt didn’t blend the pixels at the boundary of a dithering pattern. now they also get detected. here is the new version:

https://anonfiles.com/file/7dec254d83caa5a0f03235aa06713422

and a comparison between original, v1.1 and v1.2 on altered beast (with a thresh value of 0.0). it’s the original resolution so you probably have to zoom to see the difference:

#19

Very good.

How this work for high contrast ditherings like those in the power bar in MK3?

Ex:

Could you make a pseudo code explanation of this?

1 Like
#20

could you post some examples where it would matter? how I understand it, the dithering effect is based on the color bleeding of crt scanlines which are projected horizontally. but I guess you could just use the same approach for vertical lines. of course it would be interesting to use surrounding pixels of neighbor lines to detect difficult patterns.

made another small improvement. so far mdapt didn’t blend the pixels at the boundary of a dithering pattern. now they also get detected. here is the new version:

https://anonfiles.com/file/7dec254d83caa5a0f03235aa06713422

and a comparison between original, v1.1 and v1.2 on altered beast (with a thresh value of 0.0). it’s the original resolution so you probably have to zoom to see the difference:

[/quote]

Do you have a Github account?

if so, I could give you commit access to our common-shaders repository so that you can update new versions directly. Hyllian also commits his shaders to the same repo.

Interested?

1 Like