Checkerboard-Dedither shader

Hyllian · 15 September 2022 13:57

This is a byproduct from sgenpt-mix shaders. I decided to split it between checkerboard dithering and pseudo transparency (vertical lines dither). So, sgenpt-mix will stay as is and later will only treat pseudo transparencies.

This one is called Checkerboard-Dedither. It’s a multipass shader and does a better job than sgenpt-mix in this regard.

This is a first version, so I need to evolute it before insert in slang shaders repo.

If you’re interested, use shader update online from Retroarch.

I appreciate feedback about its flaws.

Some screens (raw first, dedithered later):

It works even better in combination with some crt:

Hyllian · 15 September 2022 13:57

New release:

What’s new:

Added an extra pass to trim stray pixels;
Improvements in pattern recognition, now it can smooth 2x3 checkerboards.

Some screens (raw first, dedithered later):

hunterk · 28 August 2022 12:33

wow, those are big improvements. Looks fantastic!

Hyllian · 28 August 2022 13:05

With the 2x3 checker detection it was possible to filter this in Sonic intro screen:

And Street Fighter and other capcom fighting are totally filtered:

HyperspaceMadness · 28 August 2022 14:03

Wow! This looks amazing! I feel like none of the other methods we have deal with the stray pixels like this, at least without going full jailbar de-dither.

Great stuff!

The Sonic Example is really amazing, here’s the before and after

I forsee an immanent addition to the Mega Bezel

Hyllian · 28 August 2022 15:04

Particularly, this screenshot is very hard to filter and produce a quality output:

In native res if you wanna test with other algorithms:

Pulseman10

The sawtooth artifact is hard to treat!

Hyllian · 15 September 2022 13:58

New release: Use RA shader update online to get it.

What’s new:

Some small improvements in pattern recognition;
Stray pixels trim improved;
Made a Pull Request, so it’ll be on repo soon.

HyperspaceMadness · 2 September 2022 20:01

Wow, this is really looking amazing, fantastic achievement @Hyllian!

Cyber · 2 September 2022 20:25

Great work @Hyllian. Can you do some further testing using PC-Engine and SNES Games as well please if just to fine tune and further put the algorithm through its paces? Lots of PC-Engine games use checkerboard dither patterns and the old blurring methods or even Blargg produces results that don’t look like the sharp composite output PC-Engine is renowned for.

Hyllian · 2 September 2022 20:40

Do you have examples of pc engine that uses dithering?

For SNES, I think this one work ok as the checkerboard are even less than the genesis.

I put a param called “HUD details” that is set to 1.0 and can be increased up to 5.0. If you increase it trim more stray pixels, so you can tweak it for other system or per game.

Cyber · 2 September 2022 21:07

Of course, off the top of my head,

Lords of Thunder - Water level, waterfall
Last Alert - Cut scenes
Ys Book I & II - Intro
Ys IV - Ending
Madden - Grass texture
Devils Crush - Title Text and some of the bricks in the main table.

These are just a few but unfiltered dithering is a thing I’ve noticed often in PC Engine games, especially the ones that are trying to look more realistic.

I’ll try to add to the list later as I see notice more.

Hyllian · 2 September 2022 21:01

It’ll be more helpfull if you post native res screenshots of these game moments.

False positive or false negative will always be a side effect of these kind of filtering. It would need some very advanced AI to separate all true dithering from false ones.

Cyber · 2 September 2022 21:01

No problem, that will have to wait until whenever I get a chance.

Neofuuma · 2 September 2022 21:57

Here is one from the Lords of thunder level

Hyllian · 2 September 2022 22:08

I’m sure the checkerboard-dedither can filter that watterfal! The Pattern is standard.

Hyllian · 3 September 2022 18:42

Using my new crt preset that waterfall can be filtered:

Using only checkerboard-dedither:

Cyber · 3 September 2022 19:05

Wow!

Looks great! Hopefully minimal false positives! I haven’t forgotten about the other examples by the way, just don’t have the time right now to take the screenshots and upload but I would still like to contribute, if only to help test and verify the functionality.