These shaders get rid of posterization (a kind of color banding). I made it in two passes to be used before any other filter. These artifacts are more common in 32-bit era games, mainly for PS1 and Saturn.
It’s already in common-shaders.
Some screens:
xBR original
xBR deposterized
xBR original
xBR deposterized
And more comparisons: http://screenshotcomparison.com/comparison/157391/picture:0
I’ve added some cgps inside xbr folders that use it.
BTW, it isn’t limited for use with xBR shaders. It can be combined with any shader. Because of this, I put it inside misc folder.
Hey Hyllian, good to hear from you again! That looks great. The gradients are much smoother after the de-posterizing process.
Yes. Tested here with N64 core and it can get rid of some bandings like those.
It isn’t a standard blur filter. It only actuate over very subtle bandings.
I implemented the bilateral filter a while back. depending on the radius and threshold it gets rid of dithering, banding or posterization. should have similar results I think.
I think Bilateral is in another category: denoisers.
Denoisers are more aggressive and have a purpose a bit different.
These deposterize shaders work over very subtle color variances. They can’t denoise an image nor get rid of dithering, for example. They will only get rid of banding.
well yes, it can also denoise. in general it’s an edge-preserving blur. basically a gaussian filter but it not only takes the distance in consideration, it also checks the color difference. depending on the chosen sigma values it can be well used for smoothing out abrupt color gradients without affecting edges or single outstanding pixels.
BTW, Sp00kyfox, have you already thought about implementing a NL Means denoiser in shader format? It’s the last frontier in denoising. There are some implementations in C++ and OpenCL out there, but they are very slow.
I heard of that filter and actually used it myself in video filtering. but I don’t have interest in implementing it and honestly I don’t see the purpose for such specialised denoising filters in videogame filtering. from what I can remember of NLMeans the real strength comes with a temporal dimension anyways. but we usually don’t have moving noise let alone noise in general. there is a GPU accelerated version for avisynth which is called KNLMeansCL, which is way faster than the original CPU based implementation but still way too slow for realtime. as a side note, in theory it might be superior but in my own experience mdegrain is still one of the best regarding avisynth denoisers (smdegrain is quite a nice wrapper for that, if you don’t already heard of it).
edit: tried to compare your result with bilateral by using your screenshot but failed to realize that you used it first with scale factor 1. anyways, your filter uses a hard specialized banding condition which bilateral can’t emulate. fine detail with surroundings of similar color tends to get blurry or flat with bilateral even with low sigma values.
ps: I’m still working on improving ScaleSFX, the next step would be adding AA-handling. I wanted to ask you if you could explain your approach to me of combing both filters in xBR-hybrid (maybe via pm). that would be very helpful!
[QUOTE=Sp00kyFox;33251]I heard of that filter and actually used it myself in video filtering. but I don’t have interest in implementing it and honestly I don’t see the purpose for such specialised denoising filters in videogame filtering. from what I can remember of NLMeans the real strength comes with a temporal dimension anyways. but we usually don’t have moving noise let alone noise in general. there is a GPU accelerated version for avisynth which is called KNLMeansCL, which is way faster than the original CPU based implementation but still way too slow for realtime. as a side note, in theory it might be superior but in my own experience mdegrain is still one of the best regarding avisynth denoisers (smdegrain is quite a nice wrapper for that, if you don’t already heard of it).
edit: tried to compare your result with bilateral by using your screenshot but failed to realize that you used it first with scale factor 1. anyways, your filter uses a hard specialized banding condition which bilateral can’t emulate. fine detail with surroundings of similar color tends to get blurry or flat with bilateral even with low sigma values.[/QUOTE] I didn’t know sdemgrain, good to know. I have some hope on a good denoiser for videogames since I saw some results from waifu2x with its denoiser activated. It seems to works well.
[QUOTE=Sp00kyFox;33251] ps: I’m still working on improving ScaleSFX, the next step would be adding AA-handling. I wanted to ask you if you could explain your approach to me of combing both filters in xBR-hybrid (maybe via pm). that would be very helpful![/QUOTE] Sure. Just send me your questions more specifically.
another thing, is it intended that your shader works RGB-plane-wise? the first argument to the conditional operator in your res assignment is a conjunction of bool3 vectors which itself is a bool3, so the selection of the second and third argument occurs element-wise and therefore possibly with different results across red, green and blue.
Yes, all calculations are done per channel separately.
ok, wanted to make sure. I messed around with your code and wanted to see how it looks like with a combined calculation but it weakens the shader effect so much that it becomes uneffective.
Should I use this shader before mdapt/gdapt/cbod or after? I’m guessing before
Yes, always before and using 1x scale. It needs two shaders slots.
depends on the shader. at least for mdapt and gdapt it’s probably not a good idea since they’re heavily depending on pattern recognition. and hyllians deposterize shader is also messing with the dithering. look at the following example:
original VS deposterize gdapt + deposterize VS deposterize + gdapt
Yes, that dithering is different from any other I have already seen. It’s very erratic.
I was wrong then, swapping mdapt and deposterize can lead to better results too. So, it should be tested both configs to see which one is better.