Bilateral Filter - another SmartBlur

RetroArch Additions Shaders
#1

Another attempt on the smartblur idea. After some experiments I found out about the so called “bilateral filtering” which is similar to gaussian blur but also calculates the weights based on the color distance. This way the blurring becomes weaker with the color difference of the pixels in question. It’s really nice for smoothing color transitions (debanding) or hiding dither (denoise). A problem though is that the kernel can’t be separated like with gaussian blur since the weights depend on the central pixel.

I tried my best though to make it as performant as possible. All operations are done with float4 to make usage of vector and matrix operations. Still the complexity is quadratic in kernel radius r, so at higher resolutions and with big r it’s probably gonna be slow. Anyway… there are 3 parameters you can adjust:

Radius - determines the radius r of the kernel (diamater = 2*ceil®+1)

Color Threshold - it’s not a hard threshold but a sigma value for an exponential function which maps the color difference to weights

Central Weight - the central pixel has the standard weight of 1. but you can increase it with this parameter to give the picture some of it’s texture and details back. this parameter is scaled relatively in the total sum of weights.

you can use it on the original resolution (higher impact) or on top of other shaders. set the scale factor to 1.0, filtering option depends on your shader combination

the code should be pretty easy to understand (except maybe for the choice of some constants, don’t mind them =D). I could keep the for loops in, since it seems that the cg compiler is automatically unrolling them if the number of iterations is static. would be nice though if others would have a look at it to see if there still possibilites to optimize.

anyway, here is the shader: http://pastebin.com/N38YXJCb

wikipedia articel for this method: https://en.wikipedia.org/wiki/Bilateral_filtering

imgur gallery with example shots: https://imgur.com/a/kshC6

#2

That’s really interesting. It does a great job of smoothing out the sky dithering without over-smoothing other stuff.

Good work, man :smiley:

EDIT: added it to the repo under the ‘dithering’ subdirectory.

#3

the application of the shader is actually more general. I thought it’d better fit in the blur section (it’s related to gaussian blur after all). in many dithering cases (where the color difference is too big) bilateral filtering does nothing.

btw some feedback about the performance would be nice. I have a very recent system but I’m interested to see how it works on other systems.

#4

I thought it wasn’t necessary to create another thread to talk about the same subject.

I just added a new bilateral shader to common-shaders repository. It’s called Fast-Bilateral.cg.

This isn’t so general as Sp00kyFox implementation. I needed a fast implementation without many config points, so I made it. It uses a kernel of 5x5 pixels only, and fixed. The only param to set is the range sigma, which controls the amount of blurring of the filter.

This shader is useful to denoise mainly games from the 32-bit generation. Many psx games have noisy assets with jpeg artifacts. It’s very apparent in dark areas.

The blurring param must be tweaked per game, because if you set it too high you may wash out some real texture details, on the other hand, if you set it too low, some noise persist in the game.

I created a new folder called ‘denoisers’ and put it inside. This is the main purpose of this shader: to denoise. (@Sp00kyFox, if you prefer, I can move your bilateral shader there too).

Another note: it should be used at 1x scale and must be the first shader in a chain. It prepares the game for other scalers. So I took some screenshots to show how Super-xBR can benefit from it:

Super-xBR:

retroarch-0428-093102x9es9.png853×768

Fast-Bilateral + Super-xBR:

retroarch-0428-0931174yisj.png853×768

Super-xBR:

retroarch-0428-093725cifha.png1024×768

Fast-Bilateral + Super-xBR:

retroarch-0428-09373974eqe.png1024×768

http://screenshotcomparison.com/comparison/171026/picture:0

#5

That’s a pretty dramatic change on that second set of pics. You could really see the jpg artifacts on the roof of the car in the first shot, while the second shot looks like that smooth 90s CG. Looks like it helps with the color banding, too.

#6

I have a problem with the regular bilateral.cg shader on AMD cards. I would get errors with the shader. It points on line 100 and 110 saying "profiles requires index expression to be compile-time constant. Also for lines 95 and 105, it says profile does not support “for”. This also affects my n64 vi filter .cgp files too.

#7

It has never worked for me too (AMD card here). I think it uses an advanced shader feature (indexed array inside loops) and AMD cards don’t support it I think. For it to work you have to unroll the loop and get rid of the indexed arrays. In my bilateral implementation I had the necessity to use a loop, but it wouldn’t work, so I had to create a macro and call 25 times in a roll. LOL

#8

@Hyllian yeah, I think it makes sense to put my version also to denoisers. don’t wanna have them all over the place. your examples are looking nice!

I wanna mention that the bilaterial filtering itself has a problem though:

The bilateral filter has limitations despite its popularity. It has been noticed […] that the bilateral filter may suffer from “gradient reversal” artifacts. The reason is that when a pixel (often on an edge) has few similar pixels around it, the Gaussian weighted average is unstable. In this case, the results may exhibit unwanted profiles around edges, usually observed in detail enhancement or HDR compression.

there is an algorithm called “Guided Filter” which fixes that, but I don’t see a smart to implement it as a shader. here are two related papers:

http://arxiv.org/pdf/1505.00996.pdf (this one has an easy to understand pseudocode) http://research.microsoft.com/en-us/um/people/jiansun/papers/GuidedFilter_ECCV10.pdf

a modification to the original bilateral filter is the “Biliteral Median Filter” where you don’t use the mean as the output but instead a weighted median. it seems to have better properties at preserving the intensity of edges and is definitely an easier approach than the Guided Filter. the problem here is that one would need to implement a sort algorithm in a shader. maybe you have a clever way to do this. here is the paper which describes it:

#9

Wow, the posterisation effect from xBR is almost gone! This is incredible!

#10

No xBR there, that’s Super-xBR (they’re distinct filters). Though xBR can benefit from bilateral filter too.

#11

Yes, I know. ^^ But also Super-xBR has a posterisation effect, which makes transitions in some games not that nice. Unfortunately it makes some textures not that nice.

Example: xBRZ (yes, I know it’s neither xBR nor Super-xBR, but the posterisation effect will be there as well) http://forums.ppsspp.org/attachment.php?aid=4416 Bilinear: http://forums.ppsspp.org/attachment.php?aid=4415

Therefore every approach which reduces the posterisation (either if it is dithering, SmartBlur, xSoft and now Bilateral) is a great improvement for the picture quality. This would increase the texture quality in PSX, N64, Saturn, Dreamcast, PSP or DS games dramatically. :slight_smile:

#12

[QUOTE=papermanzero;37989]Yes, I know. ^^ But also Super-xBR has a posterisation effect, which makes transitions in some games not that nice. Unfortunately it makes some textures not that nice.

Example: xBRZ (yes, I know it’s neither xBR nor Super-xBR, but the posterisation effect will be there as well) http://forums.ppsspp.org/attachment.php?aid=4416 Bilinear: http://forums.ppsspp.org/attachment.php?aid=4415

Therefore every approach which reduces the posterisation (either if it is dithering, SmartBlur, xSoft and now Bilateral) is a great improvement for the picture quality.[/QUOTE]

xBRZ, on the other hand, IS XBR! See my explanation here: http://filthypants.blogspot.com.br/2014/06/true-hq2x-shader-comparison-with-xbr.html?showComment=1430382548352#c8309218819721257465

About the posterization, yes, any filter that detect edges are prone to show posterization. BTW, I made other shaders to reduce posterization and put inside ‘misc’ folder. They’re called deposterization (two passes), though this bilateral is more effective.

Super-xBR presents less posterization than xBR/xBRZ. I intend to make a special xBR version with some posterization treatment in the future, though.

#13

[QUOTE=Hyllian;37990]xBRZ, on the other hand, IS XBR! See my explanation here: http://filthypants.blogspot.com.br/2014/06/true-hq2x-shader-comparison-with-xbr.html?showComment=1430382548352#c8309218819721257465

About the posterization, yes, any filter that detect edges are prone to show posterization. BTW, I made other shaders to reduce posterization and put inside ‘misc’ folder. They’re called deposterization (two passes), though this bilateral is more effective.[/QUOTE]

Yes, I know. I saw and read already your explanation. :slight_smile: My personal opinion, is that xBR is much more advanced nowadays than the small modifications of xBRZ. Super xBR finally is the ultimate approach at the moment. So thank you very much for your efforts and improving it further.

I would like to see and test how the shader comibniation (super xBR and bilateral) are working on textures (not fullscreen) in games like Silent Hill to see how the textures are becoming.