xBR algorithm tutorial

RetroArch Additions Shaders
#150

I’m having a look at hyllians warp filters right now and it seems there is an issue which I don’t quite understand if it’s a bug or a hardware specific thing. the erosion.cg shader only gives me a black picture without any kind of error message. …

edit: seems it’s a hardware related problem. I reduced the fragment shader of erosion.cg to a basic core…

/*    FRAGMENT SHADER    */
half4 main_fragment(in out_vertex VAR, uniform sampler2D decal : TEXUNIT0, uniform input IN) : COLOR
{
	half3 E = tex2D(decal, VAR.t3.yw).rgb;
	return half4(E, 1.0);
}

which still results in a black screen.

edit2: here is an ati fix of both shaders https://anonfiles.com/file/16adc28bf655 … 23bf2438e4

#151

I have a feeling the problem lies in the swizzle part, because in bicubic-fast.cg I made some different approach and it works in my ati card. Look how I made in bicubic-fast:


		tex,
		float4(tex,tex) + float4(   -dx,    -dy,    0.0,    -dy), 
		float4(tex,tex) + float4(    dx,    -dy, 2.0*dx,    -dy),

And the swizzle in erosion shader:

OUT.texCoord = texCoord;
	OUT.t2 = texCoord.xxxy + half4( -dx, 0, dx,    -dy); //  A  B  C
	OUT.t3 = texCoord.xxxy + half4( -dx, 0, dx,      0); //  D  E  F

When I get home I’ll try the bicubic approach on erosion to see if it solves the problem for ati cards without degrading performance. If yes, this solution should be done in other shaders that are atiphobic (xbr mlv4, for example).

Maybe doing this should fix the problem:

OUT.texCoord = texCoord;
	OUT.t2 = float4(texCoord.x, texCoord.x, texCoord.x, texCoord.y) + half4( -dx, 0, dx,    -dy); //  A  B  C
	OUT.t3 = float4(texCoord.x, texCoord.x, texCoord.x, texCoord.y) + half4( -dx, 0, dx,      0); //  D  E  F
#152

Just updated repo with a new shader: smart-blur.cg

It has a threshold (configurable) to apply blur only in low frequency color transitions. It helps to minimize posterization artifacts in xbr outputs. So, it’s a good second shader to any xbr shader.

Here’s a screenshot comparison without and with smart-blur applied: http://screenshotcomparison.com/comparison/73035

The shader and preset are in the repo:
https://github.com/libretro/common-shad … rt-blur.cg https://github.com/libretro/common-shad … t-blur.cgp

#153

That really works some magic on those gradients! Good stuff, man :slight_smile:

#154

Yes, it does a very good job in some games. In others it isn’t very good. I need to tweak the thresholds a bit more to find a sweet spot. And the shader needs some optimizations too.

I specially like that in some games the backgrounds seem to have some kind of depth of field effect! =D

#155

xbr really benefits from it. still a fan from the hybrid variant =D anyway… don’t know if it makes any difference in performance but wanted to mention

bool eq(float3 c1, float3 c2) {
    float3 df = abs(c1 - c2);
    return (df.r < threshold.r) && (df.g < threshold.g) && (df.b < threshold.b);
}

can be simplified to this

bool eq(float3 c1, float3 c2) {
    return all(abs(c1 - c2) < threshold);
}

see: http://http.developer.nvidia.com/Cg/all.html http://http.developer.nvidia.com/Cg/any.html

the algorithm itself seems suprisingly simple. but I don’t know why the 4 pixels outside of the 3x3-box around the center are involved. seems like it could be a problem for small details if the shader is used at lower scale factors.

#156

Tks, Sp00kyFox,

I’ll optimize it soon.

The four pixels are necessary to broad the blur, otherwise the effect would be too tiny.

The hybrid uses another approach. I’m thinking if it’s possible to relate both approaches some way.

#157

okay, I guess what confused me is why those pixels aren’t used in the equality check. consider the central pixel in a white 3x3 box which is adjacent to a black line/area. your equality condition is obviously true so the result color gets darkened despite the big color difference where the white pixel adjacent to the black border stays white. that’s probably not intended.

#158

You’re right! That’s an inconsistency.

BTW, I’ve changed a bit the approach and it’s now two times faster. It should be used with LINEAR enabled (though I couldn’t see any differency when using NEAREST. Weird!).

This is v2: http://pastebin.com/Punji1rP

#159

I’ve made a shader that tries to unblend any shader combination. That is, for example, let’s say you upscale a game using Retroarch and a combination of shaders. And it results in something too blurred for your tastes. If you put unblend.cg after the last shader, the output is unblended and the final palette will be more or less, the original one.

It works surprisingly well in some shaders. For example, it can turns ddt.cg output in something similar to xBR:

ddt standard output:

ddt at 2x and unblend.cg as a second shader to upscale to any size using linear samples:

Even xBR can benefit from this. Using xBR at only 2x (for a fast usage), and using unblend.cg as a second shader, turns the output into something very similar to a much higher xbr (5x, for example).

xBR at 2x only output:

xBR at 2x with unblend.cg as a second shader to upscale to any size using linear samples:

But, it doesn’t work with shaders that changes too much the original source. For example, it doesn’t work with crt shaders, because the scanlines aren’t in the original source, so it can’t be unblended. It works well with shaders that try to interpolate the source, like: bicubics, lanczos, ddt, bilinear, jinc2, advancedAA, etc.

The unblend.cg shader is very simple and only works as the last shader with linear enabled: http://pastebin.com/AHxyrUFN

EDIT: The ddt I’ve used in the first shots is an updated one. The current ddt in the repo doesn’t work well with unblend.

EDIT2: The way unblend works is like a FAKE signed distance field (SDF) method. It treats the input as an SDF map and do some kind of alpha test, though it doesn’t choose between two colors (black or white), it decides among a bunch of colors in the region of the low res input. It’s a bit prone to errors, though. I think I’ll try a gaussian fitler to see how it can fake an SDF compared to ddt.

#160

Can I get some help? I need to convert the newest version of the CG shaders of xbr multipass shader (mlv4) into GLSL.

Unfortunately, I’m not good with scripting and coding and can’t figure out how to use github to do this.

I using retroarch on PC but the problem I’m facing is that I can’t combine the 4 pass mdapt with this one. I saw the newest version of xbr has a change that may make this possible now.

If anyone could help me, I would appreciate it! :slight_smile:

#161

Has anyone tried removing the branches and using floating point masks? I’ve modified the SABR shaders and was able to go from 2x to 4x GBA upscale on a Nexus 4.

Not sure if it will have as much as an effect for CG on a PS3 (mobile allows less depth on branching)

Sample:

#162

[QUOTE=commiebits;22264]Has anyone tried removing the branches and using floating point masks? I’ve modified the SABR shaders and was able to go from 2x to 4x GBA upscale on a Nexus 4.

Not sure if it will have as much as an effect for CG on a PS3 (mobile allows less depth on branching)

Sample: https://github.com/commiebits/emulator-shaders/blob/e27cca745a49d7e72107309223363b9356ed4cb4/assets/sabr.shader[/QUOTE]

I have a feeling it helps PS3, yes. Thanks for sharing.

#163

News!

Long time since something interesting happened to add to xBR. Now, finally I figured out something I was looking for years! It was so hard to find (maybe I’m dumb) I have created the entire xbr-hybrid variant to overcome this (LOL). Now I’ll finally create a xBR for more genera tasks and more powerful with cartoon AND digitized images. What I’m saying is that the next xbr will be very different from the others as I had to rewrite the shader almost from scratch. It now can detect edges in horizontal and vertical directions (all the old ones only detect in diagonals). There are more two benefits from this approach: firstly, it’s possible to get rid of some annoying artifacts in dithering regions by control the level of interpolation (it can turn totally bilinear, for example); and second, some artifacts of digitized games that some vertical and horizontal lines couldn’t merge, now it can… The only drawback is that I can’t detect corners with this new approach, so the old ones must keep their place yet. The other problem is that I can only scale for 2x, though using jinc2-sharper as a last shader can scale the result to any size almost seamlessly. I think the quality is comparable to nedi with much less artifacts. In this approach I have to test many other things before releasing, so I’ll let some screenshots:

Look at the flamethrower reservatory, the dithering is destroyed by artifacts:

p3pLFpc.png768×672

Now, using the bilinear control, it’s possible to take xbr out of the dithering and keep it in other parts (like the character, for example):

pD2ZdXm.png768×672

more here: http://imgur.com/a/e4mqp

#164

Ok, as a POC, I just added a first version to common-shaders: https://github.com/libretro/common-shaders/tree/master/xbr/super-xbr

It’s a multipass shader with some parameters you can tweak on the fly:

Bilinear Weight: it’s set to 0.4. If you want full xbr effect, turn it to 0.0. Above zero, it can let some patterns untouched. It must be good for games plagued by dithering or with very subtle color gradients (digitized games).

The last pass is a jinc2 customized. It has an anti-ringing, though it isn’t set by default. Just increase it and you can get rid of rings. But beware, the anti-ringing may tone down some texture details too.

It doesn’t have corner detection, so fonts and huds are smoothed to the bones! I didn’t figure out yet how to treat corners in this new implementation. As it’s almost a different filter than the standard xbr, I call it Super xBR from now on. It furthers the concept of edge detection by applying rules to vertical and horizontal edges. By doing this, color gradient changes in these directions are much more smooth then before. For this new implementation multipass is mandatory, or else the speed in single pass would be too slow (for now I think…).

#165

Improved the transition between bilinear and xBR algorithm. It’s already in common shaders. Now the transition is almost seamless!

more screens here: http://imgur.com/a/nRYc2

#166

Great work! (as usual) Can’t wait to try it, btw I’m using the 2xbr-hybrid(scale 2x)+smart-blur, do you think this new filter is better than this?

#167

[QUOTE=final_87;22827]Great work! (as usual) Can’t wait to try it, btw I’m using the 2xbr-hybrid(scale 2x)+smart-blur, do you think this new filter is better than this?[/QUOTE] Well, the visual pleasure is personal. But, for me, I’d say yes.

#168

Added jinc2 interpolation to parts of code when edges are detected. The results improved a bit more! It’s already uploaded to common-shaders: https://github.com/libretro/common-shaders/tree/master/xbr/super-xbr

#169

I just updated common-shaders with the new Super xBR. Now it is more smoother!

Here’s an example:

Upscaled with NNEDI3 (a great filter of Avysinth that use neural networks to scale) at 4x:

clown_nnedi3.png768×1024

Here’s the same image upscaled by Super xBR at 4x using standard config:

2KpcvKO.png768×1024

And here’s the same image upscaled by Super xBR at 4x using a sharper config:

GIYVRqd.png768×1024

New version is in this link: https://github.com/libretro/common-shaders/tree/master/xbr/super-xbr