xBR algorithm tutorial

RetroArch Additions Shaders
#189

At last, after almost a year without improvements, now I have a new xBR shader with a new corner detection and a new color difference algorithm.

I’ve used the color difference algorithm from Sp00kyfox’ ScaleNx. I tweaked it a bit, because the standard one was giving me some weird results in my tests, so I had to change some params.

The repo is already updated: https://github.com/libretro/common-shaders/blob/master/xbr/xbr-lv2-accuracy-multipass.cgp

With the new color diff algorithm, some artifacts (red/blue spike artifacts) are fixed. These artifacts are very apparent in Final Fight 3. The difference between the standard xbr shader and the new one can be seen here:

http://screenshotcomparison.com/comparison/164239/picture:0

The only drawback for the new color diff algo is the speed, it’s a heavy code. For now, it only works fullspeed in multipass. So, this new shader run in two passes and is called xbr-lv2-accuracy-multipass. I think I can update the single pass shaders at least with the new corner detection, but not the color diff algo. So, for now, the new multipass shader is the best one in IQ.

Standard xBR:

New xBR (color diff and corner detection improvements)

#190

Quite nice. When you want super clean image quality with minimal detail loss, this seems like the way to go.

#191

For games without anti-aliasing, most of 8-bit games, you should give a try to ScaleFx-3x too, it does an amazing job with stairs up to level 6 (as long as they don’t have AA)! For now, xBR can only get up to lvl4 (with AA).

#192

I fixed a small bug in smart-blur shader that was causing some black halo artifacts. And added a new preset to use smart-blur with xbr-accuracy.

A comparison with/without smart-blur: http://screenshotcomparison.com/comparison/164360

It smooths some outlines.

Other comparison: http://screenshotcomparison.com/comparison/164365

#193

Ah, that looks really nice with the smoothed transitions along the color bands, and it makes the pattern in the chrono trigger background more mellow. Good stuff all around!

#194

@Hyllian nice to see that the metric helps you too! as I was starting with the development of ScaleFX I tested several fast methods for color metrics but this one was surely the best. yeah it costs more but still more performant than going the full way and doing a Lab-colorspace conversion. feel free to use my implementation for it. it just looks a litte convoluted since I implemented it in a way so it does 4 metric calculations at once which should be faster than doing them individually but I didn’t change the formula itself apart from inversing the result and scaling it to [0,1]. you can also get rid of the square root by squaring your threshold value which you only need to do once if it is constant. I coudn’t do it in my case since I needed to use the metric result as the output of a shader pass and without the square root there is a noticeable precision loss due to the frame buffer. here is the “clean” float4-version if you have a use for it:

float4 df(float4x3 A, float4x3 B)
{
	float4x3 diff = A-B;
	float4   ravg = 0.5 * (A._m00_m10_m20_m30 + B._m00_m10_m20_m30);

	diff *= diff * transpose(float3x4(2.0 + ravg, float4(4), 3.0 - ravg));
	
	return mul(diff, float3(1));
}

the result of df(A,B) is the squared difference of (A.x, B.x), (A.y, B.y), (A.z, B.z) and (A.w, B.w). maybe you have another idea to speed it up?

#195

[QUOTE=Sp00kyFox;35348]@Hyllian nice to see that the metric helps you too! as I was starting with the development of ScaleFX I tested several fast methods for color metrics but this one was surely the best. yeah it costs more but still more performant than going the full way and doing a Lab-colorspace conversion. feel free to use my implementation for it. it just looks a litte convoluted since I implemented it in a way so it does 4 metric calculations at once which should be faster than doing them individually but I didn’t change the formula itself apart from inversing the result and scaling it to [0,1]. you can also get rid of the square root by squaring your threshold value which you only need to do once if it is constant. I coudn’t do it in my case since I needed to use the metric result as the output of a shader pass and without the square root there is a noticeable precision loss due to the frame buffer. here is the “clean” float4-version if you have a use for it:

the result of df(A,B) is the squared difference of (A.x, B.x), (A.y, B.y), (A.z, B.z) and (A.w, B.w). maybe you have another idea to speed it up?[/QUOTE]

I didn’t use your function because my cg compiler was complaining about using too much registers. For some reason, if you pass matrices to functions, the compiler uses more registers than if you just use vectors.

I had to modify the original params (2, 4, 3) too. For some reason they provide wrong results for my pixel art test images. So I tweaked them and found some good triplets: (17, 20, 3) or (3, 6, 1). The default now is (17, 20, 3).

I’m using the sqrt return because I need to compare accumulated distances. For example, If I had 4 color distances d1, d2, d3 and d4:

d1 = 1, d2 = 6 d3 = 4, d4 = 4

d1^2 = 1, d2^2 = 36 d3^2 = 16, d4^2 = 16

so, if I use the sqrt, then (d1 + d2) < (d3 + d4). On the other hand, if I don’t use the square root, then (d1^2 + d2^2) > (d3^2 + d4^2).

#196

oh, I see. well I was curious if it even makes a difference so I did a little benchmark with both variants. turns out it doesn’t matter. calculating the difference individually or doing 4 at once gives me the same performance. anyways, seems weird that the blue factor in your chosen default triplet is so low. makes me wonder if it doesn’t make more sense then to completely throw the blue channel out of the window and forget about the whole trick with taking the average red value. just calculating the following shouldn’t make a huge difference then:

sqrt(17*(A.r-B.r)^2 + 20*(A.g-B.g)^2)

well I tried both triplets in ScaleFX and they seem fine. (3,6,1) has a hard time though with ScaleFX-hybrid on the DKC intro screen, notice the yellow lines inside the big letters: http://screenshotcomparison.com/comparison/164649

I think I stay with the original triplet, for ScaleFX it seems it is the most balanced choice.

#197

Even if blue is low, it needs to be there, otherwise some artifacts appear.

Anyway, I think due to the algorithm used, some triplets are better than others. It isn’t absolute I think.

#198

Hylian I have a question - I’m using preset of 2xbr hybrid v4 (2x, nearest) + crt hylian (don’t care, nearest) and this is by far the best looking shader combo that I have ever saw (the character models are smoothly filtered and looks natural with crt shader) BUT I have noticed that almost every xbr shader gives me some weird jittering visible on the edges of fonts, character models, backgrounds etc - something like the clouds from the .gif from last page. It seems when something is moving or there are some flashing behind, sprites in front gets disorted on the edges a bit - why is that? Xbr hybrid have this much more visible than standard xbr, but hybrid in my opinion looks superior due to preserving original outlines and curves. Maybe the problem is related to this point filtering that You mentioned before? I don’t have a clue where even I can see this option in RA.

#199

Have you tried the preset crt-super-xbr.cgp inside ‘crt’ folder? For me it looks superior to any hybrid. The hybrid versions are obsolete now, I quit its development because the transition between xbr and reverseAA isn’t seamless. Maybe this is the cause of some artifacts you’re seeing in some edges. Super-xBR replace hybrids in any situation.

#200

Yeah, I actually tried this before and today just to make sure - it looks messed like hell for me, just look at pictures [

My settings in RA are integert scaling OFF, full screen stretched (but tried also with integer ON and scrreen ratio was okay, but the artifacts still existed as shown above) HW bilinear filterin ON (setting this OFF gives me no better image at all) and of course all settings from preset crt-super br.cgp left untouched by myself, just apply and go. Ratio is set to 16:9 and my screen display is set to its native resolution 1360x768 Samsung TV.](http://postimg.org/image/f2obyumw1/)

#201

Some questions:

1- Does it happen only in gba games? Could you try a snes game and tell me if the same artifacts happen?

2- Turn hw bilinear OFF, integer scale ON, aspecto ratio CUSTOM, and tell me if the same artifacts happen.

3 - What’s your system? HW and SW.

#202

1-Tried snes game “Hyper Metroid” on settings that You mentioned above, the results are even worse (same warped graphics, but the aspect ratio is a garbage right now, some shots below): 2- Again, the screens shows it all : / 3- Intel Xeon 3ghz, dual core (same as well known core 2 duo E8400), Radeon HD7770 1GB GDDR5, 4gb RAM ddr2, Windows 7 x64 Ultimate, screen display is Samsung TV 28" 1360x768, also set to this native resolution in Windows.

#203

I don’t have that hack.

Anyway, I tested Super Metroid here and couldn’t reproduce those artifacts:

It tested with many aspect ratios and didn’t find any of those artifacts. I can’t reproduce it.

#204

It looks just amazing! This is the effect that I’m looking to achieve - but what could be wrong on my end? Don’t have a clue sadly… BTW - this is hack called HYPER METROID, and as a gamer I strongly reccomend trying this out as it is (yes, I;m a fan of metroid series) by far the best 2D metroid game to date, sorry for little offtop but You just have to try it out, it’s like a powered up sequel to gaming godness - super metroid.

Hmm maybe I should change some RA specific graphic options in some way to get proper working effedt? Right now using opengl video driver, could this be the culprit? Or maybe You can think of anything else on this end?

#205

No problem with Hyper Metroid too:

I’m using OpenGL too. No idea what’s the problem.

#206

Tried d3d mode, but it crashed my RA xp I had to manually switch back to gl option in notepad settings file to even start up again. Ehh such a great shader and thats annoying as hell that I don’t even know why something is messed up - maybe my radeon gpu doesnt have some function/ instruction set to properly coop with super xbr? Every from this shader family is giving me same artifacts - I don’t know what is handled there so differentely that regular xbr’s works and super in not as it should.

#207

[QUOTE=belmont1990;38113]It looks just amazing! This is the effect that I’m looking to achieve - but what could be wrong on my end? Don’t have a clue sadly… BTW - this is hack called HYPER METROID, and as a gamer I strongly reccomend trying this out as it is (yes, I;m a fan of metroid series) by far the best 2D metroid game to date, sorry for little offtop but You just have to try it out, it’s like a powered up sequel to gaming godness - super metroid.[/QUOTE] Off topic, but I finished Environmental Station Alpha yesterday and it’s a great notmetroidbutit’smetroid game: http://store.steampowered.com/app/350070/

#208

Hi, i’m trying to load crt-super-xbr.cgp, but nothing happens, others like crt-easymode-halation, are working.

edit:I’m in lubuntu 16.04