xBR algorithm tutorial

I really can’t remember where I got this.

But when I need a pixel art for tests, I’m used to search in www.pixeljoint.com

Great you got it running thanks to hunter’s tip.

I really don’t know why there are some much incompatibilities among GL shader treatment by different drivers. It’s a paing in the ass.

But, I intend to buy a new notebook to investigate and turn my shaders more compatible.

have a question about the xBR LV3 cg implementation. I usually prefer to activate bilinear filtering in retroarch to soften the rough edges of interpolation shaders, but in this case it makes no difference. which makes me think that the output resolution is a fixed multiply (which is probably higher than my monitor), at least the scale paramter in the cgp seems to have no effect. I tried to add bicubic-normal.cg as a third step, but I can’t get it to work right. any ideas?

This multipass LV3 xBR is scale agnostic. You don’t need a third shader to soften it.

But, if you think the edges aren’t soften enough, you can change this line in the last pass vertex shader:


    co.delta = float2(IN.video_size.x/IN.output_size.x, 0.5*IN.video_size.x/IN.output_size.x); // Delta is the thickness of interpolation

What it does, for example, it grabs the inverse of output resolution, considering the input. So, if you are scaling by 4x, then delta becomes float2(1/4; 1/8). You can put a constant in that equation and try bigger values, which will soften more the output. For you information, SABR uses float2(0.4, 0.4) in any scale. You can adjust it for your tastes. But beware, 0.5 is too soft! and 0.1 is too rough! So, use the range [0.1;0.5] to find your sweet spot.

meh, not quite what I’m looking for. I don’t know, just looks a little bit crispy at some areas, which bilinear filtering usually takes care of. here is an example of what I mean (its turtles4 on the snes) and how it’s supposed to be with softening (used a graphic tool):

would like to soften it up a bit maybe with a third shader. didn’t see much change with the delta parameter.

Then, what you need is a blur shader.

Try those in mudlord folder. Inside them there’s a parameter you can change to control the level of blur.

yeah I actually know how to combine shaders xD but now I’m back to the problem I already had as I tried bicubic as a third step. it seems as if any shader I use afterwards is totally ignoring the image result of xBR and instead uses the original unfiltered one. I don’t know if it is related but I’m also getting this warning in the log if I’m using the shader (with the unmodified cgp):

RetroArch [ERROR] :: [D3D9 Cg]: Fragment error: E:/…/xbr-lv3-multipass/xbr-lv3-pass0.cg(185) : warning C7050: “px0” might be used before being initialized E:/…/xbr-lv3-multipass/xbr-lv3-pass0.cg(185) : warning C7050: “px3” might be used before being initialized E:/…/xbr-lv3-multipass/xbr-lv3-pass0.cg(185) : warning C7050: “px1” might be used before being initialized E:/…/xbr-lv3-multipass/xbr-lv3-pass0.cg(185) : warning C7050: “px2” might be used before being initialized

Those px# are completely ignored in second pass if they aren’t initialised in the first pass.

But, if you want get rid of those warnings, initialise them after this line:



lin0 = lin1 = lin2 = lin3 = bool4(1, 1, 1, 1);

px0 = px1 = px2 = px3 = bool2(0, 0);

I doubt it’ll fix anything.

yeah, third shader is still using the unfiltered image. it’s like xbr-lv3-pass1.cg prevents its image from being send to the next step. maybe it’s a issue with retroarch and maister has to take a look into this.

Post the retroarch log here and maister or squarepusher can see what’s happening.

to get the log:

If you use phoenix, it’s File -> Show Log if not, --verbose on command line.

There’s another test you can do:

put ddt.cg as third shader and change the second scale to 2x.

ok, I feel kinda stupid now. this way it works fine with additional shaders. thanks for the help again.

edit: played a few snes games with it, looks gorgeous. good work hyllian!

yeah i noticed that a while back. If you don’t define the xbr scale to something concrete (anything but “don’t care”) then it gets ignored on subsequent passes. Seems to be the case for all xbr shaders I have tried. Also you might want to try advanced aa as a linear pass after xbr. I really enjoy that look and it solves the issue you were wanting to get rid of. Also 4xsoft linear looks good too with same effect. Just some ideas to try and find your best preference.

Seriously though this looks great. I think it is safe to say that this has become the most advanced project of its kind. Looks even better than the “depixelizing pixel art” algo IMO. Hopefully the gl issues can be worked out soon too so I can use these again (since I ditched windows)!

I’m getting something similar to Awakened’s screenshot with displaced pixels with both DirectX and OpenGL (OpenGL is worse though.)

picture:

I do, at least, have a theory as to the source of the problem. Some months ago I was trying to understand/improve upon the reverse-AA algorithm (which is a story for another day) and teach myself cg in the process, when I stumbled upon something odd: When I borrowed the standard code many existing shaders (including XBR) use for referencing the surrounding pixels, and tried to make some very, very simple shaders that just shift the picture around (e.g. return the value of the neighboring pixel that’s up one, left one) I sometimes got completely messed up results at 4x. I speculate that either the tex2D function is not always implemented as advertised, or it just doesn’t do what we think it ought to do.

It works exactly as I expect for the system (PS3) I work. And I just use the Cg Specs from nvidia itself.

I just don’t know what other systems are doing in their drivers, because these shaders should work properly.

And yes, most shaders need to look neighbor pixels to the central one. And tex2D only needs two parameters, the first is the texture to look for and the second is the address of the pixel you’re looking for in that texture. As many shaders use nearest neighbor sampling, there are some texture addresses more prone to errors than others. These addresses are generally at the pixel borders, where a miserable small digit bad rounded may induce tex2D to return two different colors, depending on how the drivers implement tex2D.

I think Maister can explain this much better than me.

Well, couldn’t it be the Cg compiler compiling to non conform HLSL or GLSL?

I’m in love with the mdapt + xBR preset combination. It makes Mega Man 2 look so amazing! Great work on xBR, easily my favorite shader. Now mobile GPUs just need to get up to snuff so I can enjoy this on the go.

That’s great. What’s your system (CPU, GPU, SO and DRIVERS)? Retroarch is running in OGL or D3D mode?

Intel i7 3770K nVidia 660GTX TI 16GB RAM

Debian 7.1 stable nVidia 325.15 drivers(latest from nVidia) OpenGL mode

Seems to run everything fullspeed that I threw at it(even bsnes), I’m also running the latest cores and retroarch from git. Do note that the mdapt 5xBR combination in the dithering/mdapt-4p folder crashed RetroArch with a segmentation fault when I tried to load the cg shader, but loading the GLSL converted shaders work just fine. Not sure if it’s RetroArch, the shader, or maybe even the CG compiler I have installed.

Ok, nice machine! I really don’t know what causes nvidia gpus be so picky with these shaders…