Improving bilinear filtering (upscale vs downscale)

RetroArch Nintendo consoles
#1

I don’t know if this has been talked about before, but the other day I was thinking about how unusually muddy and blurry the bilinear filter looks, when compared to 3DS VC and the GBP, those upscale to 360x240 and 570x380 respectively that definitely needs filtering but how do they manage to make it not look so blurry? Well, I was doing some tests with upscaling and downscaling images in an image editor, and what I found is that downscaling provides a sharpish look despite the filtering.

Upscale(current) vs. downscale:

What I did was upscale with no filtering to 720x480 and downscaled to 606x404 with bilinear, what RetroArch does is upscale 240x160 to whatever size you want causing the muddy, blurry look.

I’m not saying this is exactly what Nintendo does but it does provide an improvement. The idea of rendering the GBA games to 720x480 and then downscaling sound like they might hurt performance, I thought about it and figured that if this is what Nintendo does(or something like this) then they obviously didn’t break compatibility.

Here’s a full 3DS sized comparison:

I already tried looking at the source and did some testing but nothing showed results.

If anyone would like to see how it would look, you can try finding something called “Interactive multi-game demo disc ver. 16” That demo disc contains a GBA emulator for Gamecube, it’s quite impressive, it’s used to run a demo of Mario vs DK, but can be easily edited to use any game you want. The emulator is fixed at one view size(2x GBA resolution) but filtering is still softly present and to show it; all you have to do is run it in Dolphin and use fullscreen, you might notice Dolphin won’t scale it to a proper size which helps showcase the filtering and it will still look very sharp.

#2

That’s a terribly inefficient and unlikely way to go about filtering an image. The short version is you don’t need to perform extra scaling operations just to filter an image, you’d just filter the image. Any graphics hardware worth a damn can do some simple anti-aliasing for practically no cost to performance.

Here’s Nintendo doing exactly that on the GameCube, in the official hardware demo distributed with the SDK. Unfiltered first, with anti-aliasing second.

No extra scaling needed; you just switch on anti-aliasing of video and the GameCube (or Wii, or most likely, 3DS) gives you a smoother, blurrier, filtered (choose your preferred adjective) image.

Here’s Mario vs. Donkey Kong, per your suggestion (cute border!):

So what’s going on? A couple of things, some more interesting than others. There’s a 2* scale on the Game Boy Advance image, most likely point. There’s a fun (very faint) “screen door” overlay which attempts to (and does a pretty good job of) simulating the distinct pixel borders on the GBA and other Nintendo handhelds’ screens. This is the same complaint you sometimes hear about the 3DS XL, where the effect is even more prominent than usual. The same thing is done by one of the Game Boy overlays in RetroArch. Lastly, there’s anti-aliasing on the whole image, as discussed above, which does some smoothing without obliterating the image as a unilaterally applied bilinear scale would (does).

The only other thing I’d mention is that Dolphin probably has its own filtering in addition to (or instead of, if you’re hardware rendering, I assume) Nintendo’s, so it’s probably not a reliable testbed for what filtering looks like at non-integer scales. To be clear, all the shots I’ve posted up to this point are from Wii hardware, not Dolphin.

All this said, I do think there’s a major improvement that could be made to the “bilinear” mode in RetroArch. Basically, wherever possible, you’d do a point filter to the nearest integer, then bilinear from there. So if we take something like the Game Boy/Color, which we’ll assume someone is scaling to the full-height 534480. Instead of making this a one-step bilinear scale, you’d do a point scale to 480432 (3* scale) and then do the bilinear filter up to 534*480. This will result in much sharper pixels (simulated shots, not from hardware).

To demonstrate, you start with this:

You point filter to this:

And then you bilinear filter to this:

Because the alternative is this, which I think is what you’re seeking to avoid; scaling right from 160 to 534:

But eh, maybe that’s a matter of taste.

Conclusions: you seem to want the option in the Wii/GC ports to enable anti-aliasing, and possibly a second option or a replacement of current behavior RetroArch-wide (i.e. across all platforms) with a combined “point-then-bilinear” scaling which keeps pixels somewhat sharper than a full bilinear scale. Both of these things seem like nice additions to me, as I’m sure there’s people who’d like them. Might be worth filing an issue on GitHub.

#3

Good point but I made sure there was nothing going on that I didn’t know about, check this: It’s more or less the same thing, there’s the obvious difference of brightness but the pixel lining is pretty much the same. That’s the same Mario emulator on the left Dolphin upscaled to 808x548 and on the right a point filtered 480x320 bilineared to 808x548

I just received an s video cable for my GC and tried out a comparison between RA and the Game Boy Player side by side and from what I could tell it’s most likely a point filtered upscale from 480x320 with bilinear to the “full” screen size.

I submitted the issue and tried to make it as clear as possible.

#4

there is already a shader for that in the main dir of the shader repo: https://github.com/libretro/common-shad … ilinear.cg

#5

Unfortunately, Wiis can’t do any shaders, so it would have to be hardcoded :confused:

#6

Oh boy! It’s been a while since we’ve had one of these fun technical talks.