Welcome to DIY Integer Scaling.
So I did some thinking about resolutions and aspect ratios, and this is what I concluded: for ideal scaling, the x- and y-resolutions must be integer-scaled, though the multipliers don’t have to be the same, and the final resolution should be in a 4:3 aspect ratio, so that it looks correct on the square pixels of an LCD screen. Here’s the formula:
(Xres * Xmult) / 4 = (Yres * Ymult) / 3
Xres and Yres are the native x and y resolution output by the console, and Xmult and Ymult are the scaling factors for the width and height, respectively. Solving for the mults gives you a ratio, so the goal is to find integer values for both that satisfy the equation. For example, Contra runs at a native 256 x 224, so the equation yields 7(Ymult) = 6(Xmult). This means Ymult is 6 and Xmult is 7, so the optimal viewport is 1792 x 1344.
See the problem? You’d need a 4k display to actually display that (I guess 1440p would work). And it only gets worse; the genesis runs at 320 x 224, requiring a 14x/15y(!) scale. It isn’t practical, or even possible. So here are some compromises:
-
Use a shader. The uneven pixels are much harder to see, and should resolve the shimmer artifacting. Bilinear is really fast, but just makes things fuzzy. CRT shaders and the like mask scaling artifacts quite well in my experience, but they might be too expensive for a vWii (also a matter of taste). Nearest neighbor doesn’t solve the problem, and neither will “sharpening” filters, because they don’t hide the uneven sizes of the pixels by blurring them, like most other shaders. I don’t know if the vWii works with shaders besides bilinear, so if it doesn’t, this means you’ll just have to pick bilinear if you take this option. Follow the instructions in (3) for resolution setup.
-
Ignore aspect ratio. You can simply integer scale the native resolution to a size just under your screen resolution. A 4x scale for the NES works at 1080p (1024 x 896). This gives you black bars on the sides and top/bottom, and you’ll have to mess with the viewport x/y to center it. No shimmering with nearest neighbor filtering, but it’s slightly…squashed looking. It’s better to scale the x and y separately, but instead of aiming exactly for 4:3, settle for “close enough”. 1280 x 896 is a decent approximation for Contra (x5, y4). This probably gives the best results overall if you want sharp scaling, since it has no shimmering, good size, and decent aspect ratio.
-
Ignore shimmering. This is the easiest to implement…just set the viewport to 1440 x 1080, turn off integer scaling, and let 'er rip. It’ll be exactly 4:3, and you’ll only have black bars on the left and right, but any movement on screen is gonna get messy—now both horizontal and vertical motion. This should work for all cores. You’ll have to set the viewport x, but not y, since it will fill the vertical space of the display.
Note that (1) and (3) are essentially the same, plus or minus a shader. I’ve included more detail than strictly necessary so that you can (hopefully) apply this to other resolutions as necessary, and so that other people can fact-check me etc. (I did test these setups, though, so I think this is all correct).
I highly recommend the 240p test suite for checking for scrolling issues etc. There are versions of it for most popular consoles. It’s also easier to test these out on a normal computer, since you can keep the cfg file open in notepad and easily edit it between tests, instead of running an SD card back and forth between two devices and constantly rebooting the Wii U. You can even hook a laptop to the tv and see it on the display you’re using! Just need an HDMI cable.
tl;dr: No shimmering, correct aspect ratio, sharp pixels; pick two. The bit at the end of (2) is what I would use if I didn’t love my CRT shader/actual CRT to bits. You’ll have to do the math for other consoles with different output resolutions, though, so it’s more work if you want more systems running.
Have fun tinkering with your setup! I hope you like math.