A better SNES hi-res blend shader?

RetroArch Additions Shaders
#1

As most of us might know, some SNES games run in a mode called pseudo-hires, which a few games such as Jurassic Park and Kirby’s Dreamland 3 used to create transparency effects. Thing is, the parts of the screen that were intended to be transparent required blending from the TV side, otherwise it results in said parts looking like a bunch of disconnected vertical bars.

There are several shaders that achieve this blending effect. Off the top of my head, I know GTU, snes-hires-blend, and Blargg’s NTSC filters can do this. Standalone Snes9x also has an option within the display settings for this, which appears to just enable a filter similar to snes-hires-blend.

The problem with these shaders and filters, however, is that they tend to horizontally blur the image, either altogether or only on screens that implement pseudo-hires. From what I know, the SNES did no filtering on its own, as it essentially came down to composite cables and the typical blurriness of CRT TVs to do that, so adaptive blending that blurs whenever pseudo-hires is detected is definitely not accurate. GTU is more consistent, as when set to a horizontal resolution of 256 pixels, it just blurs everything regardless. However, would it not be nice to have both a sharp picture AND the intended transparency effects?

Well, I was messing around with an ancient version of Snes9x for fun (version 1.39 to be exact), and lo and behold, not only did it have support for pseudo-hires, but it blends it perfectly with zero horizontal blur, something modern Snes9x and no shader I know of manages.

Please see my attachment for a comparison of the various ways pseudo-hires is handled across several SNES emulators, with and without blending.

So, my question is: does a shader exist that can replicate what a version of Snes9x from 2002 does? If not, is it possible to create one?

#2

Yes, I also would like to see improvements on this side, all sega genesis games for example need this kind of processing and they look extremely blurred. Using the internal SNES shader should be desirable, but if for any reason is not possible in avisynth there is a function for similar situations (cleaning up interlaced left overs) called Vinverse. …

This is the code:

function VinverseD(clip clp, float "sstr", int "amnt", int "uv")
{
uv   = default(uv,3)
sstr = default(sstr,2.7)
amnt = default(amnt,255)
uv2  = (uv==2) ? 1 : uv
STR  = string(sstr)
AMN  = string(amnt)
vblur  = clp.mt_convolution("1","50 99 50",U=uv,V=uv)
vblurD = mt_makediff(clp,vblur,U=uv2,V=uv2)
Vshrp  = mt_lutxy(vblur,vblur.mt_convolution("1","1 4 6 4 1",U=uv2,V=uv2),expr="x x y - "+STR+" * +",U=uv2,V=uv2)
VshrpD = mt_makediff(Vshrp,vblur,U=uv2,V=uv2)
VlimD  = mt_lutxy(VshrpD,VblurD,expr="x 128 - y 128 - * 0 < x 128 - abs y 128 - abs < x y ? 128 - 0.25 * 128 + x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
mt_adddiff(Vblur,VlimD,U=uv,V=uv)
(amnt>254) ? last : (amnt==0) ? clp : mt_lutxy(clp,last,expr="x "+AMN+" + y < x "+AMN+" + x "+AMN+" - y > x "+AMN+" - y ? ?",U=uv,V=uv) 
return(last)
}
#3

there is a pretty easy solution which I already recommended in the mdapt shader thread. just use this cgp preset as a base for further shader configuration, no need for a special shader:

shaders = ?

shader0 = stock.cg
filter_linear0 = true
scale_type_x0 = source
scale_x0 = 0.5

...
#4

The only problem with just shrinking everything by half is that games that use the high-res mode for text get blurry (IIRC, Marvelous and Seiken Densetsu 3 are two examples). Not a huge deal, but it’s worth considering.

IIRC, the reason snes9x’s high-res blending is so nice is that it only blurs the layer that has the pseudo-transparency and leaves the other layers intact. This isn’t really possible in the frontend, but could potentially work as a core option, if it’s still available in recent code. However, this isn’t any more accurate to what an SNES did than detecting 512 res and squishing it. The “accurate” options are GTU/NTSC.

#5

Halving the image leads to several issues beyond what hunterk noted. As I said the above Vinverse post effect is 95% spot on, just compare the images, and is very light on resources. You don’t need to wait for core updates nor use dirty blurry filters like GTU/NTSC. I don’t know C but if you want me to convert the above to infix notation just tell me if you are willing to turn it into a shader.

#6

Sure, I’d be happy to give it a shot, though I can’t really make any promises. It might also help if you have a plain-language explanation with it, like “it takes the pixel to the left and…”

If I get stuck, maybe aliaspider will be willing to help out, too.

#7

it’s probably possible but just wanna mention as an experienced avisynth user that these functions aren’t so easy as they might look at first glance. these functions are build upon the shoulders of giants. as you can see vinverse calls some functions like “mt_convolution()” or “mt_makediff()” which are part of the MaskTools library for avisynth. these are probably steps which you have to separate into multiple passes to recreate the process of using multiple custom masks. so I’m a little doubtful about the idea.

http://manao4.free.fr/mt_masktools.html

could you explaint that further? I forgot about the hires text in a few games but apart from that I don’t see the problem with this method. :confused:

#8

Whoever is the moderator who deleted my posts, please send me back my own posts for which I invested a considerable amount of time to research (a step by step detailed explanation of the vinverse function). Thank you.

#9

there was a server crash so a lot of this discussion is gone, my last post is also lost. but I try to recreate it with this one:

well, after my argument with dogway I compared all the mentioned methods we have to deal with snes hires games and I actually had to find out that the snes-hires-blend.cg shader indeed has problems. the idea behind is a little bit too naive, it just blends the picture horizontally instead of using pairwise disjointed pixel pairs. this is why it makes the image blurry. I wrote a new shader which should replace the old one on the shader repository. if you compare the final return lines of both version you can easily see the problem:


snes-hires-blend.cg - old version
return IN.video_size.x > 256 ? lerp(l,r,0.5) : r;

snes-hires-blend.cg - fixed
return (IN.texture_size.x > 256) ? (fp == 0 ? lerp(c,r,0.5) : lerp(c,l,0.5)) : c;

Download

and now for some comparison screenshots on all the methods with Jurassic Park. you probably need to zoom in to see the difference between the old and fixed version of the shader:

this is the avisynth script I used for the last shot:

imagesource("screenshot.png")

pointresize(width, 2*height)
turnleft().converttoYUY2(matrix="PC.601")

vinverse()

converttoRGB32(matrix="PC.601").turnright()

the cgp method and the fixed version give optimal and equivalent results. vinverse is the worst option though, there are very visible halo effects and the transparency is inconsistent. so there is really no need to implement this as a shader, just stick with the cgp method or the fixed snes-hires-blend I wrote.

#10

We didn’t erase anything, we had a server crash an we had to restore to an earlier backup

#11

@Sp00kyFox ohhhh, good call, dude. That makes a lot of sense and looks great. I just pushed it up to the repo a little bit ago.

#12

The vinverse version is the next best thing after the fix you supplied. It has ringing because we haven’t touched the sstr parameter which is defaulted to 2.7. The cgp version as I said unables you to use any other shader on top, so thanks for the snes-hires fix, this can be great for sega genesis games.

Too pity I can’t recover my post, libretro forum is not even registered by google nor waybackmachine.

edit: Probably the vinverse method is still the best alternative, the fixed shader has problems with genesis games.

no snes-hires…with snes-hires…with snes-hires fixed

Maybe we need a gen-hires shader?

#13

even with sttr=0 vinverse creates a worse result than the unfixed version of snes-hires-blend.cg. I really don’t see why you think this is a good solution but feel free to implement it.

of course you can combine other shaders with the cgp method, the whole purpose of it is that it is a preset to modify it further. the problem is actually with the other solution. because snes-hires-blend.cg doesn’t change the resolution the picture still remains at this unusual hires-mode (512x224 instead of 256x224). the pseudo transperancy patterns are now merged but pixels are still doubled in horizontal direction and this throws off filters like xBR or mdapt which expect that the pixelart isn’t scaled but at it’s original resolution. therefore I suggest using the cgp method.

of course it does, duh. the only game the genesis uses a high resolution mode for is Sonic 2 in versus mode and not for the purpose of pseudo transparency. this thread is about dealing with the snes hires mode alone which is a specific and easy case since apart from the vertical transparency lines everything is just doubled horizontally (count the pixels!). this is not the case for snes games who doesn’t use this mode or even other systems. I slowly get it why you think vinverse would be such a good solution. but in the general case it’s a lot more complicated because you can’t locally decide whether a vertical line pattern is an actual part of the graphic content or supposed to be pseudo transparency. and in that regard vinverse acts too dumb, it can’t distinguish between those two things.

no, we don’t since there is no hires-mode on the genesis. what you want is a filter which gets rid of dithering and pseudo transparency in general, in other words mdapt or gdapt (hence the abbreviation). and no, they don’t work perfectly. but that’s the whole problem with it, it’s always a tradeoff between detection and error-proneness. if you want accurate behavior like the devs intended it you need to use crt-shaders. again to prove my point here are some more comparisons for good measure:

#14

We are discussing about pseudo-transparency, it just happens that in snes “pseudo-transparency = hires”, so snes-hires works for it. I know by a matter of fact megadrive games are not hires. I would think you could understand what I meant with “gen-hires”, it was not to deal with high resolution megadrive games (obviously because there are not) but the pseudo-transparency effects that plague their graphics. And when I say pseudo-transparency I mean pseudo-transparency, not dithering patterns.

As you could see on the jurassic park shots the grass pattern stays untouched while on the lion king the sky is (IMO) severely degraded. Using scanline shaders this dithering blends fine enough and pass by unnoticed, but the pseudo-transparency really stands out. I hope you get the point I want to make.

#15

by your obvious expectation that the snes-hires-blend shader would be a solution for pseudo-transparency in general I don’t think that you really quite understand the differences between these scenarios. otherwise you wouldn’t have came up with the idea to use it on genesis games in the first place despite your supposed better knowledge about it.

again, what you want is mdapt (“merge dithering and pseudo-transparency”) or gdapt. and I didn’t designed them as a solution for two problems but the way to detect those patterns just effects both areas. dithering and pseudo-transparency are using the same patterns, mostly checkerboard and vertical lines. therefore you can’t seperate them as strictly as you think, they are related. with this in mind you can’t say that you want to get rid of pseudo-transparency but keep the dithering, this doesn’t work.

actually there is no pseudo-transperancy going on in this lion king scene. the landscape is dithered as well as the sky just with different patterns, there is no graphic underneath it. which again makes me wonder if you even understand the terms you use. a vertical line pattern can be dither and a checkerboard pattern can be pseudo-transparency just as the reverse case.

#16

By the tone in your wording you look excited, I’m here discussing a thing that the OP started, not on a crusade so I’m not going to discuss this point if you allow me.

I’m not sure if at this point we need to spoon-feed every single argument. Put a name to this:

I understand that it’s not as easy as with hires pseudo-transparency, but as maths and programming illiterate as I am, I’m already testing with masktools to come up with a code that puts my thoughts into paper.

#17

Here I go, one hour quick fiddling. It’s certainly doable:

uv   = 3
sstr = 2.7
amnt = 255
uv2  = uv
STR  = string(sstr)
AMN  = string(amnt)
clp=last

vblur  = clp.mt_convolution("50 99 50","1",U=uv,V=uv)
hblur  = clp.mt_convolution("1","50 99 50",U=uv,V=uv)
masky  = mt_luts (vblur, hblur, mode="avg", pixels=mt_circle(1)+string(0), expr="x y - abs 2 > 255 0 ?" )
vblur = mt_merge(clp,vblur,masky)

# vinverse function sharpening routines
vblurD = mt_makediff(clp,vblur,U=uv,V=uv)

Vshrp  = mt_lutxy(vblur,vblur.mt_convolution("1 4 6 4 1","1",U=uv2,V=uv2),expr="x x y - "+STR+" * +",U=uv2,V=uv2)
VshrpD = mt_makediff(Vshrp,vblur,U=uv2,V=uv2)
VlimD  = mt_lutxy(VshrpD,VblurD,expr="x 128 - y 128 - * 0 < x 128 - abs y 128 - abs < x y ? 128 - 0.25 * 128 + x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
mt_adddiff(Vblur,VlimD,U=uv,V=uv)
(amnt>254) ? last : (amnt==0) ? clp : mt_lutxy(clp,last,expr="x "+AMN+" + y < x "+AMN+" + x "+AMN+" - y > x "+AMN+" - y ? ?",U=uv,V=uv)

It’s hacky, basically get the difference between a vert blur and a horiz blur. You can do it wiser, compare a single 4x1 pixel window against the pixel above, if it matches… blablabla

this is the result:

#18

you’re making mistakes with your statements about the discussed methods and with the usage of terms, so don’t complain if someone corrects and explains it to you.

that’s a correct and well-known example of a pseudo transparency pattern as opposed to the lion king scene. like I already said, mdapt and gdapt can handle both cases:

I can’t reproduce your result, please post a complete script. converting the image to yv24 (so it can be used with masktool functions) and applying your script does nothing:

imagesource("screenshot.png").converttoYV24(matrix="PC.601")
#turnleft() # doesn't help either
#your script

if I understand your approach correctly you’re just ignoring checkerboard patterns. then again they also can be used for pseudo-transparency (like in SFA3).

#19

The way you talk to me is a bit aggressive, just letting you know…

You are lost in a war of terms, pseudo-transparency.… ‘vertical’ dither… in the end of the day it’s the same whatever is the goal it’s used for. They are interleaved columns of the same (vertical) pixels (for dithering reasons) or related (vertical) pixels (for transparency reasons). And the best proof for showing they are the exact same thing is that my code works on both.

Here the image:

Here my code: http://pastebin.com/DuBnH8S9

The reason I exclude other types of dithering patterns (and hence your gdapt and mdapt shaders) is because 1) they don’t expose a problem when used along CRT shaders on top, and 2) either your shaders or vanilla vinverse leave some very ugly and random dots that stand out too much, even with CRT shaders on top.

#20

well that was exactly my point, right? I wasn’t the one who persisted to strictly separate between these two if you can remember :wink:

can’t say that it works. ratioresize() is a custom method of yours? anyway since I have example pictures in the original resolution (320x224) I guess the line is unnecessary. script loads fine but like before it doesn’t seem to do anything (apart from a little smearing here and there).

https://imgur.com/a/5Jpw5

CRT shaders already have the ability to blend dithering patterns anyway because of the pixel bleeding they emulate. it doesn’t matter what pattern is used. try for example the crt-royale shader and play around with the beam_horiz parameters. the whole point of an anti-dithering shader is to get the dither blending without the need to emulate a CRT which then allows you to use scalers like xBR which you can’t use on top of CRT shaders.

well that’s the fundamental problem of this task, balancing between detection-rate and error-proneness. that’s why you have a lot of parameters to tweek for yourself. a filter can’t decide locally wheter a specific pattern is supposed to be dither or actual graphical content. you’ll always have blending errors or dithering which wasn’t detected. restricting it to vertical line patterns doesn’t avoid that (think about text, pillars or the hud in your lion king screenshot). I can only repeat what hunterk said, if you want flawless/accurate behavior you’ll need to use CRT/NTSC shaders. the snes-hires case is a special case where you don’t need to put effort in any kind of detection because you can do the same thing on every part of the picture and it works perfectly.

just wanna add, this doesn’t mean I would advise against implementing your own anti-dithering shader. new approaches can always be helpful. only your motivation to do so seems questionable. just forget the thought that there is a perfect and error-free solution for this problem.