ScaleNx - Artifact Removal and Algorithm Improvement

Download: … b887d5f79f

I noticed that the scaleNx filters are missing from the shader repository. Even if they are outdated by more superior edge interpolation algorithms today I still find them useful for low resoultion games to remain that pixelated picture while still give it more refinement.

I implemented both scale versions (2x, 3x) based on the pseudocodes presented here:

But there were two thing I wasn’t satisfied with. First, both versions are creating weird little artifacts which are especially annoying with a low resolution source image (well known problem of this algorithm). And second, while scale2x smoothes out 45° edges just fine the 3x variant has its problem with them. You can see both issues is this comparison shot:

So I created a post processing shader (ScaleNx-Cleaner) which looks for these artifacts and removes them by replacing those pixels with the surrounding ones. And I made an advancement to the rule set of the 3x variant called Scale3xEX which should make the original obsolete:



Wow, big improvements. Great job! You need me to push these up to the common-shaders repo?

yeah, that would be nice. thx!

Finally pushed these up to the repo. Sorry for the delay.

no problem. It’s like a bad habit but I’m already working on an improved version. I wasn’t really satisfied with removing the artifacts afterwards instead of avoiding them right away. so I’m gonna up a new version soon. this will make the cleaner shaders obsolete.

ok, got a new version, no need for the extra cleaner pass anymore. I also modified the algorithm so it properly works with equality thresholds. here is the download:!yYohhDCA!jgEe41OL5rueTLzSP3GMYIBaGgc7sjyfy95-8hb8KXc

for the interested reader I’m gonna explain both algorithms:


if anyone used the original scale2x or EPX one can clearly notice single outstanding pixels here and there which are very annoying and the only drawback in my opinion for this method which otherwise pretty much keeps the pixelart style intact.

ok, why does these artifacts (see the first post in this thread for an example screen) come up? first, let’s give those pixels some names:

  A B C		  E0 E1
K D E F L		E2 E3
  G H I

where E is the central pixel and E0 to E3 are the 4 subpixels of E (and A0 to A3 are the subpixels of A and so on). well, let’s say E0 is the artifact pixel. what happened is that the condition to replace E0 is met simultaneously with the ones of B2 and D1. this implies (look at the original algorithm for that) that the surrounding pixels of E0 (which are E1, E2, B2 and D1) are all of the color E and therefore E0 itself is an outstanding pixel because it isn’t E anymore.

the way to fix this is obvious. just check if the conditions for B2 and D1 are also met, if that’s the case then don’t touch E0. well here is the original and the improved algorithm in pseudocode for comparison. note: to allow the algorithm to work correctly with threshold comparisons the so called parent condition is slightly altered but still aquivalent to the original one in the case of true equality checks.


E0 = B=D & B!=F & D!=H ? 0.5*(B+D) : E
E1 = B=F & B!=D & F!=H ? 0.5*(B+F) : E
E2 = D=H & B!=D & F!=H ? 0.5*(D+H) : E
E3 = F=H & B!=F & D!=H ? 0.5*(F+H) : E


E0 = B=D & B!=F & D!=H & (E!=A | E=C | E=G | A=J | A=K) ? 0.5*(B+D) : E
E1 = B=F & B!=D & F!=H & (E!=C | E=A | E=I | C=J | C=L) ? 0.5*(B+F) : E
E2 = D=H & B!=D & F!=H & (E!=G | E=A | E=I | G=K | G=M) ? 0.5*(D+H) : E
E3 = F=H & B!=F & D!=H & (E!=I | E=C | E=G | I=L | I=M) ? 0.5*(F+H) : E

the actual shader code looks a bit different due to subterm optimization.


the cause of artifacts and how to prevent them is analogue to the case of Scale2x. the interesting flaw from Scale3x is that it doesn’t properly handle 45° lines, on those lines Scale3x only adds pixels but doesn’t “shave” them off. so to fix this we have to add another condition. but first let’s have a look at the subpixels of E at scale factor 3:

  A B C		  E0 E1 E2
K D E F L		E3 E4 E5
  G H I		  E6 E7 E8

again… A0 to A8 are the subpixels of A and so on. okay, let’s say the pixels E, C and G are building a 45° line. then the condition for B8 and D8 is met but now at scale factor 3 E0 is standing out like a sore thumb. so to fix this we again use a very simple rule, just check if the condition for B8 or D8 is satisfied. if this is the case then “remove” E0 (replace it with B=D). and here is the pseudocode comparison:


E4 = E

E0 = B=D & B!=F & D!=H ? 0.5*(B+D) : E
E2 = B=F & B!=D & F!=H ? 0.5*(B+F) : E
E6 = D=H & B!=D & F!=H ? 0.5*(D+H) : E
E8 = F=H & B!=F & D!=H ? 0.5*(F+H) : E

E1 = (B=D & B!=F & D!=H & E!=C) | (B=F & B!=D & F!=H & E!=A) ? B : E;
E3 = (B=D & B!=F & D!=H & E!=G) | (D=H & B!=D & F!=H & E!=A) ? D : E;
E5 = (B=F & B!=D & F!=H & E!=I) | (F=H & B!=F & D!=H & E!=C) ? F : E;
E7 = (D=H & B!=D & F!=H & E!=I) | (F=H & B!=F & D!=H & E!=G) ? H : E;


E4 = E

E0 = (B=D & B!=F & D!=H & (E!=A | E=C | E=G | A=J | A=K)) | (B=D & C=E & C!=J & A!=E)
   | (B=D & E=G & A!=E & G!=K) ? 0.5*(B+D) : E
E2 = (B=F & B!=D & F!=H & (E!=C | E=A | E=I | C=J | C=L)) | (B=F & A=E & A!=J & C!=E)
   | (B=F & E=I & C!=E & I!=L) ? 0.5*(B+F) : E
E6 = (D=H & B!=D & F!=H & (E!=G | E=A | E=I | G=K | G=M)) | (D=H & A=E & A!=K & E!=G)
   | (D=H & E=I & E!=G & I!=M) ? 0.5*(D+H) : E
E8 = (F=H & B!=F & D!=H & (E!=I | E=C | E=G | I=L | I=M)) | (F=H & C=E & C!=L & E!=I)
   | (F=H & E=G & E!=I & G!=M) ? 0.5*(F+H) : E

E1 = (B=D & B!=F & D!=H & (E!=A | E=C | E=G | A=J | A=K) & E!=C)
   | (B=F & B!=D & F!=H & (E!=C | E=A | E=I | C=J | C=L) & E!=A) ? B : E;
E3 = (B=D & B!=F & D!=H & (E!=A | E=C | E=G | A=J | A=K) & E!=G)
   | (D=H & B!=D & F!=H & (E!=G | E=A | E=I | G=K | G=M) & E!=A) ? D : E;
E5 = (F=H & B!=F & D!=H & (E!=I | E=C | E=G | I=L | I=M) & E!=C)
   | (B=F & B!=D & F!=H & (E!=C | E=A | E=I | C=J | C=L) & E!=I) ? F : E;
E7 = (F=H & B!=F & D!=H & (E!=I | E=C | E=G | I=L | I=M) & E!=G)
   | (D=H & B!=D & F!=H & (E!=G | E=A | E=I | G=K | G=M) & E!=I) ? H : E;

for a more detailed explanation of the original algorithms go visit:

Awesome write-up :smiley:

I’ll get the repo updated with your latest code ASAP.

Hi there, just wanna announce that I revisited my modification of scaleNx. I guess my original version was a bit hard to understand since I cramped everything into one step. Now I seperated it into multple passes and added a few enhancements. This should not only make it easier to understand and modify but also faster because of fewer redundant calculations.

here is a little preview, 2 x scale3xSFX (click for full resolution):

I still need to clean up the code, download will be ready in the next few days.


Looks great. It does an awesome job with curves :slight_smile:

if anyone is wondering, just wanna mention that I’m still working on it. I saw the potential for a more sophisticated interpolation, so it isn’t just an exercise without any real purpose. here is a wip screenshot, I added lvl3 edge detection (you can see it on the left side of shantae’s hair)

1 Like

It’s looking interesting, Sp00kyFox.

Is it intended only for low color games (8 and 16-bit systems) or for most modern anti-aliased ones? Do you think it would be hard to add any anti-aliasing treatment to the algorithm, or it isn’t your goal?

Should rename this shader ‘waterpaint’

Amazing the results that can be achieved though

sorry to keep you waiting. I spend the time pushing it forward to a release status and it’s ready!


screenshot album:

there is a user-defined threshold value. the 9x-cgp can be rather demanding on your system, so be warned. screenshots or suggestions for shader combos are welcome.

[QUOTE=Hyllian;28966]It’s looking interesting, Sp00kyFox. Is it intended only for low color games (8 and 16-bit systems) or for most modern anti-aliased ones? Do you think it would be hard to add any anti-aliasing treatment to the algorithm, or it isn’t your goal?[/QUOTE] thx. well I developed it with gameboy and NES games in mind. the edge detection is general enough so that it also deals with anti-aliased graphics but it doesn’t differentiate. it does an ok job on it I guess but it doesn’t compete in that regard with specialized filters, the optimal usage is certainly with pixelart. to be honest with you I don’t really know how one would deal with anti aliased material so I would need to study the topic first before I can answer that question. my algorithm itself is not ‘too complex’ though, so feel free to take a look. I’ll try to explain it in further detail in the near future.

It reminds me of xbr-noblend filter in that gallery. It indeed treats anti-aliasing some way, so you have some kind of color distance measure.

indeed, xbr-noblend looks similar and in my test runs I used it for comparisons. one of the goals of the original idea was to modify the scaleNx algorithm so that it can deal with color gradients (where the original one only checks for equality). I experimented with a few metrics but finally stayed with the one I used in some of my previous shaders which I think yields better results than the often used YUV-method:

I’m back with a new version. I slightly improved the algorithm by changing the way the color threshold works. until now it could actually could get in the way of itself if multiple valid but contradicting edge candidates are available. now it prioritizes those with minimal distance value. as a result you can pretty much crank up the color threshold value as much as you like without destroying the picture or lessen the edge interpolation. also calling it ScaleFX now (SFX - Sp00kyFox, you see what I did there? ^^)

I also worked on combining the reverse AA filter with ScaleFX. came up with a solution by doing the edge detection on the original picture but taking the subpixels in the last pass from rAA. this works surprisingly well. since I didn’t incorporate the algorithm itself the neat thing is that one can replace rAA with any other preshader as long as it outputs a 3x-version of the original frame.




ScaleFX-hybrid (rAA)

Great update, Sp00kyFox! It seems to be a good alternative to xbr-hybrids.

I just added your newest version to common-shaders.

tl;dr: new version, supports lvl6 edges, way more performant. here is the download and and in the spirit of the SF5 release a gallery with some street fighter games:



Development Notes

back with another major update. I was working on the edge detection and the shader now supports up to level 6 (which is 6 pixels between each stair). the super mario title screen is a good example:

old version:

new version:

during implementing it I noticed a problem. until now I determined the edge level by counting the pixels beginning from the stair into one direction. this led to improper sloping at the end of a staircase pattern. this is also fixed now by looking into both directions. compare the hud of shantae.

original, old version, new version:

adding level 6 support and looking into both directions costs a lot more texture lookups though. lucky enough there was potential for a huge performance gain. until now I did the edge analysis in the last pass which runs at scale factor 3. so every texture lookup and calculation is done again and again for each of the nine subpixels. I added a fifth pass which only takes the result from the last pass by doing one lookup. therefore the previously last pass can now run at scale factor 1 and runs a lot faster because of this. especially in the case of using ScaleFX twice (to get a 9x filtered image) you should notice a big improvement in performance.

regarding file and folder structure. there is an overlap between the normal and hybrid variant. in fact only pass0 and pass1 are different, so I removed the other ones and changed the cgp files accordingly.

It’s looking great, Sp00kyFox!

The stair at lvl6 is amazing!

So, looking at both directions and one of them can be shorter than the other. Then you choose the level based on the shorter one? (it’s the heuristic I have used on xbr mlv4 more or less, because I was getting some weird results at the time too.)

Looking at the SF gallery, are you using the pure or hybrid one there?

To tell you the truth, I prefer your pure ScaleNx than the hybrid one, for the same reason I prefer pure xbr over its hybrid version. The algorithms don’t seam very well, and that’s why I abandoned the development of it in favor of the new super-xbr ones.

Maybe you should look at the way super-xbr work to begin a development of a Super-ScaleNx? An algorithm that could work well on digitized games and anti-aliased ones.