Weird scaling issues?

RetroArch Linux devices
#4

‘Crop overscan’ is currently kinda weird and gets handled by each core differently. There’s no magic bullet for cropping NES games that will make them all perfect, as some games only have junk on one side and lose necessary content if you cut off all sides.

If you’re particular about aspect ratio and/or misshapen pixels, always keep ‘crop overscan’ off and either just deal with the dead space / visual garbage or see if the overscan cropping options in the image-adjustment.cg shader do the trick.

1 Like
#5

[QUOTE=hunterk;21044]‘Crop overscan’ is currently kinda weird and gets handled by each core differently. There’s no magic bullet for cropping NES games that will make them all perfect, as some games only have junk on one side and lose necessary content if you cut off all sides.

If you’re particular about aspect ratio and/or misshapen pixels, always keep ‘crop overscan’ off and either just deal with the dead space / visual garbage or see if the overscan cropping options in the image-adjustment.cg shader do the trick.[/QUOTE]

I didn’t know about the image-adjustment.cg shader, thanks for the tip! I’ll definitely check that out. It seems like I’m learning more about the versatility of RA every day.

I’m still curious though as to why there is uneven scaling with “crop overscan on.” I know almost nothing about this subject, but what it looks like is that each pixel (output by the ‘NES’) has been stretched vertically by the emulator so that it fills the 4:3 area after cropping the top and bottom of the image. However, every so often there is an entire horizontal row of pixels that isn’t stretched by the same amount. The result is that in a pattern like the blocks in SMB, some of the lines are ~one pixel thinner.

I don’t understand why you can’t just crop the image, determine how much the entire image needs to be stretched to fill the 4:3 area, divide this amount of vertical space (in pixels) by the total remaining vertical space in pixels and then stretch each pixel vertically by that amount. Is it because you have to stretch by whole pixels or something? I don’t really know how any of this works.

EDIT: even leaving crop overscan off results in unevenly sized pixels. One only has to look at the brick pattern in SMB to see lines of unequal thickness.

1 Like
#6

[QUOTE=Awakened;21001]You can turn on integer scaling or use a pixellate shader pass to fix the uneven scaling. I think you’d do pixellate as the first shader, then scanline on top of that, but if that doesn’t work or look right, try the opposite. I don’t know if the raspberry pi 2 is fast enough for two passes, but both those shaders are very low overhead.

There’s a setting in Settings->Video Settings called Crop Overscan (Reload) that you can turn off and restart RA to apply. That should then display the extra top and bottom overscan.[/QUOTE]

Integer scaling is no good because then the image is letterboxed. The pixellate shader is too resource intensive for the rasp pi model 2.

I don’t so much mind the overscan, I just would like lines in a game to be of uniform thickness, the way they’re supposed to be.

I need to post a screenshot showing the problem, but when I try uploading a screenshot it gets resized to the point where you can’t see the problem anymore.

1 Like
#7

Turning on Bilinear filtering in the video options will also fix the unevenness, but will make things blurry (Edit: Looks like that gets disabled when you enable a shader, so I don’t know if there is a way to combine it with the scanline.glsl shader). Pixellate is basically meant to fix that problem with very minimal blurring; too bad it doesn’t work on it. All the CRT shaders I’ve used fix it too, but I’ve heard even crt-easymode is too much for the Pi 2.

From what I’ve read before, the issue happens because of rounding errors when you scale to non-integer values without a filter. It’s most noticeable to me on the health bar in Mega Man games.

#8

Yes, it’s a result of unavoidable inconsistencies in texel sampling when upscaling by nearest neighbor at non-integer scale factors. Indeed, switching to integer scaling and/or bilinear filtering will correct it, and you can switch to bilinear with a shader applied by going into the shader options and setting the (last) shader’s ‘filter’ option to ‘linear’ instead of ‘don’t care.’ You can also check out maister’s sharp-bilinear.cg, which is similar to pixellate.cg but may be faster on your hardware (if it looks weird, try switching to/from linear filtering; I think it requires one or the other to work properly).

You could also stick to integer scaling and use a fancy border overlay to fill in the black space :slight_smile:

#9

[QUOTE=Awakened;21070]Turning on Bilinear filtering in the video options will also fix the unevenness, but will make things blurry (Edit: Looks like that gets disabled when you enable a shader, so I don’t know if there is a way to combine it with the scanline.glsl shader). Pixellate is basically meant to fix that problem with very minimal blurring; too bad it doesn’t work on it. All the CRT shaders I’ve used fix it too, but I’ve heard even crt-easymode is too much for the Pi 2.

From what I’ve read before, the issue happens because of rounding errors when you scale to non-integer values without a filter. It’s most noticeable to me on the health bar in Mega Man games.[/QUOTE]

I can confirm that crt-easymode is too much for the rasp pi 2- which is strange. Has it been optimized for the new processor?

Pixellate works but causes an unacceptable drop in frame rate.

Bilinear filter is no good because of the bluriness, but it does get rid of the unevenness.

Is there a filter that fixes the unevenness without the bluriness? I tried adding the shader sharp-bilinear but that doesn’t fix the uneveness.

Is this just something the retropie user has to live with? It’s not a huge deal but it’s a slight annoyance.

Forgive my ignorance, but would it be possible to change crop overscan to just crop an extra 2 pixels or so (or however much is needed) in order to fix the rounding error?

1 Like
#10

[QUOTE=hunterk;21073]Yes, it’s a result of unavoidable inconsistencies in texel sampling when upscaling by nearest neighbor at non-integer scale factors. Indeed, switching to integer scaling and/or bilinear filtering will correct it, and you can switch to bilinear with a shader applied by going into the shader options and setting the (last) shader’s ‘filter’ option to ‘linear’ instead of ‘don’t care.’ You can also check out maister’s sharp-bilinear.cg, which is similar to pixellate.cg but may be faster on your hardware (if it looks weird, try switching to/from linear filtering; I think it requires one or the other to work properly).

You could also stick to integer scaling and use a fancy border overlay to fill in the black space :)[/QUOTE]

Well, that went right over my head :slight_smile:

I think I sort of understand.

sharp-bilinear fixes the unevenness, but it doesn’t work with scanline.glsl which is the only scanline shader that I can get to work with the retropie.

So it looks like my options with the Pi are to either use bilinear filtering, which fixes the unevenness but adds bluriness, or to use integer scaling plus a CRT border, or to use sharp-bilinear which fixes the unevenness but prevents scanlines from being used. Or just live with some slight unevenness. None of these is a perfect solution, but oh well. It’s a somewhat minor issue- I just want everything to be as perfect as possible. :slight_smile:

Why are the only filter options nearest neighbor and bilinear? Is it theoretically possible to have a filter that fixes the unevenness like bilinear, but without adding bluriness?

Btw, RA on the Pi 2 is really sweet - it runs noticeably better (less input lag) on the Pi than on my Windows 7 PC. I’m kind of blown away by it, actually.

1 Like
#11

CRT-easymode is a shader, so it’s dependent on GPU rather than CPU, so it’d be the same on RPi1 and RPi2, which share the same GPU.

Did you set the filtering on the sharp-bilinear to linear? It does the trick here with linear but not with nearest filtering. EDIT: ah, glad you got it going. Yeah, that scanline shader conflicts with it.

To get rid of it with cropping, you would need to scale it up to the next largest integer (presumably 5x) and then crop the remainder, which would be a lot of lost pixels.

There’s another complication in that you’re using 4:3–which is good; that’s what I use–so even if you did integer scaling on the y-axis, you’d have non-integer scaling on the x-axis because of the pixel aspect ratio being something other than 1:1 (which is why there’s a 1:1 PAR option in the aspect ratio indexes, in case you would rather have a funky aspect ratio than occasional warped/blurry pixels). This is actually what the pixellate shader was specifically designed to deal with: integer scaling on the y and then very slight blurring on the x and still get to keep your 4:3 aspect ratio.

In other words, it’s simply not possible to get perfect scaling and have a 4:3 aspect ratio (for most systems; I’m sure there’s an exception somewhere, but NES isn’t one of them). Pixellate and sharp-bilinear get you as close as possible, and you can minimize warping/blurring even further by using integer scaling on the y axis.

EDIT: yeah, running RetroArch in KMS/EGL context from a linux console is awesome. You can get roughly the same feeling from Windows if you set Hard GPU Sync to 0 frames, but it greatly increases CPU overhead.

1 Like
#12

[QUOTE=hunterk;21087]

*clipped

There’s another complication in that you’re using 4:3–which is good; that’s what I use–so even if you did integer scaling on the y-axis, you’d have non-integer scaling on the x-axis because of the pixel aspect ratio being something other than 1:1 (which is why there’s a 1:1 PAR option in the aspect ratio indexes, in case you would rather have a funky aspect ratio than occasional warped/blurry pixels). This is actually what the pixellate shader was specifically designed to deal with: integer scaling on the y and then very slight blurring on the x and still get to keep your 4:3 aspect ratio.

In other words, it’s simply not possible to get perfect scaling and have a 4:3 aspect ratio (for most systems; I’m sure there’s an exception somewhere, but NES isn’t one of them). Pixellate and sharp-bilinear get you as close as possible, and you can minimize warping/blurring even further by using integer scaling on the y axis.

EDIT: yeah, running RetroArch in KMS/EGL context from a linux console is awesome. You can get roughly the same feeling from Windows if you set Hard GPU Sync to 0 frames, but it greatly increases CPU overhead.[/QUOTE]

edit: realised my post didn’t make sense because I was misunderstanding the meaning of “pixel aspect ratio.”

Is the scaling problem the reason why Wii U VC games have the bilinear filter (since the Wii U is 1080p)? I always wondered about that.

Is there a way to make the bilinear filter have less of a blurring effect? Would 720p as opposed to 1080p make a difference?

is there a way to get scanline.glsl to play nice with bilinear filter-sharp? Are they just fundamentally incompatible or could they be coded differently so that they work together?

What about trilinear filtering? Would that reduce bluriness compared to bilinear filtering?

Sorry for the onslaught of questions, and thanks for all your answers so far!

Edit: Progress!! I set a custom resolution for RA of 1024x720 in the RA config. Then I set my preferred resolution for nestopia to 720p via the launch options, and set RA render res to “config.” I then turned on integer scaling and turned off crop overscan. With this setup, I get fullscreen 4:3 picture with no scaling artifacts, and I can use the scanline.glsl shader! This is awesome. Just thought I’d pass this along for anyone who is having similar issues. This might be the best setup for retropie users who want the most “authentic” look possible. So it seems that it is possible to get full screen picture in 4:3 mode with no scaling artifacts by playing around with the resolution options.

Edit 2: another update - looks like I am still getting some slight scaling artifact on the x axis that was unnoticeable when I did my first test. I must be slightly off with 1024 - I was assuming 256x240 as the internal resolution of the NES. Is this not correct?

It seems that I can fix the vertical scaling at 1080p by setting the RA render res to 1536x1080. Again though, I get some barely noticeable scaling artifacts on the x axis. I have to turn off integer scaling for this to work, since I’m multiplying by 6 and 4.5. I wonder why I’m still getting these slight artifacts on the x axis.

#13

bump; edited/updated last post.

#14

Yeah, you can get rid of warped pixels on both axes by disabling crop overscan, setting your display to 720p and then choosing the 1:1 PAR option from the aspect ratio index. For NES/SNES, this equates to a display aspect ratio 8:7, which is tall and skinny as compared with 4:3. If you’re okay with that, you should be all set. You might also be interested in another thread on this forum that goes into great depth about “correct” aspect ratios for each system that will avoid pixel warping and make circles more round (or something like that).

IMO, 4:3 is “correct” and everything else is conjecture, but some people disagree very strongly on this issue so it’s better not to argue about it (again).

#15

[QUOTE=hunterk;21129]Yeah, you can get rid of warped pixels on both axes by disabling crop overscan, setting your display to 720p and then choosing the 1:1 PAR option from the aspect ratio index. For NES/SNES, this equates to a display aspect ratio 8:7, which is tall and skinny as compared with 4:3. If you’re okay with that, you should be all set. You might also be interested in another thread on this forum that goes into great depth about “correct” aspect ratios for each system that will avoid pixel warping and make circles more round (or something like that).

IMO, 4:3 is “correct” and everything else is conjecture, but some people disagree very strongly on this issue so it’s better not to argue about it (again).[/QUOTE]

I agree, 4:3 is the “most correct” resolution, since it’s what all CRTs displayed without any cropping or stretching of the image, which was done only to disguise flaws in the CRT picture and which varied considerably between individual sets (even sets of the exact same model).

I’m just wondering why I can’t get perfectly scaled pixels when in a 4:3 ratio.

If the internal res of the NES is 256x240, then it seems like I should get perfect uniform pixel scaling with a custom RA res of 1536x1080 with my display set to 1080p. Instead I get perfect vertical scaling but still get some very slight horizontal artifact. Likewise, it seems I should get perfect scaling with a RA render res of 1024x720, with my display set to 720p. Again, I get perfect vertical scaling and slight horizontal artifacts.

It just occurred to me- is this because the 1024 or 1536 is still being upscaled to 1280 or 1920? So, is the problem a result of how “nearest neighbor” is handling the scaling on the x axis?

#16

If you don’t have black bars on the sides (i.e., you’re using the full 16x9 resolution), then yes. Pixel warping happens when you don’t scale by an even integer scale factor. 1080 divided by 240 equals 4.5, so there will be warping on the y axis. 720 is an exact multiple, so there won’t be warping. 1024 on the x axis should give you an even integer, as long as you’ve chosen 1:1 PAR, which gives you square pixels that match up with the square pixels of your display.

That is, you need to keep in mind both pixel aspect ratio (which needs to match the square pixels of modern displays; old CRTs had non-square pixels) and display aspect ratio, which needs to be an even integer multiple.

#17

[QUOTE=hunterk;21135]If you don’t have black bars on the sides (i.e., you’re using the full 16x9 resolution), then yes. Pixel warping happens when you don’t scale by an even integer scale factor. 1080 divided by 240 equals 4.5, so there will be warping on the y axis. 720 is an exact multiple, so there won’t be warping. 1024 on the x axis should give you an even integer, as long as you’ve chosen 1:1 PAR, which gives you square pixels that match up with the square pixels of your display.

That is, you need to keep in mind both pixel aspect ratio (which needs to match the square pixels of modern displays; old CRTs had non-square pixels) and display aspect ratio, which needs to be an even integer multiple.[/QUOTE]

Sorry, I’m doing a really poor job of explaining things.

What I want is to play in the 4:3 display aspect ratio, with uniform scaling of the pixels, i.e., “NES pixels” that all get scaled to the same size/dimensions in “TV/monitor pixels”. I can get uniform scaling on the vertical axis (ie., all pixels are the same size vertically) if I set RA’s resolution to either 1536x1080 or 1024x720 and adjust my monitor’s resolution accordingly (to either 1080p or 720p, respectively). But I still get some slight horizontal artifacts (i.e., some of the pixels vary in how wide they are horizontally).

I am guessing that the 1024 pixels or 1536 pixels is being upscaled to the full 1280 or 1920 pixels respectively. I guess this is being done by the “nearest neighbor” algorithm, since that’s what I have selected, I had thought that since 1024 and 1536 are less than 1280 or 1920 that it just wouldn’t use the rest of the screen and so it wouldn’t get upscaled and would be in the proper display aspect.

So I guess there is really no way to avoid using the bilinear filter if you want to eliminate (or rather, hide) the scaling artifacts when in a 4:3 aspect ratio on a 16:9 TV at 720 or 1080 resolution(?)

#18

Your conclusion is correct but your reasoning is not. Ultimately, it doesn’t matter much. You can’t get 4:3 aspect with nearest neighbor without warping on at least one of the axes on a 1080p/720p display. To avoid the warping, you need to use another scaling algo. Bilinear is basically free, computationally, because it leverages the GPU’s built-in hardware interpolator, but it’s blurry. Pixellate is intensive, computationally, because it manually samples the texture 4 times. Sharp-bilinear is much lighter than pixellate because it leverages the hardware interpolator to avoid the 4x samples. There are other algos that you could try that would be less blurry, like lanczos and quilez, but they’re not as sharp as pixellate or sharp-bilinear and are generally more computationally intense.

Anyway, I merged the scanline and sharp-bilinear shaders for you (set the filter to ‘linear’): EDIT: ah, crap. I just remembered you’re confined to glsl shaders… I’ll try to get this one converted for you at some point.

/*
 * sharp-bilinear.cg
 * Author: Themaister
 * License: Public domain
 * 
 * Does a bilinear stretch, with a preapplied Nx nearest-neighbor scale, giving a
 * sharper image than plain bilinear.
 */

struct input
{
  float2 video_size;
  float2 texture_size;
  float2 output_size;
};

struct sine_coord
{
   float2 omega;
};
 
void main_vertex
(
	float4 position	: POSITION,
	float4 color	: COLOR,
	float2 texCoord : TEXCOORD0,

   uniform float4x4 modelViewProj,

	out float4 oPosition : POSITION,
	out float2 otexCoord : TEXCOORD,
	uniform input IN,
   out sine_coord coords : TEXCOORD2
)
{
	oPosition = mul(modelViewProj, position);
	otexCoord = texCoord;
	coords.omega = float2(3.1415 * IN.output_size.x * IN.texture_size.x / IN.video_size.x, 2.0 * 3.1415 * IN.texture_size.y);
}

#pragma parameter SHARP_BILINEAR_PRE_SCALE "Sharp Bilinear Prescale" 4.0 1.0 10.0 1.0
#pragma parameter SCANLINE_BASE_BRIGHTNESS "Scanline Base Brightness" 0.95 0.0 1.0 0.01
#pragma parameter SCANLINE_SINE_COMP_A "Scanline Sine Comp A" 0.05 0.0 0.10 0.01
#pragma parameter SCANLINE_SINE_COMP_B "Scanline Sine Comp B" 0.15 0.0 1.0 0.05

#ifdef PARAMETER_UNIFORM
uniform float SHARP_BILINEAR_PRE_SCALE;
uniform float SCANLINE_BASE_BRIGHTNESS;
uniform float SCANLINE_SINE_COMP_A;
uniform float SCANLINE_SINE_COMP_B;
#else
#define SHARP_BILINEAR_PRE_SCALE 4.0
#define SCANLINE_BASE_BRIGHTNESS 0.95
#define SCANLINE_SINE_COMP_A 0.05
#define SCANLINE_SINE_COMP_B 0.15
#endif

float4 main_fragment(in sine_coord co : TEXCOORD2, float2 texCoord : TEXCOORD0,
   uniform sampler2D s0 : TEXUNIT0, uniform input IN) : COLOR
{
   float2 texel = texCoord * IN.texture_size;
   float2 texel_floored = floor(texel);
   float2 s = fract(texel);
   float region_range = 0.5 - 0.5 / SHARP_BILINEAR_PRE_SCALE;

   // Figure out where in the texel to sample to get correct pre-scaled bilinear.
   // Uses the hardware bilinear interpolator to avoid having to sample 4 times manually.

   float2 center_dist = s - 0.5;
   float2 f = (center_dist - clamp(center_dist, -region_range, region_range)) * SHARP_BILINEAR_PRE_SCALE + 0.5;

   float2 mod_texel = texel_floored + f;
   
   float2 sine_comp = float2(SCANLINE_SINE_COMP_A, SCANLINE_SINE_COMP_B);
   float3 res = tex2D(s0, mod_texel / IN.texture_size).rgb;
   float3 scanline = res * (SCANLINE_BASE_BRIGHTNESS + dot(sine_comp * sin(texCoord * co.omega), float2(1.0, 1.0)));   
   

   return float4(scanline.x, scanline.y, scanline.z, 1.0);
}

This should minimize warping at 4:3 and still give you scanlines, though they don’t look as nice as with straight-up bilinear filtering.

#19

Thanks for the explanation! I was thinking that my explanation for the uneven scaling was probably too simple to be correct :slight_smile: I should probably just take a class in video technology or something. I had no idea when I started down this path that it would be such a rabbit hole.

Anyway, I merged the scanline and sharp-bilinear shaders for you (set the filter to ‘linear’): EDIT: ah, crap. I just remembered you’re confined to glsl shaders… I’ll try to get this one converted for you at some point.

Awesome! That’s just what I wanted and I think a lot of pi users would appreciate it as well since we are pretty shader-limited. Thanks!

1 Like
#20

So just now I thought “why don’t I just use integer scaling, and then use my TV’s controls to stretch the picture to full screen? Brilliant!” But then I noticed that I get horizontal warping at 1080p even with integer scaling turned on. Why? Is scanline.slgl or nearest neighbor causing it?

Looking forward to that shader!

edit: playing around some more, I find that setting my monitor to 720p, turning on integer scaling and turning off crop overscan produces a very nice picture with scanline.slgl. You can then simulate crop overscan by stretching the picture vertically with your TV controls. There is still the slight warping on the x axis but it occurs in a way that is undetectable 99% of the time. A brick wall in SMB looks perfect on all parts of the screen. That pesky life bar in mega man still looks off, but it’s not terrible. This is as close to “perfect” as I’ve been able to get so far.

It’ll be nice to not have to switch resolutions or fiddle with my tv’s picture adjustment though, and to not have that warping on the x axis. Also, I imagine the scanlines will look better at 1080p. :slight_smile:

#21

Ok, here we go: http://www.mediafire.com/download/gfr4hygmyqwqo4v/sharp-bilinear-scanlines.glsl let me know if that does/doesn’t work for you.

1 Like
#22

[QUOTE=hunterk;21178]Ok, here we go: http://www.mediafire.com/download/gfr4hygmyqwqo4v/sharp-bilinear-scanlines.glsl let me know if that does/doesn’t work for you.[/QUOTE]

Looks great and runs smooth at 720p! Causes a slight drop in frame rate at 1080p. This is still a big improvement over what I consider the next best option, which would be to use scanline.glsl or phosphor.glsl at 720p with integer scale on and crop overscan off, adjusting for overscan with the TV’s controls if necessary (or, alternately, turning crop overscan on and stretching the picture to fill screen with the TV’s controls). Using phosphor with or without a bilinear filter produces an okay result, if you don’t mind the slight jailbar effect at 720p. Lcd3x and ntsc-pass 4 are also okay but like scanline.slgl they suffer from the slight horizontal artifacts without a bilinear filter.

This has the advantage of not having to mess with the TVs controls and works with crop overscan on or off, and effectively disguises the warping on the horizontal axis while producing scanlies. This is now the best option for retropie users to get the most “CRT-like” picture.

Thanks again!

1 Like
#23

Sweet. Glad to hear it works :slight_smile:

1 Like