Filling pillarbox space with scaled/blurred game output

Hiya. Blank pillarboxes and letterboxes are boring as hell, don’t you think? I think so too. For most of the time I’ve been using RA, I would use a devilspie2 rule to undecorate the window and fit it to the full size of the screen, leaving the wallpaper beneath visible. I think we could do better than that, though.

You know how on the thumbnails of youtube videos that use a vertical orientation, the pillarbox is filled with the video zoomed in, darkened, and (I think, maybe not) blurred? I would like to have an effect like that in RetroArch, applied in real-time. It doesn’t seem like such a thing exists right now.

Would it be possible to start a bounty for a shader like that? If not, where should I go to learn how to write screen shaders for RA so I could eventually write one of these myself?

it already exists. Check out borders/bigblur. The only catch is that you need to set RetroArch’s aspect ratio to 16:9 (or whatever the AR of your display is) and disable integer scaling.

Oh, cool! :smiley: When was this border added?

It’s been awhile. A year or two at least.

Thank you for informing me of this. I’m not sure this is exactly what I want since it seems hard-coded to certain resolutions, assumes you want integer scaling (personally, I want the video to fill the entire height of my screen), and doesn’t shade or darken the border to make it less distracting, but I think this might make a fine starting point for making something that better suits what I want.

What sort of resources should I look into for learning how to create and modify these shaders?

I recently updated the Cg version to be resolution-agnostic, but I haven’t applied those changes to the GLSL or slang versions yet. I’ll try to tackle that in the near future. In the meantime, try the Cg version and see if it’s closer to what you want.

The updated version also has runtime parameters to set custom aspect ratios for the game image, use integer scaling or not (with the ability to use an integer scale that’s one step larger than the screensize), add scanlines (only looks good with integer scaling) and to use anti-aliased sharp pixels (i.e., sharper than bilinear but without the pixel-warping of non-integer nearest neighbor scaling).

I could add border-darkening to it easy-peasy.

The easiest way to learn shader programming, IMO, is just to get in there and start playing around with the ones that already exist. It’s all just math, so twiddling numbers and equations can get you quite far. Other than that, the syntax is mostly C-like, and the shader compilers typically give helpful error messages with line numbers (the line numbers are a couple of lines off in GLSL shaders, unfortunately).

I really like the changes on the most recent CG version. That’s almost exactly what I want, save for shading the border area. How would I combine other shaders, like NTSC shaders, with this? I tried loading up the shaders in the bigblur preset as additional passes after the NTSC shader. I know that works fine if I want to use image-adjustment to cut off overscan, but when I throw in bigblur after that, it results in an unmodified image appearing in half of the space. Here’s the preset file from my attempt:

shaders = "6"
shader0 = "/home/tiz/.config/retroarch/shaders/shaders_cg/ntsc/shaders/ntsc-pass1-svideo-2phase.cg"
filter_linear0 = "false"
wrap_mode0 = "clamp_to_border"
frame_count_mod0 = "2"
mipmap_input0 = "false"
alias0 = ""
float_framebuffer0 = "true"
srgb_framebuffer0 = "false"
scale_type_x0 = "absolute"
scale_x0 = "1280"
scale_type_y0 = "source"
scale_y0 = "1.000000"
shader1 = "/home/tiz/.config/retroarch/shaders/shaders_cg/ntsc/shaders/ntsc-pass2-2phase-gamma.cg"
filter_linear1 = "false"
wrap_mode1 = "clamp_to_border"
mipmap_input1 = "false"
alias1 = ""
float_framebuffer1 = "false"
srgb_framebuffer1 = "false"
scale_type_x1 = "source"
scale_x1 = "0.500000"
scale_type_y1 = "source"
scale_y1 = "1.000000"
shader2 = "/home/tiz/.config/retroarch/shaders/shaders_cg/misc/image-adjustment.cg"
wrap_mode2 = "clamp_to_border"
mipmap_input2 = "false"
alias2 = ""
float_framebuffer2 = "false"
srgb_framebuffer2 = "false"
shader3 = "/home/tiz/.config/retroarch/shaders/shaders_cg/blurs/blur11fast-horizontal.cg"
filter_linear3 = "true"
wrap_mode3 = "clamp_to_border"
mipmap_input3 = "false"
alias3 = ""
float_framebuffer3 = "false"
srgb_framebuffer3 = "false"
shader4 = "/home/tiz/.config/retroarch/shaders/shaders_cg/blurs/blur11fast-vertical.cg"
filter_linear4 = "true"
wrap_mode4 = "clamp_to_border"
mipmap_input4 = "false"
alias4 = ""
float_framebuffer4 = "false"
srgb_framebuffer4 = "false"
shader5 = "/home/tiz/.config/retroarch/shaders/shaders_cg/borders/resources/bigblur.cg"
wrap_mode5 = "clamp_to_border"
mipmap_input5 = "false"
alias5 = ""
float_framebuffer5 = "false"
srgb_framebuffer5 = "false"
parameters = "NTSC_CRT_GAMMA;NTSC_MONITOR_GAMMA;target_gamma;monitor_gamma;overscan_percent_x;overscan_percent_y;saturation;contrast;luminance;black_level;bright_boost;R;G;B;ZOOM;XPOS;YPOS;TOPMASK;BOTMASK;LMASK;RMASK;GRAIN_STR;aspect_x;aspect_y;ZOOM;integer_scale;overscale;scanline_toggle;interp_toggle;THICKNESS;DARKNESS"
NTSC_CRT_GAMMA = "2.500000"
NTSC_MONITOR_GAMMA = "2.000000"
target_gamma = "2.200000"
monitor_gamma = "2.200000"
overscan_percent_x = "0.000000"
overscan_percent_y = "0.000000"
saturation = "1.000000"
contrast = "1.000000"
luminance = "1.000000"
black_level = "0.000000"
bright_boost = "0.000000"
R = "1.000000"
G = "1.000000"
B = "1.000000"
ZOOM = "1.000000"
XPOS = "0.000000"
YPOS = "0.000000"
TOPMASK = "0.000000"
BOTMASK = "0.000000"
LMASK = "0.000000"
RMASK = "0.000000"
GRAIN_STR = "0.000000"
aspect_x = "64.000000"
aspect_y = "49.000000"
ZOOM = "1.500000"
integer_scale = "1.000000"
overscale = "0.000000"
scanline_toggle = "0.000000"
interp_toggle = "0.000000"
THICKNESS = "2.000000"
DARKNESS = "0.350000"

And here is a screenshot of my work computer with a 2nd-gen intel iGPU trying to do the thing (I have no expectations that this thing will be able to do it with good performance, I’m just trying to see if it works right and looks the way I want):

EDIT: I tried to combine image-adjustment only with bigblur, and then tried to do overscan correction. The image in the main area was unmodified, but I saw the blurred image shrink. Is this indicative of a limitation in RA’s shader architecture, or am I just doing it wrong?

It has to do with the shaders you’re mixing and the order you’re mixing them in. That NTSC shader does some funky stuff to the scaling of the image and then bigblur does some more funky stuff and the results are unpredictable.

Bigblur also needs some modification to allow for putting shaders in front of it, as it currently always loads the raw core output for the center image:

// A recreation of the blur border commonly used for portrait cell phone videos
// Created by hunterk
// License: public domain

#pragma parameter aspect_x "Aspect Ratio Numerator" 64.0 1.0 256. 1.0
#pragma parameter aspect_y "Aspect Ratio Denominator" 49.0 1.0 256. 1.0
#pragma parameter ZOOM "Border Zoom" 1.5 0.5 10 0.5
#pragma parameter integer_scale "Force Integer Scaling" 1.0 0.0 1.0 1.0
#pragma parameter overscale "Integer Overscale" 0.0 0.0 1.0 1.0
#pragma parameter scanline_toggle "Scanline Toggle" 0.0 0.0 1.0 1.0
#pragma parameter interp_toggle "Sharpen Linear Scaling" 0.0 0.0 1.0 1.0
#pragma parameter THICKNESS "Scanline Thickness" 2.0 1.0 12.0 1.0
#pragma parameter DARKNESS "Scanline Darkness" 0.35 0.0 1.0 0.05
#ifdef PARAMETER_UNIFORM
uniform float aspect_x;
uniform float aspect_y;
uniform float ZOOM;
uniform float integer_scale;
uniform float overscale;
uniform float scanline_toggle;
uniform float THICKNESS;
uniform float DARKNESS;
uniform float interp_toggle;
#else
#define aspect_x 64.0
#define aspect_y 49.0
#define ZOOM 1.5
#define integer_scale 1.0
#define overscale 0.0
#define scanline_toggle 0.0
#define THICKNESS 2.0
#define DARKNESS 0.35
#define interp_toggle 0.0
#endif
// END PARAMETERS //

#include "../../compat_includes.inc"
INITIALIZE_PASSPREV(3, 0)
uniform COMPAT_Texture2D(decal) : TEXUNIT0;
uniform float4x4 modelViewProj;

struct out_vertex
{
	float4 position : COMPAT_POS;
	float2 texCoord : TEXCOORD;
	float2 tex_border : TEXCOORD1;
};

out_vertex main_vertex(COMPAT_IN_VERTEX)
{
#ifdef HLSL_4
	float4 position = VIN.position;
	float2 texCoord = VIN.texCoord;
	float2 t1 = VIN.t1;
#endif
	out_vertex OUT;

	OUT.position = mul(modelViewProj, position);

	float2 out_res = COMPAT_output_size;
	float2 corrected_size = COMPAT_video_size * float2(aspect_x / aspect_y, 1.0)
		 * float2(COMPAT_video_size.y / COMPAT_video_size.x, 1.0);
	float full_scale = (integer_scale > 0.5) ? floor(COMPAT_output_size.y /
		COMPAT_video_size.y) + overscale : COMPAT_output_size.y / COMPAT_video_size.y;
	float2 scale = (COMPAT_output_size / corrected_size) / full_scale;
	float2 middle = float2(0.49999, 0.49999) * COMPAT_video_size / COMPAT_texture_size;
	float2 diff = texCoord.xy - middle;
	OUT.texCoord = middle + diff * scale;
	float2 zoom_coord = (((texCoord - middle) / ZOOM) * float2(COMPAT_output_size.x / COMPAT_output_size.y, 1.0)
		/ float2(aspect_x / aspect_y, 1.0)) + middle;
	OUT.tex_border = zoom_coord;
	return OUT;
}

#define fragcoord (tex.xy * (texture_size.xy/video_size.xy))

float4 scanlines(float4 frame, float2 coord, float2 texture_size, float2
	video_size, float2 output_size)
{
	float lines = fract(coord.y * texture_size.y);
	float scale_factor = floor((output_size.y / video_size.y) + 0.4999);
	return (scanline_toggle > 0.5 && (lines < (1.0 / scale_factor * THICKNESS)))
		? frame * vec4(1.0 - DARKNESS) : frame;
}

float2 interp_coord(float2 coord, float2 texture_size)
{
	vec2 p = coord.xy;

	p = p * texture_size.xy + vec2(0.5, 0.5);

	vec2 i = floor(p);
	vec2 f = p - i;

	// Smoothstep - amazingly, smoothstep() is slower than calculating directly the expression!
	f = f * f * f * f * (f * (f * (-20.0 * f + vec2(70.0, 70.0)) - vec2(84.0, 84.0)) + vec2(35.0, 35.0));

	p = i + f;

	p = (p - vec2(0.5, 0.5)) * 1.0 / texture_size;
	return p;
}

float4 border(float2 texture_size, float2 video_size, float2 output_size,
	float frame_count, float2 tex, COMPAT_Texture2D(decal), float2 tex_border, COMPAT_Texture2D(ORIG))
{
	float4 effect = COMPAT_Sample(decal, tex_border);
	
	float2 coord = (interp_toggle < 0.5) ? tex : interp_coord(tex, texture_size);
	float4 frame = COMPAT_SamplePoint(ORIG, coord);
	frame = scanlines(frame, tex, texture_size, video_size, output_size);
	if (fragcoord.x < 1.0 && fragcoord.x > 0.0 && fragcoord.y < 1.0 && fragcoord.y > 0.0)
		return frame;
	
	else return effect;
}

float4 main_fragment(prev PASSPREV3 : TEXUNIT1, COMPAT_IN_FRAGMENT) : COMPAT_Output
{
	return border(COMPAT_texture_size, COMPAT_video_size, COMPAT_output_size,
		COMPAT_frame_count, VOUT.texCoord, decal, VOUT.tex_border, PASSPREV_texture(3));
}
COMPAT_END

Here’s a basic preset using that shader and putting 4xbrz in front of it, though you’ll notice that larger initial framebuffers make the blur passes less effective:

shaders = 4

shader0 = ../xbrz/shaders/4xbrz.cg
scale_type0 = source
scale0 = 4.0
filter_linear0 = false

shader1 = ../blurs/blur11fast-horizontal.cg
filter_linear1 = true
scale_type1 = source

shader2 = ../blurs/blur11fast-vertical.cg
filter_linear2 = true
scale_type2 = source

shader3 = resources/bigblur.cg
filter_linear3 = true

parameters = "integer_scale"
integer_scale = 0.0

If the only thing you’re using the image-adjustment shader for is overscan cropping, that can be added to the border shader(s) pretty easily, instead. I’ve already done that with the effect-border-iq slang shader but haven’t added it to the Cg one. As you can see, these particular shaders are sort of in flux at the moment, and I’m trying different things in each one :slight_smile:

Hey man. I noticed you’ve been doing some more work on the CG version of the shader, and that the previous pass version is now in the updater. I tested it before you put it in the updater with the NTSC shader, and it did work as expected, but when I stuck image-adjustment in the middle, I only got the center 50% or so of the image. Same thing with the updater version. But indeed, I was only using it to crop overscan. I can see that you have something for that, but it handles overscan differently than image-adjustment. Image-adjustment has an overscan percent dealie that crops overscan by horizontally or vertically zooming by a specified percent, which keeps the image maximized in the window, whereas this shader just cuts it off and lets the border show through. Could we add that?

Or would it be better to figure out why image-adjustment is acting funky so that work isn’t duplicated in the shader? I’ve noticed a bunch of work that seems duplicated among the various border shaders. Why not just one shader that grabs the shaded game image and sticks it on top of the border image, however it seems to be generated (blur, shiny, snow, etc)? That is to say: game shader passes -> border shader passes -> final shader that grabs the last game shader pass and displays that as desired? Then you wouldn’t have to duplicate work like overscan adjustment, scanlines, etc. You could let other shaders do that if the user wanted them. I still don’t really understand how these shaders work, even though it’s just a bunch of math. I don’t know what any of this math is doing or why.

Also, one last question… why is the default aspect ratio denominator 49 instead of 48?

I really appreciate all the work you’ve been doing on this.

The other type of overscan cropping that the image-adjustment shader does causes the image to be stretched, which in turn messes up scanlines and/or causes uneven pixel sizes, etc. That is, all of the problems that come with non-integer scale factors.

The reason they’re set up how they are is all because of scaling. Typically, up/down-scaling shader effects looks like crap (at least for CRT-type effects, scanlines get all uneven, mask effects turn into rainbows, etc.; smoothing shaders like xBR usually look fine, though), so you can’t just do those effects in previous passes and then do the border scaling. Unfortunately, the way RetroArch works, you can’t apply effects to the entire screen unless RetroArch’s aspect ratio is set to match the display (e.g., 16:9 AR) and then the the border shader takes the image and scales it according to the shader’s settings with the effect filling in the rest of the space. This means that any effects you already applied are going to look bad, so we wait and do them all in the same pass instead.

I tried doing the effects in a separate pass, which can be good because you can run the effects at a lower scale factor (i.e., less than the final “don’t care” scale) for better performance but it makes them look stretched when you add them back in later. In the slang version, I went ahead and put all of the effects into their own functions in the same shader, but that requires the GPU to have a lot of shader registers, which any GPU supporting Vulkan should have, but I didn’t think that was a safe enough assumption to make for GLSL and Cg.

As for the 64:49 default aspect ratio, it’s the aspect ratio that all the cool kids are using these days. Mesen uses it as its default, etc. It’s only like 3% off from normal 4:3, but it avoids looking overly fat when you crop off the top/bottom overscan down to 224 height.

The overarching concept with the border shaders is this: people use integer scaling to get even scanlines/pixel sizes but that leaves a lot of unused black space around the image. These shaders can provide even scanlines and proper scaling at any aspect ratio while filling the leftover space with a nice effect and they include all of the options needed to make a certain type of aesthetic without mucking around with a bunch of the video settings.

I see. Thank you for the explanation. Oh! I also forgot to thank you for adding the brightness option for bigblur, that’s got the effect I was looking for more or less perfect. I super appreciate that.

1 Like

Hey, I’m sure double posting is a sin around here like any good internet forum, and I do apologize for it, but I actually just realized why the image-adjustment shader was breaking in the middle of the chain, and wanted to make sure you got pinged for it. Both bigblur and image-adjustment have a parameter called ZOOM, and bigblur’s default value overrides image-adjustment’s, which means not only does the border get zoomed in, so does the image-adjustment. If I change the name of bigblur’s parameter to BORDER_ZOOM, that allows me to reintroduce image-adjustment into the middle of the chain with no further parameter conflicts.

I’m a weird dude who isn’t running integer scaling and wants his game image touching the top edges of the screen along with overscan correction, so I’m glad I was able to figure that out.

oh nice. good catch. Yeah, I can make that change in the repo, as well.