RetroPie cocktail shader

goldfinger007 · 15 January 2017 09:56

I have built a small cocktail cabinet and I am trying to use hunterk’s cocktail-shader.glsl to create the dual screens. When I first apply it, I get the following screen.

I am doing this by making a cfg file in ~/RetroPie/roms/nes folder called “Contra (USA).zip.cfg”. The content of the file is


video_shader = /opt/retropie/emulators/retroarch/shader/shaders/cocktail-cabinet.glsl
video_shader_enabled = true

Then I tried to apply the rotation to show each player the game and it turned out like this.

It looks even more zoomed in. Here is the new cfg file…


video_allow_rotate = true
video_rotation = 3
video_shader = /opt/retropie/emulators/retroarch/shader/shaders/cocktail-cabinet.glsl
video_shader_enabled = true

I have been trying to mess around to fix this but nothing I have tried seems to work. Or make it worse.

Can someone help me out and point me in the right direction?

hunterk · 15 January 2017 07:54

If you are already running in portrait use the cocktail-cab-portrait shader

goldfinger007 · 15 January 2017 10:00

Thank you hunterk.

Ok, That is starting to look better.

But it is still not stretching out over the entire screen Overlapping each other at the top now. How can I go about moving the screens further apart.

hunterk · 15 January 2017 07:54

Use the ‘preview shader parameters’ menu and adjust the settings in there until it looks good. Write those values down and you can create a preset that passes those values to the shader automatically. The ‘menu shader parameters’ menu should let you save the settings to a preset but I personally prefer making my presets by hand in a text editor.

goldfinger007 · 15 January 2017 10:01

I’ve been trying to find that menu in retropie / retroarch. It is running “1.3.6 - No Core”. And sometimes when I try to navigate the menu, when I try to go back to the previous screen, it won’t change but the title will say “MAIN MENU”

I’ve been trying to manually edit the cocktail-cab-portrait.glsl file. Changing the height, width and location settings to try and make it look better. (What does each number meant to change?)

Should I be doing it this way or should I be editing the GAME.cfg file to make it look good?

hunterk · 15 January 2017 07:54

Open a game, go back into the menu and ‘shaders’ should be down at the bottom. The parameters menus are in there.

The variables are named a little strangely because I just copied them from the non-portrait version, though they don’t really function the same. IIRC, height/width control the height and width of the doubled images and location controls their separation. Sometimes, if you move stuff around, you end up with parts cropped off, in which case you’ll need to use the zoom function along with the height/width to make everything visible.

One other thing: I think you’ll need to make a custom viewport that is the dimensions of your screen (custom_viewport_width of 1080 and custom_viewport_height of 1920 or whatever) because there’s no 9:16 aspect ratio built in. EDIT: I just added 9:16 AR, so it will be available in the latest nightlies.

goldfinger007 · 15 January 2017 07:54

Ok. found out how to open that menu from the game. I also figured out how to update the custom viewport. My config is now


video_shader = "/opt/retropie/emulators/retroarch/shader/shaders/cocktail-cab-portrait.glsl"
video_shader_enabled = "true"
aspect_ratio_index = "22"
custom_viewport_width = "1024"
custom_viewport_height = "1280"
custom_viewport_x = "0"
custom_viewport_y = "0"

But now, the shader is not being applied. When I try to manually set it in the retroarch menu, I see the config there but it is not being applied. When I manually apply it, it starts to work.

Thank you for pointing me in the right direction hunterk.

also, I got the cocktail-cab-portrait.glsl from this link

github.com

libretro/glsl-shaders/blob/master/misc/cocktail-cabinet.glsl

#pragma parameter height "Image Height" -1.145 -6.0 6.0 0.01
#pragma parameter width "Image Width" 2.0 0.0 10.0 0.05
#pragma parameter location_y "Image Location Y" 0.75 -4.0 4.0 0.005
#pragma parameter location_x "Image Location X" -0.5 -4.0 4.0 0.005
#pragma parameter ZOOM "Image Zoom" 1.0 0.0 2.0 0.005

#if defined(VERTEX)

#if __VERSION__ >= 130
#define COMPAT_VARYING out
#define COMPAT_ATTRIBUTE in
#define COMPAT_TEXTURE texture
#else
#define COMPAT_VARYING varying
#define COMPAT_ATTRIBUTE attribute
#define COMPAT_TEXTURE texture2D
#endif

#ifdef GL_ES
#define COMPAT_PRECISION mediump

This file has been truncated. show original

goldfinger007 · 15 January 2017 10:01

When I do apply the shader manually I get this when I try to play the game.

funny thing is that the “video” that plays during the opening is find. Only during game play.

Log file here…

http://pastebin.com/s50ibWGs

hunterk · 15 January 2017 07:54

Ah, yeah, that’s an issue with GLES that I have to work around but always forget about… Try this updated version, which shouldn’t have that problem anymore and has a few more parameters to tweak:

github.com

libretro/glsl-shaders/blob/master/misc/cocktail-cab-portrait.glsl

#pragma parameter height "CocktailTable Image Height" 1.99 -3.0 3.0 0.01
#pragma parameter width "CocktailTable Image Width" 0.8 -5.0 5.0 0.05
#pragma parameter location "CocktailTable Image Location" -0.235 -2.0 2.0 0.005
#pragma parameter zoom "CocktailTable Zoom" 0.51 -2.0 5.0 0.01

#define mul(a,b) (b*a)

#if defined(VERTEX)

#if __VERSION__ >= 130
#define COMPAT_VARYING out
#define COMPAT_ATTRIBUTE in
#define COMPAT_TEXTURE texture
#else
#define COMPAT_VARYING varying 
#define COMPAT_ATTRIBUTE attribute 
#define COMPAT_TEXTURE texture2D
#endif

#ifdef GL_ES

This file has been truncated. show original

goldfinger007 · 15 January 2017 10:01

Thanks hunterk. Now it looks like this.

goldfinger007 · 15 January 2017 07:55

I’m getting close. I had to modify the bottom code to look like this


void main()
{
    vec4 _color;




//fix for clamping issues on GLES
vec2 fragCoord1 = TEX0.xy * InputSize / TextureSize;
vec2 fragCoord2 = TEX1.xy* InputSize / TextureSize;


_TMP0 = vec4(0.0);
if ( fragCoord1.x < 0.9999 && fragCoord1.x > 0.0001 && fragCoord1.y < 0.9999 && fragCoord1.y > 0.0001 )
_c0005 = TEX0.xy + vec2( location_x, separation + location_y);
_TMP0 = COMPAT_TEXTURE(Texture, _c0005);
_TMP1 = vec4(0.0);
if ( fragCoord2.x < 0.9999 && fragCoord2.x > 0.0001 && fragCoord2.y < 0.9999 && fragCoord2.y > 0.0001 )
_c0007 = TEX1.xy + vec2( -location_x, separation - location_y);
_TMP1 = COMPAT_TEXTURE(Texture, _c0007);


    _color = _TMP0 + _TMP1;
    FragColor = _color;
    return;
}
#endif

I have no idea on what i’m doing…

goldfinger007 · 15 January 2017 10:01

Now I have this for the main…


void main()
{
    vec4 _color;




//fix for clamping issues on GLES
vec2 fragCoord1 = TEX0.xy * InputSize / (TextureSize * vec2(0.9999));
vec2 fragCoord2 = TEX1.xy * InputSize / TextureSize;


_TMP0 = vec4(0.0);
if ( fragCoord1.x < 0.9999 && fragCoord1.x > 0.0001 && fragCoord1.y < 0.9999 && fragCoord1.y > 0.0001 )
_TMP0 = COMPAT_TEXTURE(Texture, TEX0.xy + vec2( location_x, separation + location_y));


//_TMP1 = vec4(0.0);
//if ( fragCoord2.x < 0.9999 && fragCoord2.x > 0.0001 && fragCoord2.y < 0.9999 && fragCoord2.y > 0.0001 )
//_TMP1 = COMPAT_TEXTURE(Texture, (TEX1.xy + vec2(-location_x, (separation - location_y))));


    _color = _TMP0 + _TMP1;
    FragColor = _color;
    return;
}

Now when the separation parameter is 0.0, the image looks fine. Once I start to decrease the value to -0.1, it starts to make the top pixels stretch.

hunterk · 15 January 2017 07:55

The issue is that GLES doesn’t have the same clamp_to_border function that desktop GL has, which is what typically keeps the edge pixels from stretching off into infinity. So, we have to sort of fake that by telling the shader to only show pixels that are slightly in from the edges. If they’re still extending, that means that the if statements aren’t doing their job well enough, so try moving the checks in by an order of magnitude (that is, change the 0.9999s to 0.999 and the 0.0001s to 0.001) and see if that helps.

goldfinger007 · 15 January 2017 10:01

YESSSS!!! Here is the code that works for me.


void main()
{
    //vec4 _color;
    output_dummy _OUT;

    //fix for clamping issues on GLES
    vec2 fragCoord1 = TEX0.xy * InputSize / TextureSize * vec2(0.9999);
    vec2 fragCoord2 = TEX1.xy * InputSize / TextureSize * vec2(0.9999);

    _TMP0 = vec4(0.0);
    if ( fragCoord1.x < 0.9999 && fragCoord1.x > 0.0001 && fragCoord1.y < 0.9 && fragCoord1.y > 0.1 )
    _TMP0 = COMPAT_TEXTURE(Texture, TEX0.xy + vec2( location_x, separation + location_y));

    _TMP1 = vec4(0.0);
    if ( fragCoord2.x < 0.9999 && fragCoord2.x > 0.0001 && fragCoord2.y < 0.9 && fragCoord2.y > 0.1 )
    _TMP1 = COMPAT_TEXTURE(Texture, TEX1.xy + vec2(-location_x, separation - location_y));

    _OUT._color = _TMP0 + _TMP1;
    FragColor = _OUT._color;
    return;
}

It only worked by reducing the y fragCoord.

Now I just need to play with it a bit more to get the top few pixels to show. Thank you for putting up with my issue hunterk.

hunterk · 15 January 2017 07:55

eyyyy, looking good

glad I could help!

dankcushions · 15 January 2017 07:58

so if i’m following this right, this shader needed to be changed for GLES devices? any chance this GLES version could be uploaded to the shader repo for others? seems cool!

hunterk · 15 January 2017 07:58

It’s actually already done. Here’s the landscape version:

github.com

libretro/glsl-shaders/blob/master/misc/cocktail-cabinet.glsl

#pragma parameter height "Image Height" -1.145 -6.0 6.0 0.01
#pragma parameter width "Image Width" 2.0 0.0 10.0 0.05
#pragma parameter location_y "Image Location Y" 0.75 -4.0 4.0 0.005
#pragma parameter location_x "Image Location X" -0.5 -4.0 4.0 0.005
#pragma parameter ZOOM "Image Zoom" 1.0 0.0 2.0 0.005

#if defined(VERTEX)

#if __VERSION__ >= 130
#define COMPAT_VARYING out
#define COMPAT_ATTRIBUTE in
#define COMPAT_TEXTURE texture
#else
#define COMPAT_VARYING varying
#define COMPAT_ATTRIBUTE attribute
#define COMPAT_TEXTURE texture2D
#endif

#ifdef GL_ES
#define COMPAT_PRECISION mediump

This file has been truncated. show original

and here’s the portrait version:

github.com

libretro/glsl-shaders/blob/master/misc/cocktail-cab-portrait.glsl

#pragma parameter height "CocktailTable Image Height" 1.99 -3.0 3.0 0.01
#pragma parameter width "CocktailTable Image Width" 0.8 -5.0 5.0 0.05
#pragma parameter location "CocktailTable Image Location" -0.235 -2.0 2.0 0.005
#pragma parameter zoom "CocktailTable Zoom" 0.51 -2.0 5.0 0.01

#define mul(a,b) (b*a)

#if defined(VERTEX)

#if __VERSION__ >= 130
#define COMPAT_VARYING out
#define COMPAT_ATTRIBUTE in
#define COMPAT_TEXTURE texture
#else
#define COMPAT_VARYING varying 
#define COMPAT_ATTRIBUTE attribute 
#define COMPAT_TEXTURE texture2D
#endif

#ifdef GL_ES

This file has been truncated. show original

The shaders in that repo get pulled in and overwrite the script-translated ones, so if you update your glsl shaders, they should be in there.

fatavatar · 15 January 2017 07:59

I ended up having to fairly intrusively modify this shader (cocktail-cab-portrait) in order to work for NES on 1080p 16:9, so if anyone would like the end code I will more than happily provide it. However using either the modified or stock version: when I try this with the atari2600 emulator I just get a black screen. Has anyone had any luck with this? I’m fairly new to setting one of these up, so what would cause a difference between these two emulators in regards to the shader?

hunterk · 15 January 2017 07:59

sure, i’d be interested in seeing your modifications.

Hmm, looks fine with Stella here on my end.

fatavatar · 15 January 2017 07:59

So looking back, i was drunk off my ass last night. I would not classify this as ‘intrusive’ per se :). Here are the parameter settings that eventually worked for me:


#pragma parameter height "CocktailTable Image Height" 0.99 -3.0 3.0 0.01#pragma parameter width "CocktailTable Image Width" 1.99 -5.0 5.0 0.05
#pragma parameter separation "CocktailTable Image Separation" -0.45 -2.0 2.0 0.005
#pragma parameter zoom "CocktailTable Zoom" 0.5 -2.0 5.0 0.01
#pragma parameter location_x "CocktailTable Location X" 0.0 -1.0 1.0 0.01
#pragma parameter location_y "CocktailTable Location Y" 0.0 -1.0 1.0 0.01

And the modified main() block:


void main()
{
    output_dummy _OUT;




    //fix for clamping issues on GLES
    vec2 fragCoord1 = TEX0.xy * InputSize / TextureSize * vec2(0.9999);
    vec2 fragCoord2 = TEX1.xy* InputSize / TextureSize * vec2(0.9999);


    if ( fragCoord1.x < 0.9999 && fragCoord1.x > 0.0001 && fragCoord1.y < 1.9999 && fragCoord1.y > 0.
4001 )
    {
        _c0005 = TEX0.xy + vec2( location_x, separation + location_y);
        _TMP0 = COMPAT_TEXTURE(Texture, _c0005);
    }
    if ( fragCoord2.x < 0.9999 && fragCoord2.x > 0.0001 && fragCoord2.y < 1.9999 && fragCoord2.y > 0.
4001 )
    {


        _c0007 = TEX1.xy + vec2( -location_x, separation - location_y);
        _TMP1 = COMPAT_TEXTURE(Texture, _c0007);
    }
    _OUT._color = _TMP0 + _TMP1;
    FragColor = _OUT._color;
    return;
}

I have never dealt with anything like shaders before, but the syntax looks like C/C++. If thats the case whatever is in the repo now looks like it has a bug with those if statements. Additionally the y coordinate cutoffs pretty much ended up cutting off the image (looks like it may have been OK for a 4:3 aspect ratio?) and NOT fixing that clamping issue, so I had to adjust those as well. Having basically no understanding of what things did, this ended up being a two-hour needle in haystack situation :(.