RetroManager - overlay manager

[Tarrasque]

Hi all.

I developed a little utility to massively manage overlays and other config files for RetroArch.

Current feature list:

• It can detect the “viewport” area inside the overlay and generate RetroArch config files so that a game runs (almost) perfectly inside it.

• It can resize overlays to the desired size keeping the aspect ratio.

• The resize operation can be based on the full size of the overlay or the viewport, thus maximizing the game playable area and cropping “excess” overlay.

You can find it on GitHub. It’s a command line utility written in Python so it should easily running any system, provided you have Python 3.* installed.

It’s still very rough and I’d like to have feedback to fix and improve it.

tarrasque73/RetroManager/blob/master/README.md

[Tarrasque]

Some examples of overlay manipulation, using one of the excellent overlays made by @OrionsAngel.

python retromanager.py overlay_resize "MAME 2016" dkong

Basic rescale, using the original (1920x1080 in this case) dimensions. Used this way the overlay image is simply copied to the destination folder with no processing altogether.

python retromanager.py overlay_resize "MAME 2016" dkong -tx 960 -ty 720 -bc FF0000

Rescale to 4:3 (960x720), using an ugly red to showcase the backgroud filling.

python retromanager.py overlay_resize "MAME 2016" dkong -rm inner -mx 30 -my 30 -tx 1440 -ty 720 -bc FF0000

Rescale to HDReady resolution maximizing the viewport area and leaving some margin around it.

[Tarrasque]

Running the above examples substituting “generate_all” for “overlay_generate” will also create the config files needed for RetroArch to configure the viewport size and, if you want them, a basic crt shader preset.

They are created under the output folders and must be copied to the folders where RetroArch expects them (default is \config<core>)

The file created for command python retromanager.py generate_all "MAME 2016" dkong is:

aspect_ratio_index = "23"
custom_viewport_height = "859"
custom_viewport_width = "657"
custom_viewport_x = "629"
custom_viewport_y = "109"
input_overlay = "C:\Emulation Stuff\Retroarch\layouts\\dkong.cfg"
input_overlay_enable = "true"
input_overlay_hide_in_menu = "false"
video_shader_enable = "true"
video_fullscreen = "true"

The file for other commands differs only in the calculated custom-viewport_* entries.

Here’s a screenshot of the game running with the following shader.

#reference "..\..\shaders_cg\crt\crt-geom.cgp

[ArsInvictus]

It seems like with your template setup in these scripts it would be pretty easy to generate a MAME LAY file as well. That would be fantastic for people like me who need to generate for both platforms. Is there any chance you could add a switch and template to generate those as well? A basic LAY file is actually pretty simple and you would only need to swap out the file.png and the x/y/height/width values.

Here’s an example LAY file for my 4K Dragon Breed overlay for instance:

<!-- dbreed.lay -->

<element name="bezel">
	<image file="dbreed.png" />
</element>

<view name="artwork">
	<screen index="0">
		<bounds x="1005" y="420" width="1825" height="1400" />
	</screen>
	<bezel element="bezel">
		<bounds x="0" y="0" width="3840" height="2160" />
	</bezel>
</view>

[Tarrasque]

Hi.

Yes, it is simple to add support to other layout files. Once you get a basic template, it’s basically the same stuff. The whole poiint of the project was, for me, to have a centralized tool for massive batch manipulation of configuration files and assets for Retroarch/MAME, so I’m glad to improve it.

Since I don’t know much the mame layout format, could you help me understand it a little better? I guess the dimensions in the tag are what I call the “viewport”, i.e. the actual part of the monitor where the game is shown. And if i’m correct the tags contains the cohordinates depending on your target monitor resolution, right?

I’ll be happy to add support for the mame layout format as soon as I can, but probably will be after the Christmas vacations.

Next steps will pobably be functions to generate and maintain playlist files.

[ImnoTapLumber]

@Tarrasque: maybe this can help you:

https://wiki.mamedev.org/index.php/LAY_File_Basics_-_Part_I

(There is no Part II if you’re wondering )

[ArsInvictus]

Thanks @Tarrasque, yes, that’s all correct. LAY files can be more complicated and has support for multiple views for the same game, but for cases like mine it’s very simple and I just have a screen and bezel configuration.

The screen is the viewport, or transparent area, while the bezel entry contains the dimensions for the overall screen.

The link provided by @ImnoTapLumber is excellent and it’s where I learned how to create these.

[ImnoTapLumber]

Here is a more detailed guide:

https://docs.mamedev.org/techspecs/layout_files.html

---

I guess the dimensions in the tag are what I call the “viewport”, i.e. the actual part of the monitor where the game is shown. And if i’m correct the tags contains the cohordinates depending on your target monitor resolution, right?

No.
If in the view tag a bounds tag is defined, this defines the width, height, and placement of the objects and is scaled and centered with correct aspect ratio to your display size.
It becomes your “canvas”. All other elements have to take into account the offset and width/height of the bounds tag.

	<view name="Artwork">
        <bounds x="512" y="0" width="1024" height="768" /> <!-- this is your "canvas"-->
		<bezel element ... etc.
			<bounds ... etc.

Everything outside this canvas will be cut/cropped.

If no bounds is defined, then your biggest element is scaled to fit your display. (with correct aspect ratio. [can produce black bars])
This can be an addition of 2 elements as well, if the offset+image is bigger than the other elements.

[ImnoTapLumber]

A more visual explanation:

The bounds tag (red) is scaled and centered to fit your display resolution(black) (no matter how small/big the bounds width/height is):

If no bounds tag is used, everything is scaled/centered based on the largest width/height of your elements
For example a bezel (green) and a instruction image with x/y offset (blue)
The red dotted line symbolizes the final size of your artwork:

If the bounds tag is defined, everything outside will be cut/cropped (dark blue).
A smaller bezel will produce black bars (dark red)

Offset in the bounds tag affects the coordinates of the other elements
e.g. if the offset in the bounds tag is greater than in the other tag, everything outside will be cut/cropped
Here the offset of the bounds is x="240" y="100"
but the bezel starts at x="0" y="0" (which is in fact x="-240" y="-100" from the bounds left top corner) The dark green area is not visible on the display (the dark red area is black as well on your monitor):

The different view element tags are rendered as follow:

• bezel (blend mode: normal) opaque on top of the rest
• backdrop (blend mode: multiply) to simulate a image behind the gamescreen
• overlay (blend mode: screen) usually for black & white games to make them more colorful
• screen (blend mode: normal) the actual gamescreen

Example Space Invaders Deluxe:
bezel on top,
two backdrop images,
one color overlay,
game,
display/monitor size

[Tarrasque]

Interesting. The MAME layout files seem much more complex and versatile than I thought.

Considering just the basic layout sample that @ArsInvictus provided, I was thinking of converting it into a template like this for my purposes:

<element name="bezel">
	<image file="{imagename}" />
</element>

<view name="artwork">
	<screen index="0">
		<bounds x="{viewport_x}" y="{viewport_y}" width="{viewport_width}" height="{viewport_height}" />
	</screen>
	<bezel element="bezel">
		<bounds x="0" y="0" width="{bezel_width}" height="{bezel_height}" />
	</bezel>
</view>

where the “curly variables” would be filled in with the viewport values the scripts calculates form the bexel transparent area, and with {bezel_width} and {bezel_height} set to the actual bezel size.

This of couse assumes that the bezel size is equal (or at least fits without rescaling) to the monitor size. But this is also the assumption of the already existing overlay generation functions, that’s why you can specify a target size to resize the overlay image before calculation.

[ArsInvictus]

This would work for my purposes. If I had something as complex as the Space Invaders example I could just do that by hand.

My thinking is that I would like to create a script where I drop a PNG file into an image directory and then run “build” on that directory. The script would then use the name from that PNG file to create a mame directory, the LAY file based on the dimensions of the transparent area of that PNG and then compress it into a zip file for distribution. It would also create the CFG files necessary for Retroarch and the proper directory structure for that as well. It would be a huge time saver and would allow me to also generate the precisely correct configurations for all my overlays without manual effort.

One question, could an offset be supplied for a set number of pixels that would allow me to define a border space around the screen image area? The monitor graphic elements are curved and I might need to account for that curvature in the calculations.

[Tarrasque]

This would work for my purposes. If I had something as complex as the Space Invaders example I could just do that by hand.

My thinking is that I would like to create a script where I drop a PNG file into an image directory and then run “build” on that directory. The script would then use the name from that PNG file to create a mame directory, the LAY file based on the dimensions of the transparent area of that PNG and then compress it into a zip file for distribution. It would also create the CFG files necessary for Retroarch and the proper directory structure for that as well. It would be a huge time saver and would allow me to also generate the precisely correct configurations for all my overlays without manual effort.

I’m not sure if I got your point entirely correct (pardon me but English is not my primary language).

If what you want is to manipulate the layout or cfg files, let’s say zip them, move them, then that part is currently out of the script on purpose.

I think it’s better to focus on making the script do one thing only and as better as I can. Most of the things you wish can be easily in “postproduction”, after you run the script.

For example, on windows you can create zip files with overlay cfgs and pngs easily with a command like:

for %i in (*.cfg) do zip.exe a %~ni.zip %~ni.cfg %~ni.png

So, I’m not thinking about this at the moment, nor do I thinkthings that can be done easily by the native operating system batch scripting will ever be part of RetroManager.

We can think about - and share, of course - small scripts that leverage RetroManager to achieve these results.

One question, could an offset be supplied for a set number of pixels that would allow me to define a border space around the screen image area? The monitor graphic elements are curved and I might need to account for that curvature in the calculations.

Cuttently, the “transparency detection” feature works like this:

On each of the 2 directions, running along the middle section, I mark the first and the last “transparent enough” pixel. Transparent enough is a threshold level that allows to take in account overlays, such the excellent @Orionsangel sets, that have glass scratch marks over the screen.

Plus, just in order to be safe, I require that you find at least X pixels under the threshold in a row in order to mark the start of the transparent area, so you have a little “margin”.

Currently these values are hard coded in the script. There’s no reason I can’t expose them as command line switches, and I will in the near future.

[ArsInvictus]

Good to hear about the option to increase the transparency margin. Sorry for the confusion @Tarrasque , don’t worry I wasn’t suggesting your script to do all that build work, but I would create a script myself that leverages yours and just loops over the files as you suggested there

Once you add the MAME LAY file support as you suggested, I think I can do everything I need to automate my configuration processes, a huge time saver and improvement in quality.

Thank you for your work!

[Tarrasque]

Thanks. I do it for fun, but I hope it can be of help.

I just uploaded to github a new version. I haven’t had the time to update the documentation, so here’s a small breakdown of the main changes.

• the image resizing and manipulation has been stripped off the command “overlay_resize” and isis now it’s own command “bezel_resize”. The configuration file generation is now “overlay_generate”. Unless your bezels are already of the correct size you need them for your monitor, you’ll have to run it before “overlay_generate” or “layout_generate”.

• new command “layout_generate”. Works more or less exactly as “overlay_generate” but intended to write MAME config files instead of RetroArch config files. It has it’s own template directory and other stuff.

• when generating files, the commands now look for a temlate file named like the game (f.i. pacman.lay or pacman.cfg) in the appropriate template directory. If they can’t find it then they’ll look for a default template named “_default.lay” or whatever. This way you can have specific templates for games whose configurations are non standard.

There are other smaller changes which I can’t remember on the top of my head. please try it and let me know if anything does not work as expected.

[Tarrasque]

By the way, since I don’t know anything about MAME style layouts, I don’t even know how to use them to test them.

I understood that RetroArch should be able to use them since 1.8.2

I tried to follow the steps on Libretro’s news in the article posted some time ago, but nothing seems to happen. I don’t know if that’s because there’s something wrong in the files I produce or what.

Can anyone help?

[ArsInvictus]

Awesome, thank you! I’ll give it a shot, but might not be until next week. I will let you know if I have any issues

[ArsInvictus]

I haven’t tried using them in Retroarch, but you can find some examples on my dropbox that might help you out.

All I do with MAME is drop them into the artwork folder and enable the artwork in the MAME ini files and it worked automatically for me.

Here’s the dropbox with my mame (and retroarch files):

Public_4K_Bezels

[Tarrasque]

I downloaded your bezels and did a test run.

I could not test the actual output because I only have RetroArch on the machine I’m writing from, and no regular MAME. Still, the test is pretty encouraging.

This is the template I made taking your sf2 layout as a guide.

<mamelayout version="2">
	
	<element name="bezel">
		<image file="{imagename}" />
	</element>

	<view name="artwork">
		<screen index="0">
			<bounds x="{viewport_x}" y="{viewport_y}" width="{viewport_width}" height="{viewport_height}" />
		</screen>
		<bezel element="bezel">
			<bounds x="0" y="0" width="{bezel_width}" height="{bezel_height}" />
		</bezel>
	</view>
	
</mamelayout>

This is the product of running python retromanager.py layout_generate "MAME 2016" sf2 (no resizing options, so only config file generation)

The result is:

<mamelayout version="2">
	
	<element name="bezel">
		<image file="sf2.png" />
	</element>

	<view name="artwork">
		<screen index="0">
			<bounds x="1146" y="480" width="1558" height="1190" />
		</screen>
		<bezel element="bezel">
			<bounds x="0" y="0" width="3840" height="2160" />
		</bezel>
	</view>
	
</mamelayout>

Which is just a bunch of pixels off your original values:

<mamelayout version="2">
	
	<element name="bezel">
		<image file="sf2.png" />
	</element>

	<view name="artwork">
		<screen index="0">
			<bounds x="1148" y="482" width="1556" height="1196" />
		</screen>
		<bezel element="bezel">
			<bounds x="0" y="0" width="3840" height="2160" />
		</bezel>
	</view>
	
</mamelayout>

Of course this accounts to the automatic transparency detection that will never be as satisfying as hand-placement of objects.

Not bad, for the moment. When you have time to test some layouts, please let me know if there are issues.

[Tarrasque]

I updated the script introducing a third resize mode: “custom”.

I added that because, well I like having all the play areas more or less the same size, whether the game has an horizontal screen or vertical. With inner mode the script would mazimise the viewport to the target motitor size, and this would mean that horizontal 4:3 viewports would be much bigger than vertical once resized to a 16:9 monitor, unless you did a pretty amount of math on game by game bass to compute the right margins.

This is an excerpt from the readme.

• -tm sets the “mode” of the resize operation, and can be “outer”, “inner” or “custom”. Default is “outer”

  • outer mode resizes the bezel based on the dimensions of the original image. Useful when you already have your bezels in the correct dimensions and you’re only interested in viewport detection for config files generation.

  • inner mode resizes the bezel based on the dimensions of the viewport. Useful when you want to maximise the play area of the game on your monitor. In this case some part of the bezel will be cropped.

  • custom mode is similar to inner mode but lets you specify the size of the resized viewport explicitly with parameters -cx and -cy instead of fitting it in the target size, with optional margins. It’s particularly useful when you want all the resized viewport to have the same size, whether they are horizontal or vertical (see examples, below).

Examples:

python retromanager.py "MAME 2016" blktiger -rm custom -cx 1000 -cy 1000 -tx 1920 -ty 1080

python retromanager.py "MAME 2016" gyruss -rm custom -cx 1000 -cy 1000 -tx 1920 -ty 1080

In this examples, the two bezels have different orientations because the first game has an horizontal screen and the second vertical. The commands produce new bezels where the viewports have the same maximum dimension.

[Tarrasque]

I’m implementing some new features, still in heavy development ATM.

Thanks to the feedback and assets kindly provided by @ArsInvictus I’m trying to expand the utility to be a full fledged, albeit simple, “automatic cabinet maker”.

I’m not ready to publish the update code, but here’s a sample of a windows batch file that can take some already prepared assets and scale/resize/combine/paste them to create a vertical cabinet similar to the ones that ArtInvictus handcrafts.

I post the samples here because since I’m coding right now this is the best moment for me to get feedback and advice and make my mind on how to proceed.

the batch script gets basically the game name as a parameter and calls the various functionality of the utility to manipulate a series of assets.

The commands are at the moment basically 2:

cabinet_join that stacks and resizes images in a row, horizontally or vertically.

cabinet_paste pastes an image over a background.

both commands have various switches to align, resize, scale keeping aspect ratio or stretch the images. I won’t document them right now but I hope they are easily understandable.

set game=%1

rem speaker block: first stretches the wood layer on the backgorund, then stretches the speakers over it
python cabinetmanager.py cabinet_paste -bf ..\test\white_background_2160x360.png -if ..\test\speaker-panel.png -of ..\test\temp_speakers_background.png -rm stretch -va center -ha center
python cabinetmanager.py cabinet_paste -bf ..\test\temp_speakers_background.png -if ..\test\speakers.png -of ..\test\temp_speakers_composite.png -rm scale -va center -ha center -rd both

rem top block (marquee and speakers): stacks them vertically
python cabinetmanager.py cabinet_join -if ..\test\%game%_marquee.png ..\test\temp_speakers_composite.png -of ..\test\temp_cabinet_top.png -r scale -d vertical -a center

rem bottom block (metal grid and logo): first stretches the metal grid over the background them paster the logo over it
python cabinetmanager.py cabinet_paste -bf ..\test\white_background_2160x520.png -if ..\test\metal-bottom-border.png -of ..\test\temp_metalgrid.png -rm stretch -va center -ha center
python cabinetmanager.py cabinet_paste -bf ..\test\temp_metalgrid.png -if ..\test\logos.png -of ..\test\temp_cabinet_bottom.png -rm scale -va center -ha center -rd both

rem complete cabinet: paster the upper block, the bottom block and then the bezel over the final cabinet
python cabinetmanager.py cabinet_paste -bf ..\test\white_background_2160x3640.png -if ..\test\temp_cabinet_bottom.png -of ..\test\temp_cabinet_temp1.png -rm scale -va bottom -ha center -rd both
python cabinetmanager.py cabinet_paste -bf ..\test\temp_cabinet_temp1.png -if ..\test\temp_cabinet_top.png -of ..\test\temp_cabinet_temp2.png -rm scale -va top -ha center -rd both
python cabinetmanager.py cabinet_paste -bf ..\test\temp_cabinet_temp2.png -if ..\test\%game%_bezel.png -of ..\test\%game%_cabinet.png -rm scale -va center -ha center -rd both -yo 326

The output cabinet follows (I scaled it back for uploading but the original was 4K)

:

Of course it lacks a lot of nice finishing touches in respect to the ones made by ArsInvictus but I never intend the tool to completely replace an artist’s hand. The goal of my work is creating a tool that can speed up and make easy all the long and boring stuff related to image processing in creating an overlay.

It will be useful for mass production. Mockups. Temporary overlays until you find the time to do a hand-made one, or generic ones where original art is missing.