HowTo: Generate Pretty, Curated MAME Playlists the Easy(ish) Way

[QUOTE=roldemort;50646]Line 42 in your new github code should be changed into

If (InStr(A_LoopField, "<game name=") or InStr(A_LoopField, "<description>"))  ;only keep relevant lines

Sorry about that. I will fix the error in my post above too, in case someone comes and uses only the snippet.[/QUOTE]

Thanks! I made the correction.

I tested the github code now. Your line 78

 RegExMatch(datcontents, needle, datname)

need a change into

 RegExMatch(datcontents, "U)" needle, datname)

Without that change the script only save one image.

I ran a test on “MAME - ROMs (v0.179_XML).dat” and “pS_snap_fullset_170.zip” The test took 66 seconds to complete . Pretty good optimization from the one hour for the script on the first page of this thread! :slight_smile: Almost all time is spent on the regex step. I think that can be cut down further. Give me a few minutes…

You caught me. I’ve only been dropping in individual image files to rename as I’ve been testing this! :slight_smile: Good catch.

I made the change. I also renamed the github repository, since this script isn’t really a Lakka script (yet).

A new link to the file: https://github.com/markwkidd/retroarch-arcade-playlist-helpers/blob/master/friendly-image-renaming.ahk

I’ll edit my prior posts to point to this location.

New version runs the same test in 13 seconds. Method: the loop first finds the filename position in the slimmed dat and starts a simplified regex from that position.

The new version replaces your lines 66 to 105 with this

posi = 1
needle = <description>(.*)</description>

Loop, Parse, ThumbnailFileList, `n, `r
{
if A_LoopField =
    continue
SplitPath, A_LoopField,,,,filename     ;### trim the file extension from the name
posi := InStr(datcontents, """" filename """",false,posi)  ;filename position in datcontents
RegExMatch(datcontents,"U)" needle, datname, posi) ;start regex from filename position
if !datname1
 continue
newname := char(datname1)
FileCopy, %artsource%\%filename%.png, %destinationfolder%\%newname%.png, 1
}

;change forbidden chars
char(x) {
x := StrReplace(x, "'", "'")
x := StrReplace(x, "&", "_")
x := StrReplace(x, "&", "_")
x := StrReplace(x, "\", "_")
x := StrReplace(x, "/", "_")
x := StrReplace(x, "?", "_")
x := StrReplace(x, ":", "_")
x := StrReplace(x, "<", "_")
x := StrReplace(x, ">", "_")
x := StrReplace(x, "*", "_")
x := StrReplace(x, "|", "_")
return x
}

This new version also saves more images to destinationfolder (13507 images compared to 12629 images). Not sure why there is a difference.

deleted. weird double post

Not to muddy the waters, but there has also been a development in the libretro-thumbnails repo today. A complete set of MAME thumbnails was accepted: https://github.com/libretro/libretro-thumbnails/tree/master/MAME

That set was generated using a prior variation on this same script. The filenames in the RetroArch MAME thumbnail pack are therefore a 1:1 match for what we’ve been working with here, for games up to MAME 0.159. I’m thinking now about how that thumb set can relate to Alexandra’s work here, which I think is ultimately the way I would like my own collection to be displayed. 8)

Back to the script at hand. I merged in your changes, although slightly altered per my OCD naming and formatting preferences.

I can’t really say why there would be more files with your new version, but I’ll tell you why I got interested in speed enhancements to the original script by yourself and Tetsuya79. After I modified your original work to suit MAME 0.78 and Lakka a few months back, I was having erratic results.

Each time I would run the playlist generator it would come up with a different number of playlist entries. After a while I came to realize that if I ran the same script on a faster CPU I stood a better chance of getting the full list of ROMs added to the playlist, but it still wasn’t a guarantee. AutoHotKey was failing in some silent way, apparently ‘giving up’ on matching some playlist entries if it was taking too long.

Once I incorporated my speed increases from upthread, I started getting full MAME 2003 playlists every time. I wouldn’t say whether that is related to your experience now, but it is at least an interesting coincidence.

Good to hear.

Well, if at some later time there is a need to update the MAME snaps in the repo my last code change will save the person doing it approximately 50 seconds. Enjoy that chunk of extra life! :smiley:

I also hope RetroArch will get an option for some type of submenues inside the MAME playlist, something like in Alexandra’s first post.

About the difference in number of output files. I compared the outputs of the newest version (with the posi steps; 13507 images in total) and the previous version (more complex regex; 12629 images). The new output hade 1128 images not in the old output. But the old output also had 250 images not in the new output! Which is weird, not sure why yet.

Anyway, if the github set is complete for the games supported by the latest MAME core then all is good anyway. Though this weird difference could be useful to troubleshoot when moving the core to a newer MAME version.

edit: appears there is 13122 images in https://github.com/libretro/libretro-thumbnails/tree/master/MAME/Named_Snaps

We may be pushing some limit of AutoHotKey, or stumbling on a bug with their code. I don’t really see where the discrepancy would come in from the script.

About the number of files in the github thumbnail repo: When I put together the github set, I started with MAME 0.159, then added in thumbnails with the different name variations used by the Progetto SNAPS datfile in 0.139, 0.78, and 0.37b5. So the current repo is focused on the ‘fixed’ or ‘historical’ libretro MAME cores.

It would only take a few tweaks to this script as well, as some scripted resizing for images wider than 512px, in order to bring the github repo up to date for MAME 0.170.

Love what I’m seeing here; thanks for the all the optimizations. I’ll update the top posts when the dust clears.

I am slightly concerned that thumbnails for some legit games may end up getting filtered out, given the output discrepancies y’all are noticing. I may test later on.

hunterk: Great icon idea, though kind of annoying to sprite-rip from the arcade version. :slight_smile:

It irked me that the newer code weirdly failed to rename some snaps that the old code had no problem with. So I took another look now. This fixes the issue. Replace line 73 in https://github.com/markwkidd/retroarch-arcade-playlist-helpers/blob/master/friendly-image-renaming.ahk with

posi := InStr(datcontents, "game name=""" filename """",false,posi)     ;### filename position in datcontents
if !posi
 posi := InStr(datcontents, "game name=""" filename """")
if !posi
 continue

There were actually two errors. 1 My InStr previously looked only for the filename e.g. “ckong”. But e.g. cloneof=“ckong” comes earlier than game name=“ckong” in the .dat, which tripped things up. Checking for “game name=“ckong”” fixes that. 2 I had assumed the .dat game names were A-Z sorted. But after the game “zzyzzyxx2” the .dat list is in fact unsorted. When a file A was located in that end part the next file B in the loop wasn’t found because the search for it started way down at A’s position near the end of datcontents.

Test results with this script: outputs 13712 renamed files for “MAME - ROMs (v0.179_XML).dat” and “pS_snap_fullset_170.zip”. outputs 11935 renamed files for “MAME 159.dat” (source) and “pS_snap_fullset_170.zip”.

Still differs from the 13122 in the https://github.com/libretro/libretro-thumbnails/tree/master/MAME/Named_Snaps .

That may explain the difference.

(doublepost)

It would be an incredible coincidence if the processed 170 set had the same number of files as the github repo. All new games (and games that have been renamed) from MAME 0.160-0.170 are missing from the repo, and then there are also some additional files in the github repo corresponding to names from pre-0.159 MAME DATs. So the total number is close, but that is unrelated to our question of the script processing all of the thumbs.

Think of the github repo as a ‘rollup’ set designed from 0.37b5-0.159 – it has more than any one of those sets alone, but is missing the very newest stuff entirely. Maybe I’m repeating myself, but with so many MAME thumbnail scripts being discussed I want to try to be clear what the difference is.

At any rate, I’m excited to make your latest update roldemort, on the assumption that you have zapped the source of this discrepancy in thumbnail counts for 0.170. Rock and roll. Next time I have a dedicated few hours I will try to create an update to the github repo that brings in all of the 0.160-0.170 thumbs.

Ok, got it.

I ran the latest github version of the script on “MAME - ROMs (v0.179_XML).dat” and “pS_snap_fullset_170.zip” again, but this time I added script steps to copy files into three folders: 1 detected dummy files 2 successfully renamed files 3 non-dummy files that ran into a “continue” step in the script and wasn’t renamed.

Result: no files ended up in folder 3. Good!

Next I ran a crc32 check on folder 1, the dummy files. Any file with < 10 crc32 duplicates was copied to a new folder, which I browsed. I found only screenless dummy games there.

TL;DR the script now works on “MAME - ROMs (v0.179_XML).dat” and “pS_snap_fullset_170.zip”.

Two updates I wanted to share with the thread.

  1. As of the next nightly RetroArch image, there is automated support for translating prohibited characters in playlist entries to underscores. That means that playlist files no longer have to scrub these characters from the “display name” – For example, when RA sees a playlist entry titled 1943: The Battle of Midway it will automatically look for a thumbnail file called 1943_ The Battle of Midway. This means that the the playlist generator script can keep the ‘fancy names’ intact as long as the thumbnail renaming script replaces those characters with _[B].

[/B] I haven’t tried to edit the playlist script to reflect this change, and current playlists ought to be backwards-compatible once the update hits.

  1. The new RA code change clarifies the list of prohibited filename characters to be the same as No-Intro, which is one character different than the list I have been using. Here’s the No-Intro list: & \ / ? : ` < > * |

Fortunately I don’t think that MAME uses the backtick character in game names, so this doesn’t really affect us. I have, however, added a line to scrub the backtick to the AutoHotKey scripts on github just to be sure.

As of now, the Libretro thumbnail repository should support this and other MAME playlist generation scripts with an up to date set of thumb images. Exciting!

[B]I have been working on an updated, one-step script that can use the thumbnails from the Libretro thumbnail repository and put them in the correct folders at the same time as generating these multiple themed MAME playlists.

Perhaps more importantly, it can be configured via a new GUI.[/B]

Access the experimental one-step playlist/thumbnail script

Notes:

  1. Rather than using the Progetto SNAPS thumbnail files or another source, this script expects the libreto MAME thumbnail pack from http://thumbnailpacks.libretro.com/ to be unzipped in a location which the script has access to
  2. This script can download individual thumbnail files from http://thumbnails.libretro.com/ that are represented in the local ROM set if there is not a thumbnail pack available. This can theoretically be used to build a thumbnail set for a small collection of ROMs. Also handy for testing!

If you have worked with Alexandra’s original scripts you will probably be able to figure out how this one works. Speaking of which, Alexandra, apologies again for posting so much in your thread. I hope some of this is useful to you in addition to being fun for me!

I wuold share with you my step for categorized lpl and thumbnails

  1. For lpl use the bat script for separate the ROMS
  2. In RetroArch scan every single Roms folder (like Action) and rename the playlist generated in Action.lpl (do this for all other Roms Categorized Folder) Remember to rename frist to start scan new folder
  3. Downloaded the Progetto SNAPS thumbnails for Arcade and renamed the folder in “Action”
  4. Using this command for create symbolic link for other playlists

ln -sT /storage/thumbnails/Action/ /storage/thumbnails/BeatEmUp

Replace BeatEmUp with the other Roms categorized folder name

What do you think? Very sorry for my bad english

Very clever, gleam2003.

If I understand correctly (use my pre-made batch files to copy roms, make RA create a playlist for each folder, share thumbnails via symbolic link) that will make per-genre playlists with images and without using AutoHotKey scripts. I haven’t tested it but it seems plausible at first glance.

That said, I believe the playlists would only show raw rom filenames, instead of the expanded full game names, right?

[QUOTE=Alexandra;52555]Very clever, gleam2003.

If I understand correctly (use my pre-made batch files to copy roms, make RA create a playlist for each folder, share thumbnails via symbolic link) that will make per-genre playlists with images and without using AutoHotKey scripts. I haven’t tested it but it seems plausible at first glance.

That said, I believe the playlists would only show raw rom filenames, instead of the expanded full game names, right?[/QUOTE]

Full games names are show, not the raw

Yes, not use AutoHotkey Script, more simple for me

Note: Important to rename the lpl every scan because they use not the ROM folders as name but the Emulator Name and all ROMS use the same emulator Note 2: Need to set the default emu for created playlist or the ask you all time you run a ROM

Great thread. However, I think I’m still missing a step that is preventing my MAME roms from showing up in LAKKA.

Here’s how I understand it - can you please tell me if I’m not grasping something correctly…

Here’s how I understand it:

Lakka wants 0.78 Romsets that follow the Progetto-SNAPS MAME 0.78 naming structure.

I have some 0.78 MAME roms downloaded, and I have that Progetto DAT file, but I still can’t seem to rename properly them to get Lakka to see them.

Move followed some scanning and rebuilding guides, it still no luck.

Do I have the versions correct?

Thanks!!