Scan directory feature in 1.2.2

anon24419061 · 15 January 2017 06:52

our playlists are path dependent, you can’t easily share them

Mydriaze · 15 January 2017 06:52

oh, ok. It seems to be a bit complicated for me, but I will give a try. Thanks for your answer. (you have a PM )

Mydriaze · 15 January 2017 06:52

Alright, I succeed in creating the MAME Playlist. What about CPS123 and NeoGeo? I can’t find the dat file for NeoGeo or CPS123, is there another way to create the playlist?

Mydriaze · 15 January 2017 06:53

Update: this script is working, but not all the games are listed. Do you have another trick guys?

t_m_c · 15 January 2017 07:09

I had to make some changes to the AutoHotkey playlist generation script in order to get it to work with a 0.157 MAME ROM set. I’ll post it here in case these modifications come in handy for someone else.

NOTE: This is the first time I’ve worked with AutoHotkey, so my work is very hacky. Beware!

CHANGES

I added a variable to set the extension for the ROM files, which should allow folks to use sets that use .7z and .zip
I added “sourcefile” to the regex template – this seems to be present in newer MAME DATs but not the ones the original script has been tested with ?
I removed the section of the main loop which assigns different emulator cores to ROMs that match the criteria (asteroids, donkey kong, etc). I couldn’t figure out how to get this to jive with my new extension variable and I didn’t need that functionality anyway

;AUTOHOTKEY SCRIPT TO GENERATE MAME.lpl FOR RETROARCH
;### SETUP, ADD YOUR PATHS HERE

content = c:\MAME 2014 ROMS
;### MAME ROMs folder with no end slash
;### example: \roms

extension = .7z
;### Enter a . (dot/period) followed by the extension of the ROMs. Typically .zip or .7z

cores = c:\RA\RetroArch_1.2.2\cores
;### cores folder with no end slash
;### C:\RA\RetroArch_1.2.2\cores

dat = c:\RA\RetroArch_1.2.2\MAME 2014 ROMS\MAME 157.dat
;### path to a MAME ROM database file
;### example C:\files\MAME - ROMs (v0.164_XML).dat


FileDelete, %A_ScriptDir%\MAME.lpl  ;clear old file
FileRead, dat, %dat%

Loop, %content%\*%extension%
{
if A_LoopFileName in neogeo%extension%,awbios%extension%,cpzn2%extension%
 continue 
;### you can add names of other files in the folder to skip in the list above

name := SubStr(A_LoopFileName,1,-(StrLen(extension))) ;trim the file extension from the filename

needle2 = <game name=.%name%. (isbios|isdevice)
if RegExMatch(dat, needle2) 
 continue

needle = <game name=.%name%.(?:| ismechanical=.*)(?:| sourcefile=.*)(?:| cloneof=.*)(?:| romof=.*)>\R\s*<description>(.*)</description>
RegExMatch(dat, needle, datname)
if !datname1
 datname1 := name ;fallback to filename

datname1 := RegExReplace(datname1, "[/\?<>\\:*\|]","") ;replace win forbidden chars  
datname1 := RegExReplace(datname1, "'","'")  ;may need more replace like this
datname1 := RegExReplace(datname1, "&","&") 
datname1 := RegExReplace(datname1, " \(.*","")        ;trim stuff like "(World version 2.2)"
datname1 := RegExReplace(datname1, "\.$","") 
;list = %xx%`n%datname1% ;for troubleshooting

i := A_index &lt; 10 ? "000" A_index : "00" A_index

core = mame_libretro.dll

a =
(
%A_LoopFileFullPath%
%datname1%
%cores%\%core%
_
%i%|CRC

)
if (a_index == 1)
 FileAppend, %a%, %A_ScriptDir%\MAME.lpl
else
 FileAppend, `n%a%, %A_ScriptDir%\MAME.lpl
}
;msgbox % list  ;for troubleshooting

Tatsuya79 · 15 January 2017 07:13

I’m sharing this script which scans folders and generates a playlist for each system with every zipped rom contained in those folders. If you’ve got a full no-intro set and you’re bothered that every single rom isn’t in the list for whatever reason (mainly database version or mistakes) this will list everything.

It takes every *.zip and takes * as the name. Your folders and files have to be named adequately for the playlists to work with icons and thumbnails. (rom folder names like this)

You need to make a file named folders.txt (alongside the script in the same folder) with each line being the folder you want to scan, mine looks like this:


E:\Roms\Bandai - WonderSwan
E:\Roms\Bandai - WonderSwan\Bandai - WonderSwan Color
E:\Roms\Microsoft - MSX
E:\Roms\Microsoft - MSX2
E:\Roms\NEC - PC Engine - TurboGrafx 16
E:\Roms\NEC - PC Engine SuperGrafx
E:\Roms\Nintendo - Game Boy
E:\Roms\Nintendo - Game Boy\Nintendo - Game Boy Color
E:\Roms\Nintendo - Game Boy Advance
E:\Roms\Nintendo - Nintendo 64
E:\Roms\Nintendo - Nintendo DS Decrypted
E:\Roms\Nintendo - Nintendo Entertainment System
E:\Roms\Nintendo - Nintendo Entertainment System\Nintendo - Famicom Disk System
E:\Roms\Nintendo - Super Nintendo Entertainment System
E:\Roms\Nintendo - Super Nintendo Entertainment System\Nintendo - Sufami Turbo
E:\Roms\Nintendo - Super Nintendo Entertainment System\Nintendo - Satellaview
E:\Roms\Nintendo - Virtual Boy
E:\Roms\Sega - Game Gear
E:\Roms\Sega - Master System - Mark III
E:\Roms\Sega - Master System - Mark III\Sega - SG-1000
E:\Roms\Sega - Mega Drive - Genesis
E:\Roms\Sega - Mega Drive - Genesis\Sega - 32X
E:\Roms\SNK - Neo Geo Pocket
E:\Roms\SNK - Neo Geo Pocket\SNK - Neo Geo Pocket Color


;AUTOHOTKEY SCRIPT TO GENERATE MAME.lpl FOR RETROARCH
;### SETUP, ADD YOUR PATHS HERE

; Create the array, initially empty:
Array := Object()

; Write to the array:
Loop, Read, folders.txt ; This loop retrieves each line from the file, one at a time.
{
    Array.Insert(A_LoopReadLine) ; Append this line to the array.
}

; Read from the array:
; Loop % Array.MaxIndex()   ; More traditional approach.
for index, element in Array ; Recommended approach in most cases.
{
	content = %element%
	;### ROMs folder

	Stringgetpos,pos,content,\,R   ;gets the position of \ from Right  
	;;
	pos+=1
	Stringtrimleft,system,content,%pos%


	Loop, %content%\*.zip
	{
	name := SubStr(A_LoopFileName,1,-4) ;trim .zip

	a =
	(
%A_LoopFileFullPath%
%name%
DETECT
DETECT
-
%system%.lpl
	)
	if (a_index == 1)
	 FileAppend, %a%, %A_ScriptDir%\%system%.lpl
	else
	 FileAppend, `n%a%, %A_ScriptDir%\%system%.lpl
	}
	;msgbox % list  ;for troubleshooting
}

It scans all those folders in about 5s.

You can set those playlists as read-only if you don’t want them to get changed by Retroarch.

Tatsuya79 · 15 January 2017 07:16

Another example to help some noobs like myself.

I made a script to make a CD image playlist. The games are inside subfolders like:

NEC - Super Cd-rom2
               -game1
                    -game1.cue
               -game2
                    -game2.cue
               -...

Each file will be named after the cue name (minus extension). But you can change the file type to scan from cue to iso, or whatever works. Define the starting folder and the playlist/system name.


;AUTOHOTKEY SCRIPT TO GENERATE lpl for cue files FOR RETROARCH
;### SETUP, ADD YOUR PATHS HERE

dir=E:\Roms\NEC - Super Cd-rom2

playlistname=NEC - PC Engine CD - TurboGrafx-CD.lpl

extlist=cue ;;;;;;file type to scan

Loop, %dir%\*.*, 0, 1 ;;;;;;;;;;;;;;0=Retrieves only files.... 1=Recurse into subfolders

{If A_LoopFileExt in %extlist%
{
;;;Msgbox, The filename is: %A_LoopFileName%`nThe file's path is: %A_loopFileDir%`nThe file's full path with filename: %A_LoopFileLongPath%

content = %A_loopFileDir% ;;;; game image folder

name := SubStr(A_LoopFileName,1,-4) ;;;;trim extension

a =
(
%A_LoopFileFullPath%
%name%
DETECT
DETECT
-
%playlistname%
)

if (a_index == 1)
 FileAppend, %a%, %A_ScriptDir%\%playlistname%
else
 FileAppend, `n%a%, %A_ScriptDir%\%playlistname%
}}

;msgbox % list  ;for troubleshooting

Then open your playlist lpl file and kill the first empty line or that won’t work.

t_m_c · 15 January 2017 07:19

So wait, has anyone else seen the RetroTools playlist generation tool? http://libretro.com/forums/showthread.php?t=5434

Should we all be using it instead?

hunterk · 15 January 2017 07:19

If you have a thing that’s working for you, there’s no reason to jump ship, but there are a few good playlist tools floating around. In addition to that one from Hypnotoad, RandomRAUser and romjacket both have tools that do various things with playlists and are worth checking out.

Tatsuya79 · 15 January 2017 07:19

Got a MAME playlist with snapshots now, that’s kinda cool.

The MAME roms scanning script: (use the dat from the website indicated in the code, not working with official dat)

;AUTOHOTKEY SCRIPT TO RENAME MAME SNAPSHOTS FOR RETROARCH
;### SETUP, ADD YOUR PATHS HERE

content = E:\Roms\Arcade - MAME
;### MAME ROMs folder

dat = E:\Emulateurs\RetroArch64\database\MAME - ROMs (v0.174_XML).dat
;### path to a MAME ROM database file
;### example C:\files\MAME - ROMs (v0.164_XML).dat
;### get dat here http://www.emulab.it/rommanager/datfiles.php

FileDelete, %A_ScriptDir%\MAME.lpl  ;clear old file
FileRead, dat, %dat%

Loop, %content%\*.zip
{
if A_LoopFileName in neogeo.zip,awbios.zip,cpzn2.zip
 continue 
;### you can add names of zip files in the folder to skip in the list above

name := SubStr(A_LoopFileName,1,-4) ;trim .zip

needle2 = <game name=.%name%. (isbios|isdevice|ismechanical)
if RegExMatch(dat, needle2) 
 continue

needle = <game name=.%name%.(?:| ismechanical=.*)(?:| cloneof=.*)(?:| romof=.*)>\R\s*<description>(.*)</description>
RegExMatch(dat, needle, datname)
if !datname1
 datname1 := name ;fallback to filename

datname1 := RegExReplace(datname1, "[/\?<>\\:*\|]","") ;replace win forbidden chars  
datname1 := RegExReplace(datname1, "'","'")  ;may need more replace like this
datname1 := RegExReplace(datname1, "&","&") 
;### datname1 := RegExReplace(datname1, " \(.*","")        ;trim stuff like "(World version 2.2)"
datname1 := RegExReplace(datname1, "\.$","") 
;list = %xx%`n%datname1% ;for troubleshooting


a =
(
%A_LoopFileFullPath%
%datname1%
DETECT
DETECT
-
MAME.lpl
)
if (a_index == 1)
 FileAppend, %a%, %A_ScriptDir%\MAME.lpl
else
 FileAppend, `n%a%, %A_ScriptDir%\MAME.lpl
}
;msgbox % list  ;for troubleshooting

Then get the snapshots from http://www.progettosnaps.net/snapshots/ Copy them in your RetroArch humbnails\MAME\Named_Snaps Change your path again in this one:

;AUTOHOTKEY SCRIPT TO GENERATE MAME.lpl FOR RETROARCH
;### SETUP, ADD YOUR PATHS HERE

content = E:\Emulateurs\RetroArch64	humbnails\MAME\Named_Snaps
;### RETROARCH MAME SNAPS folder

dat = E:\Emulateurs\RetroArch64\database\MAME - ROMs (v0.174_XML).dat
;### path to a MAME ROM database file
;### example C:\files\MAME - ROMs (v0.164_XML).dat
;### get dat here http://www.emulab.it/rommanager/datfiles.php

FileRead, dat, %dat%

Loop, %content%\*.png
{

name := SubStr(A_LoopFileName,1,-4) ;trim .png

needle2 = <game name=.%name%. (isbios|isdevice)
if RegExMatch(dat, needle2) 
 continue

needle = <game name=.%name%.(?:| ismechanical=.*)(?:| cloneof=.*)(?:| romof=.*)>\R\s*<description>(.*)</description>
RegExMatch(dat, needle, datname)
if !datname1
 datname1 := name ;fallback to filename

datname1 := RegExReplace(datname1, "[/\?<>\\:*\|]","") ;replace win forbidden chars  
datname1 := RegExReplace(datname1, "'","'")  ;may need more replace like this
datname1 := RegExReplace(datname1, "&","&") 
;### datname1 := RegExReplace(datname1, " \(.*","")        ;trim stuff like "(World version 2.2)"
datname1 := RegExReplace(datname1, "\.$","") 
;list = %xx%`n%datname1% ;for troubleshooting

FileMove, %A_LoopFileFullPath%, %content%\%datname1%.png

}
;msgbox % list  ;for troubleshooting

If you use both scripts they should rename the png in the same way they did with the playlist entries.

I removed “mechanical” roms (cash machines mostly) from the playlist. You can also scan it for some left over like every (Bellfruit (Scorpion and delete those (there are a lot but in succession in the list).

You can also kill all those PNG that are around 55Kb and display “This Machine is not working”. There is about… 20 000 of them. It makes it faster to rename with the script if done before scanning. Or rename and kill the (Bellfruit (Scorpion and so on after that.

About 14 000 entries after all that (makes the playlist faster to load than the 34 000 before). It looks like this http://postimg.org/image/hxcgrwem3/full/ (I changed some scaling values in retroarch source).

hunterk · 15 January 2017 07:19

that’s pretty swanky. Good shit, Tatsuya79

t_m_c · 15 January 2017 07:20

I’m branching my modifications over to the Lakka forum, since that is where my script will be used: http://libretro.com/forums/showthread.php?t=5923

wertz · 15 January 2017 08:22

[QUOTE=Tatsuya79;39833]Got a MAME playlist with snapshots now, that’s kinda cool.

The MAME roms scanning script: (use the dat from the website indicated in the code, not working with official dat)

;AUTOHOTKEY SCRIPT TO RENAME MAME SNAPSHOTS FOR RETROARCH
;### SETUP, ADD YOUR PATHS HERE

content = E:\Roms\Arcade - MAME
;### MAME ROMs folder

dat = E:\Emulateurs\RetroArch64\database\MAME - ROMs (v0.174_XML).dat
;### path to a MAME ROM database file
;### example C:\files\MAME - ROMs (v0.164_XML).dat
;### get dat here http://www.emulab.it/rommanager/datfiles.php

FileDelete, %A_ScriptDir%\MAME.lpl  ;clear old file
FileRead, dat, %dat%

Loop, %content%\*.zip
{
if A_LoopFileName in neogeo.zip,awbios.zip,cpzn2.zip
 continue 
;### you can add names of zip files in the folder to skip in the list above

name := SubStr(A_LoopFileName,1,-4) ;trim .zip

needle2 = <game name=.%name%. (isbios|isdevice|ismechanical)
if RegExMatch(dat, needle2) 
 continue

needle = <game name=.%name%.(?:| ismechanical=.*)(?:| cloneof=.*)(?:| romof=.*)>\R\s*<description>(.*)</description>
RegExMatch(dat, needle, datname)
if !datname1
 datname1 := name ;fallback to filename

datname1 := RegExReplace(datname1, "[/\?<>\\:*\|]","") ;replace win forbidden chars  
datname1 := RegExReplace(datname1, "'","'")  ;may need more replace like this
datname1 := RegExReplace(datname1, "&","&") 
;### datname1 := RegExReplace(datname1, " \(.*","")        ;trim stuff like "(World version 2.2)"
datname1 := RegExReplace(datname1, "\.$","") 
;list = %xx%`n%datname1% ;for troubleshooting


a =
(
%A_LoopFileFullPath%
%datname1%
DETECT
DETECT
-
MAME.lpl
)
if (a_index == 1)
 FileAppend, %a%, %A_ScriptDir%\MAME.lpl
else
 FileAppend, `n%a%, %A_ScriptDir%\MAME.lpl
}
;msgbox % list  ;for troubleshooting

Then get the snapshots from http://www.progettosnaps.net/snapshots/ Copy them in your RetroArch humbnails\MAME\Named_Snaps Change your path again in this one:

;AUTOHOTKEY SCRIPT TO GENERATE MAME.lpl FOR RETROARCH
;### SETUP, ADD YOUR PATHS HERE

content = E:\Emulateurs\RetroArch64	humbnails\MAME\Named_Snaps
;### RETROARCH MAME SNAPS folder

dat = E:\Emulateurs\RetroArch64\database\MAME - ROMs (v0.174_XML).dat
;### path to a MAME ROM database file
;### example C:\files\MAME - ROMs (v0.164_XML).dat
;### get dat here http://www.emulab.it/rommanager/datfiles.php

FileRead, dat, %dat%

Loop, %content%\*.png
{

name := SubStr(A_LoopFileName,1,-4) ;trim .png

needle2 = <game name=.%name%. (isbios|isdevice)
if RegExMatch(dat, needle2) 
 continue

needle = <game name=.%name%.(?:| ismechanical=.*)(?:| cloneof=.*)(?:| romof=.*)>\R\s*<description>(.*)</description>
RegExMatch(dat, needle, datname)
if !datname1
 datname1 := name ;fallback to filename

datname1 := RegExReplace(datname1, "[/\?<>\\:*\|]","") ;replace win forbidden chars  
datname1 := RegExReplace(datname1, "'","'")  ;may need more replace like this
datname1 := RegExReplace(datname1, "&","&") 
;### datname1 := RegExReplace(datname1, " \(.*","")        ;trim stuff like "(World version 2.2)"
datname1 := RegExReplace(datname1, "\.$","") 
;list = %xx%`n%datname1% ;for troubleshooting

FileMove, %A_LoopFileFullPath%, %content%\%datname1%.png

}
;msgbox % list  ;for troubleshooting

If you use both scripts they should rename the png in the same way they did with the playlist entries.

I removed “mechanical” roms (cash machines mostly) from the playlist. You can also scan it for some left over like every (Bellfruit (Scorpion and delete those (there are a lot but in succession in the list).

You can also kill all those PNG that are around 55Kb and display “This Machine is not working”. There is about… 20 000 of them. It makes it faster to rename with the script if done before scanning. Or rename and kill the (Bellfruit (Scorpion and so on after that.

About 14 000 entries after all that (makes the playlist faster to load than the 34 000 before). It looks like this http://postimg.org/image/hxcgrwem3/full/ (I changed some scaling values in retroarch source).[/QUOTE]

thanks for this. got my updated fba playlist now.

markwkidd · 15 January 2017 10:02

These original scripts have spawned yet more offspring! With thanks and acknowledgment to those who worked on this before, I’ve added a GUI and a few other features intended to make this process more accessible to users & to streamline the process of matching arcade ROMs with MAME and FB Alpha thumbnails from the libretro-thumbnails collection.

If you’re interested in the AHK sourcecode or a compiled windows executable: https://libretro.com/forums/showthread.php?t=7802