Proposal for playlist file format v2

markwkidd · 23 February 2018 22:36

There has been discussion in the past about what an enhanced playlist format should look like, but I think anyone has put forward a specific recommendation yet. I’m writing this in the hopes of sparking productive discussion about what the new format should look like and ways that a more flexible playlist format could improve RetroArch.

Prologue: An Important Assumption

Per comments from Twinaphex, a new playlist format would need to co-exist with the existing format. This proposal is based on the assumption that the playlist parser would be modified to switch based on whether a v1 playlist or a v2 playlist was being loaded.

Because this proposal uses a minimal xml format, logic for playlist detection could be as simple as:

File fp = open_file(playlistname);
char firstchar = get_char(fp);
if(firstchar == "<") // XML format detected
    parse_xml_playlist(fp);
else // v1 playlist
   parse_v1_playlist(fp);

Alternately, if the new playlists were given a different file extension than lpl, that could be used to determine which parser to use.

Proposal

The next version of the RetroArch file format should use a simple XML format parsed by a minimal C XML library like xml.c or sxmlc.

This minimum viable implementation should, at least, accomplish the following improvements from the current (v1) playlist format:

Allow the specification of arbitrary thumbnail paths on a per-playlist-entry basis, addressing deduplication issues with the thumbnail repository and making it possible to display thumbnails from the “History” playlist.
Allow new tags to be added in the future without breaking the playlist file format

Minimal playlist v2 example:

<?xml version="1.0"?>
<playlist>
    <playlist_entry name="Legend of Zelda, The - Ocarina of Time (USA)">
        <content_path>storage/roms/n64/Legend of Zelda, The - Ocarina of Time (USA).n64</content_path>
     </playlist_entry>
</playlist>

Equivalent to this v1 playlist:

/storage/roms/n64/Legend of Zelda, The - Ocarina of Time (USA).n64
Legend of Zelda, The - Ocarina of Time (USA)
DETECT
DETECT
DETECT
DETECT

Mandatory tags

Each playlist file should have at minimum one playlist with at least one playlist_entry within it. Each playlist_entry should have at minimum:

name as an attribute within the parent playlist_entry tag
content_path as a child tag

global attributes

This could be left optional. attributes specified within the root playlist would be per-playlist defaults. When a playlist_entry specifies the same parameter as a global parameter in playlist, the playlist_entry overrides the global setting for that item only.

More complex example

This example imagines some tags that could be implemented in the future.

<?xml version="1.0"?>
<playlist>
    <playlist_icon>/tmp/assets/xmb/funky-n64/console_delight.png</playlist_icon>
    <content_icon>/tmp/assets/xmb/funky-n64/cartridge_delight.png</content_icon>
    <core_path>/tmp/cores/mupen64plus_libretro.so</core_path>
    <shader_path>/tmp/shaders/some_great_n64_shader.glsl</shader_path>
    <playlist_entry name="Legend of Zelda, The - Ocarina of Time (USA)">
        <content_path>storage/roms/n64/Legend of Zelda, The - Ocarina of Time (USA).n64</content_path>
        <thumbnail_path>/tmp/assets/thumbnails/n64/Legend of Zelda, The - Ocarina of Time (USA)</thumbnail_path>
    </playlist_entry>
    <playlist_entry name="Legend of Zelda, The - Ocarina of Time (EUR)">
        <content_path>storage/roms/n64/Legend of Zelda, The - Ocarina of Time (EUR).n64</content_path>
        <shader_path>/tmp/shaders/some_great_n64_shader_PAL_version.glsl</shader_path>
        <content_icon>/tmp/assets/xmb/funky-n64/euro_cartridge_delight.png</content_icon>
        <thumbnail_path>/tmp/assets/thumbnails/n64/Legend of Zelda, The - Ocarina of Time (USA)</thumbnail_path>
    </playlist_entry>
</playlist>

Possible additional tags

shader_path
overlay_path
region
user_rating (rated by the user, within RetroArch)

From existing RetroArch `database_info.h` tags

max_users
enhancement_hw
releasemonth
releaseyear
description
genre
publisher
developer (as a child <developer>with individual <developers> as nested strings, mirroring the RA DB)
origin
franchise
famitsu_magazine_rating
edge_magazine_issue
edge_magazine_rating
edge_magazine_review
tgdb_rating
bbfc_rating
elspa_rating
esrb_rating
pegi_rating
cero_rating

Possible new tags inspired by the EmulationStation gamelist.xml format

playcount - statistic, integer, the number of times this game has been played
lastplayed - statistic, datetime, the last date and time this game was played.
kidgame - does not have any mature/adult content

alfrix · 10 February 2018 23:44

Interesting, maybe a core_path in some kind of main header and optional on the playlist entries, to execute with a different one in case of problem

<main>
    <core_path></core_path>
</main>
<playlist_entry>
    <content_path></content_path>
    <display_name></display_name>
    <core_path></core_path> <!-- Overrides the main one -->
</playlist_entry>
<playlist_entry>
    <content_path></content_path>
    <display_name></display_name>
    <crc></crc>
</playlist_entry>

markwkidd · 11 February 2018 02:11

That is such an obvious improvement, thanks! At the risk of being verbose, I am changing your “main” to “global” and adding that concept to the proposal.

markwkidd · 12 February 2018 21:30

I’ve made a few more updates to the text for clarity, and added an example of how some new tags might be used in tandem with the global/override concept. I’m planning to leave it along for a while from here, however.

alfrix · 13 February 2018 16:35

core_name is currently used for the history and favorites playlists

markwkidd · 21 February 2018 20:21

I have found that there is an xml library within libretro-common, but the example code doesn’t build for me via mysys2/mingw64. I’ll plan to loop back around to it soon!

hunterk · 21 February 2018 20:37

I had started an issue for this a while back, as bparker had offered to implement the spec once it was nailed down (i.e., not trying to come up with it as we go), and I seem to have @ed everyone who might be interested except for you, markwkidd:

I came up with pretty much the same stuff as you, though I thought JSON, since we have a parser for that, as well, though it would mean a lot of double-quotes.

markwkidd · 21 February 2018 21:04

Great minds. Or was it “great mimes”?

It’s encouraging to hear there are others ready to take some steps. Especially since my C coding leaves a lot to be desired – I was dreading the proof of concept code that feels like it is needed soon.

Anyway, I definitely don’t mind moving this discussion to github or talking about JSON.

hunterk · 21 February 2018 21:58

nothing was happening there on github, so we may as well keep it here for the time being and we can move it over once the discussion is done. I don’t have any preference as to XML or JSON, personally. We can ask bparker whether he does.

zachmorris · 22 February 2018 01:44

I would suggest xml, but thats just me.

But apparently JSON may be faster for larger files?

For other possible tags, would there be any benefit to looking at what other projects use?

github.com

Aloshi/EmulationStation/blob/master/GAMELISTS.md

Gamelists
=========

The gamelist.xml file for a system defines metadata for a system's games, such as a name, image (like a screenshot or box art), description, release date, and rating.

ES will check three places for a gamelist.xml in the following order, using the first one it finds:
* `[SYSTEM_PATH]/gamelist.xml`
* `~/.emulationstation/gamelists/[SYSTEM_NAME]/gamelist.xml`
* `/etc/emulationstation/gamelists/[SYSTEM_NAME]/gamelist.xml`

An example gamelist.xml:
```xml
<gameList>
	<game>
		<path>/home/pi/ROMs/nes/mm2.nes</path>
		<name>Mega Man 2</name>
		<desc>Mega Man 2 is a classic NES game which follows Mega Man as he murders eight robot masters in cold blood.</desc>
		<image>~/.emulationstation/downloaded_images/nes/Mega Man 2-image.png</image>
	</game>
</gameList>

This file has been truncated. show original

Retroarch may not currently use some of those tags, but why not bake it in from the get go, so if they’re used at some point they’d be available.

markwkidd · 23 February 2018 04:17

As far as the EmulationStation gamelist format, I’d suggest we use the mainline RetroPie fork of EmulationStation as the reference since it’s active and widely used: https://github.com/RetroPie/EmulationStation/blob/master/GAMELISTS.md

I can’t guarantee this but I do think there have been changes to the gamelist.xml format since the fork from the Aloshi version.

markwkidd · 23 February 2018 04:38

Update: I have updated the OP with many new tags, most of them sourced from the RetroArch database_info.h file. I also perused the GameList.xml specification and spotted a few interesting ideas which are also part of the OP now.

Sarfarosh · 31 July 2021 01:02

That is such an obvious improvement, thanks! At the risk of being verbose, I am changing your “main” to “global” and adding that concept to the proposal.Especially since my C coding leaves a lot to be desired – I was dreading the proof of concept code that feels like it is needed soon.

Anyway, I definitely don’t mind moving this discussion to github or talking about JSON.

Sarfarosh · 24 January 2019 13:50

Hello everyone, It’s encouraging to hear there are others ready to take some steps. Especially since my C coding leaves a lot to be desired – I was dreading the proof of concept code that feels like it is needed soon.

Anyway, I definitely don’t mind moving this discussion to github or talking about JSON.

markwkidd · 24 January 2019 14:00

Do you use Discord?

The most active conversation about the playlist format is taking place on the RetroArch discord server in the #programming channel

Discord invite link: https://discord.gg/C4Huaca

Proposal for playlist file format v2

Proposal

Minimal playlist v2 example:

Equivalent to this v1 playlist:

Mandatory tags

global attributes

More complex example

Full list of basic playlist_entry and global tags

Possible additional tags

From existing RetroArch database_info.h tags

Possible new tags inspired by the EmulationStation gamelist.xml format

Full list of basic `playlist_entry` and global tags

From existing RetroArch `database_info.h` tags