How does the content scanner work?

General Development
#1

Looking at the database file, it appears we can identify content by filename, size, CRC, MD5 or SHA1. If I scan a folder of Blu-ray images does it calculate the CRC, MD5 and SHA1 of every file in the folder to see if it matches anything in the database?

Mike

#2

I think for optical disk images, it will check known serial locations and if it finds nothing will just skip.

#3

This is what I’m getting at. If were scanning an .img file, it could be a track on an optical disc or an EPROM dump, maybe a tape image or even a floppy image… Say we need to scan a file “track1.img”, which is a 40Mb - how do we decide what is it or how to scan it?

#4

looks like it’s based on filetype, and ambiguous formats like bin/img aren’t supported: https://github.com/libretro/RetroArch/blob/master/tasks/task_database.c#L517

#5

Hi folks, I have been trying to build a custom library organizer (being on a mac sometimes means I can’t access a lot of the ones out there).

For learning purposes I started looking at the libretro dat files, and looking at some of the files that compose the roms.

Most of the home systems such as SNES/Genesis it was easy to figure out as they are based on CRC/SHA1 Digest of the file.

Disc systems are still a bit challenging, and I wanted to share my experience here and perhaps learn more from others:

For Sega CD and to my surprise Saturn I used this reference: https://plutiedev.com/rom-header

Dreamcast was also easy to figure a GDI file and also read some bytes from the first track.

Gamecube seems that all serials are stored on the very first bytes of the iso file

Playstation was hard :frowning: Only because I had no easy way to extract the cue/ccd bin file contents of the System.CNF, I ended up porting (bchunk : https://github.com/hessu/bchunk) to java (which is what I’m using for my tool), and then I was able to extract PS serial data.

I have not yet figured out turbografx 16, or neogeo cd. I’m hoping PS2/PS3/PSP follow similar pattern of System.CNF files, but have not reached those systems yet.

So could anyone share their knowledge of how those files get parsed by retroarch?

Cheers