Yeah I agree this would be interesting. We’d have to figure out how to get the movies/image sequences into the retroarch shader system.
Right now the only way I think this is sort of possible is with a sprite sheet, but you can imagine that it would be a pretty low resolution image for each block in the sheet.
If we had GIF support and a way to access the frames in the GIF from the shader system this might be the best way (Because a GIF is a much simpler file format than an MP4). But we’d need to add both of these to Retroarch.
One thing to note is that it would need to load all the frames uncompressed into memory, no streaming/on demand load. This could take some memory, but since pc graphics cards have a lot of memory this might not be a big deal, it would depend on the resolution and number of frames.