New here, please let me know if I missed this in another topic or in the docs. I’ve been wondering whether I can send the video output of an emulated game into a python script to analyze what I’m seeing in realtime and build on top of it. (Example: track the ball in pong, identify bananas in Mario Kart, read out the scorelines in FIFA, …)
Really, the goal for me would be to take some of my favorite games and try to monitor scorelines as well as emit events when something important happens. What I’ll do with that, who knows… Feed it into a mqtt broker, adapt my room lighting to the game events, whatever. Lulz, you know
If this can be achieved any other way (like grabbing such events from the game running on a core itself), I’m happy to read into it as well.
But I assume that this wouldn’t be standardized or easily accessed from a compiled game, so reverse engineering events from the video feed would seem the most reliable to me.
I know I’d have to prepare this game by game, but it seems doable once I have the frames stream into a script where I can run it through the likes of OpenCV.
I’ve seen there’s a similar feature baked in for subtitle translation, where video output is sent through OCR, but if I understand correctly, that is happening only on a single frame and only on demand. If at all possible, I’d love if this can be adapted to constantly stream into a server and let me go wild on it.
Let me know if this sounds possible or completely stupid. Just trying to lay the foundations to a hobby project here… Also, thanks for any help already!