I am currently thinking how to use phones and tablets as an extended display for retroarch/libretro. Looking on consoles like Dreamcast, GameCube, NintendoDS Nintendo3DS and WiiU, they have all some kind of second screen option.
Dreamcast have their VMU GameCube had their JoyBus communication between GBA and GC. NintendoDS and 3DS have 2 screens (one touchscreen) implemented. Even the PSP and GameBoy used multi display options for multiplayer.
My idea is to use an phone or tablet (android as well as ios) as second screen to enable the features from the former consoles. It would be somehow cool to play for example Crystal Chronicles or Four Swords on the big TV screen with the smaller smartphones/tablets. Or to display the VMU status screen from the dreamcast on the phone. Or to play certain DS games with their original and real touch controls on the tablet.
I originally thought to use “Moonlight Game Streaming” (https://github.com/moonlight-stream) to implement the idea. However Moonlight only supports the features from nvidia and the display cannot be extended, which is required to display the second screen. For DS/3DS Games it is already good. But for Dreamcast/GameCube/WiiU like experiences is still lacks the second screen option. Moonlight is thinking to add a server application to add those features (https://github.com/moonlight-stream/moonlight-common-c/issues/29), but currently nobody is working on the realisation.
My question would be. Has anyone thought of a similar idea? What do you think? Would this also be a possible feature for libretro?