An input lag investigation

Im a little confused. So how much input lag do we have now with the brunnis patch for snes games? How low can it go?

Apparently there was some additional input lag specific to the (s)nes. There is one more frame of lag specific to opengl (you can work around it by using the dispmanx video driver but then you have no pixel shaders – btw, I’d love to know what good overlays you can use with it :slight_smile: I tried a few but the result was not satisfying – and there seems to be one more frame of lag which is not understood for the time being.

Also I read this, is it relevant ? “Also, Lakka has now been modified upstream to apply the rt-kernel patch for more consistent scheduling with 1Khz tick rate, preemptible etc. (frame delay setting can be increased further)”

Finally, there is this frame_delay_setting that can help you get back up to one other frame of lag but you need a very fast CPU and this is super annoying to tweak. (different optimal value per system or even per game)

/me enjoys reading this thread even if he is just a dumb user

Edit : brunnis made a summary here : http://libretro.com/forums/showthread.php?t=5428&page=18 Without frame delay, there is up to 5 frames of delay while he only measures about 2 on a real SNES. I clearly see a huge difference between RetroPie on a Pi and the NES/PCE/… FPGA core on my MiST for instance. But FPGA computers are 1) very expensive 2) support few systems.

[QUOTE=bidinou;42111]Apparently there was some additional input lag specific to the (s)nes. There is one more frame of lag specific to opengl (you can work around it by using the dispmanx video driver but then you have no pixel shaders – btw, I’d love to know what good overlays you can use with it :slight_smile: I tried a few but the result was not satisfying – and there seems to be one more frame of lag which is not understood for the time being.

Also I read this, is it relevant ? “Also, Lakka has now been modified upstream to apply the rt-kernel patch for more consistent scheduling with 1Khz tick rate, preemptible etc. (frame delay setting can be increased further)”

Finally, there is this frame_delay_setting that can help you get back up to one other frame of lag but you need a very fast CPU and this is super annoying to tweak. (different optimal value per system or even per game)

/me enjoys reading this thread even if he is just a dumb user

Edit : brunnis made a summary here : http://libretro.com/forums/showthread.php?t=5428&page=18 Without frame delay, there is up to 5 frames of delay while he only measures about 2 on a real SNES. I clearly see a huge difference between RetroPie on a Pi and the NES/PCE/… FPGA core on my MiST for instance. But FPGA computers are 1) very expensive 2) support few systems.[/QUOTE]

I am in the same boat. Just a dumb user trying to get my head around it all. There is also what seems like half a frame of lag removed from this option: https://github.com/libretro/RetroArch/issues/3100

I am really hoping Lakka with implement all of these options and turn out to be the distro with the lowest lag. (DRM-KMS, Brunnis patched cores, RT kernel 1khz with +15 Framedelay, Triple buffer option=Maximum swapchain images, dispmanx/plaindrm driver) Thats my collection thus far. :slight_smile:

tekn0 : do you mean a high framedelay can be specified regardless of the emulated platform / CPU power as long as a RT kernel is used ?

We really need a wiki page dedicated to this investigation, to help both users & developers keep track of what’s been and what could potentially be achieved.

[QUOTE=Brunnis;41854]Just made a pull request for implementing the exact same fix for bsnes as bsnes-mercury: https://github.com/libretro/bsnes-libretro/pull/16

Hopefully they’ll both be merged very soon. :slight_smile:

[/QUOTE]

Good news:

http://github.com/libretro/bsnes-libretro/pull/16 Thanks again for all the effort you put into this, very much appreciated. Reading along with all the findings in this forum was a joy too :).

Last but not least. Hopefully you have time and energy left to look at other cores also in the future. No pressure of course!

yeah, it’s in both snes9x and *-next and bsnes and *-mercury. I think CatSFC is the only SNES core we have that could still be affected.

That’s a really good idea… I’ll see what I can do.

[QUOTE=rafan;42154]Good news:

http://github.com/libretro/bsnes-libretro/pull/16 Thanks again for all the effort you put into this, very much appreciated. Reading along with all the findings in this forum was a joy too :).

Last but not least. Hopefully you have time and energy left to look at other cores also in the future. No pressure of course![/QUOTE] Yep, this is great news. The fix was just committed to both the regular bsnes and the bsnes-mercury cores. I’m taking a bit of a break from the emulator analysis stuff for the moment, but I’ll probably spend more time on it in the future. To be honest, though, I’m quite pleased with having removed 1 frame of input lag from fceumm, snes9x, snes9x-next, bsnes and bsnes-mercury. :slight_smile:

However, there’s probably still work to be done on the video pipeline of the Raspberry Pi. I haven’t gotten very far on that testing, though. I have made a few tries getting RetroArch to work on the new OpenGL driver, but no luck so far.

[QUOTE=Brunnis;41992]Yep, I don’t know exactly what they’ve changed in the RetroPie setup compared to default Raspbian, so probably better to just test with your image. Easiest would be if you could just put it on a service like Google Drive, Dropbox, mega.nz, etc. and give me a link to it. Or if you have an FTP you’d be willing to put it on.

When I looked at the commit, it seemed like they changed it to --enable-kms. Can’t find any reference to “drm” in config.params.sh.[/QUOTE]

I’ve been testing this using a standard Raspian Jessie image and using the RetroPie setup script to install RetroArch and its cores and then custom compiling Retroarch to use the KMS or “DRM” driver and this is looking very promising. I’ve tested several games using the “Brunnis patched” emulators and I’m getting some very exciting results. Here are a few of the games I’ve tested:

[TABLE=“width: 959”]

Game Frames Emulator Video Driver Audio Driver

Akumajou Densetsu 3 RetroArch\lr-FCEUMM DRM SDL2

Super Mario Bros. 3 2 RetroArch\lr-FCEUMM DRM SDL2

Mega Man II 2 RetroArch\lr-FCEUMM DRM SDL2

Castlevania III 3 RetroArch\lr-FCEUMM DRM SDL2

Castlevania 3 RetroArch\lr-FCEUMM DRM SDL2

Super Metroid 2 RetroArch\lr-Snes9x-Next DRM SDL2

Yoshi’s Island 3 RetroArch\lr-Snes9x-Next DRM SDL2

Super Mario All-Stars 2 RetroArch\lr-Snes9x-Next DRM SDL2

Seiken Densetsu 3 4 RetroArch\lr-Snes9x-Next DRM SDL2

Secret of Mana 2 RetroArch\lr-Snes9x-Next DRM SDL2

Chrono Trigger 2 RetroArch\lr-Snes9x-Next DRM SDL2

Super Mario World 3 RetroArch\lr-Snes9x-Next DRM SDL2

[/TABLE]

I’ve found a few issues when using the KMS/DRM video driver:

-When some SNES games enter and leave Hi-res mode it causes the screen to “glitch” for a split-second during the transition. -It doesn’t support overlays or shaders currently. -Can’t disable the bi-linear filtering.

I tested using the pause/frame-step method on a Raspberry Pi 3 with the scaling_governor set to performance and the core_freq and sdram_freq both overclocked to 500 to keep the sound from crackling.

Great, I’ll see if I can get it running in the same way.

Just so you know, using the pause-step method you will actually mask any differences between the video drivers. Your test results are great when it comes to evaluating the emulator lag, though. They match up well with my tests as well.

Anyway, unfortunately, you will need to record the action and count frames to really find out if the video driver makes any difference. I’ll see if I can do a test like this within the next week or so.

[QUOTE=Brunnis;42172]Great, I’ll see if I can get it running in the same way.

Just so you know, using the pause-step method you will actually mask any differences between the video drivers. Your test results are great when it comes to evaluating the emulator lag, though. They match up well with my tests as well.

Anyway, unfortunately, you will need to record the action and count frames to really find out if the video driver makes any difference. I’ll see if I can do a test like this within the next week or so.[/QUOTE] Ah, that clears up the confusion.

Also I heard the there’s no frame delay on ios? That it has 3 frame delay. I wonder if thats true.

Feckin A guys. I was checking this thread daily, until I stopped for a while due to a newborn son. Not only does the new commit completely solve the problem, I can finally start planning to enjoy the greatest system of all time with him. Kudos for a simple and easy fix. Mario World plays like butter now.

They grow so fast!

[QUOTE=Brunnis;42169] Yep, this is great news. The fix was just committed to both the regular bsnes and the bsnes-mercury cores. I’m taking a bit of a break from the emulator analysis stuff for the moment, but I’ll probably spend more time on it in the future. To be honest, though, I’m quite pleased with having removed 1 frame of input lag from fceumm, snes9x, snes9x-next, bsnes and bsnes-mercury. :-)[/QUOTE]

You rightfully should be :slight_smile:

With regards to finding more possible targets, are cores that have a frame count of 3 (or worse) with the frame advance method possible suspects?

Would it be useful for people to report them here if they find a core that does not go below a value of 3 when tested with say a suite of games?

Just tested Megaman X2 / X3, US and Japanese, on bsnes / bsnes-mercury, balanced / accuracy, pre/post Brunnis patch:

nothing wrong here.

I passed the capcom logo, title screen, played half way in the level.

Win7 x64 nvidia opengl.

Tested with the buildbot version through core updater, no issue with bsnes/bsnes-mercury on Rockman X2.

The MSU-1 chip is working.

I notice a big slow down with bsnes/bsnes-mercury balanced right on the title screen if GPU hard sync is ON @0. GPU hard sync OFF = no slow down. Brunnis patch makes no difference.

Then playing the game is OK with GPU hard sync @0. That’s strange there is that processing peak there on the title screen, perhaps a pre-loading?

I’m going to assume this is all for games that are actually patched for MSU1 audio and not just MSU1 capable games.

There have been a few committs today fixing a potential issue with the clearing of the frame event flag. For those of you who have seen issues with MSU-1 patched games, please compile bsnes (non-mercury) from source an retry.

I still haven’t been able to reproduce the issue, so no idea if the changes help.

Would be interesting to confirm whether the same occurs in bsnes-mercury.

Yeah it’s probably a serialization + MSU-1 issue that gets actually worse with the input lag hack. I don’t really understand why it happens when music playback starts, maybe the state size changes when music starts? that could explain why it works fine with super road blaster since that one is MSU-1 all the way.

Yep, the state size was one of the first things that came to my mind as well. I’ll try to look some more into the issue as soon as I get some spare time for it. Probably not this week, though.

EDIT: Just a thought… What happens if you disable the lag fix, but leave the frame_event_performed declaration in the CPU as well as the serialization of said flag? That way you’ll just include the extra variable in the serialization, but it would be completely unused. Does Mega Man X with MSU-1 support work or not?