An input lag investigation

RetroArch General
#222

That’s a really good idea… I’ll see what I can do.

[QUOTE=rafan;42154]Good news:

http://github.com/libretro/bsnes-libretro/pull/16 Thanks again for all the effort you put into this, very much appreciated. Reading along with all the findings in this forum was a joy too :).

Last but not least. Hopefully you have time and energy left to look at other cores also in the future. No pressure of course![/QUOTE] Yep, this is great news. The fix was just committed to both the regular bsnes and the bsnes-mercury cores. I’m taking a bit of a break from the emulator analysis stuff for the moment, but I’ll probably spend more time on it in the future. To be honest, though, I’m quite pleased with having removed 1 frame of input lag from fceumm, snes9x, snes9x-next, bsnes and bsnes-mercury. :slight_smile:

However, there’s probably still work to be done on the video pipeline of the Raspberry Pi. I haven’t gotten very far on that testing, though. I have made a few tries getting RetroArch to work on the new OpenGL driver, but no luck so far.

#223

[QUOTE=Brunnis;41992]Yep, I don’t know exactly what they’ve changed in the RetroPie setup compared to default Raspbian, so probably better to just test with your image. Easiest would be if you could just put it on a service like Google Drive, Dropbox, mega.nz, etc. and give me a link to it. Or if you have an FTP you’d be willing to put it on.

When I looked at the commit, it seemed like they changed it to --enable-kms. Can’t find any reference to “drm” in config.params.sh.[/QUOTE]

I’ve been testing this using a standard Raspian Jessie image and using the RetroPie setup script to install RetroArch and its cores and then custom compiling Retroarch to use the KMS or “DRM” driver and this is looking very promising. I’ve tested several games using the “Brunnis patched” emulators and I’m getting some very exciting results. Here are a few of the games I’ve tested:

[TABLE=“width: 959”]

Game Frames Emulator Video Driver Audio Driver

Akumajou Densetsu 3 RetroArch\lr-FCEUMM DRM SDL2

Super Mario Bros. 3 2 RetroArch\lr-FCEUMM DRM SDL2

Mega Man II 2 RetroArch\lr-FCEUMM DRM SDL2

Castlevania III 3 RetroArch\lr-FCEUMM DRM SDL2

Castlevania 3 RetroArch\lr-FCEUMM DRM SDL2

Super Metroid 2 RetroArch\lr-Snes9x-Next DRM SDL2

Yoshi’s Island 3 RetroArch\lr-Snes9x-Next DRM SDL2

Super Mario All-Stars 2 RetroArch\lr-Snes9x-Next DRM SDL2

Seiken Densetsu 3 4 RetroArch\lr-Snes9x-Next DRM SDL2

Secret of Mana 2 RetroArch\lr-Snes9x-Next DRM SDL2

Chrono Trigger 2 RetroArch\lr-Snes9x-Next DRM SDL2

Super Mario World 3 RetroArch\lr-Snes9x-Next DRM SDL2

[/TABLE]

I’ve found a few issues when using the KMS/DRM video driver:

-When some SNES games enter and leave Hi-res mode it causes the screen to “glitch” for a split-second during the transition. -It doesn’t support overlays or shaders currently. -Can’t disable the bi-linear filtering.

I tested using the pause/frame-step method on a Raspberry Pi 3 with the scaling_governor set to performance and the core_freq and sdram_freq both overclocked to 500 to keep the sound from crackling.

#224

Great, I’ll see if I can get it running in the same way.

Just so you know, using the pause-step method you will actually mask any differences between the video drivers. Your test results are great when it comes to evaluating the emulator lag, though. They match up well with my tests as well.

Anyway, unfortunately, you will need to record the action and count frames to really find out if the video driver makes any difference. I’ll see if I can do a test like this within the next week or so.

#225

[QUOTE=Brunnis;42172]Great, I’ll see if I can get it running in the same way.

Just so you know, using the pause-step method you will actually mask any differences between the video drivers. Your test results are great when it comes to evaluating the emulator lag, though. They match up well with my tests as well.

Anyway, unfortunately, you will need to record the action and count frames to really find out if the video driver makes any difference. I’ll see if I can do a test like this within the next week or so.[/QUOTE] Ah, that clears up the confusion.

Also I heard the there’s no frame delay on ios? That it has 3 frame delay. I wonder if thats true.

#226

Feckin A guys. I was checking this thread daily, until I stopped for a while due to a newborn son. Not only does the new commit completely solve the problem, I can finally start planning to enjoy the greatest system of all time with him. Kudos for a simple and easy fix. Mario World plays like butter now.

#227

They grow so fast!

#228

[QUOTE=Brunnis;42169] Yep, this is great news. The fix was just committed to both the regular bsnes and the bsnes-mercury cores. I’m taking a bit of a break from the emulator analysis stuff for the moment, but I’ll probably spend more time on it in the future. To be honest, though, I’m quite pleased with having removed 1 frame of input lag from fceumm, snes9x, snes9x-next, bsnes and bsnes-mercury. :-)[/QUOTE]

You rightfully should be :slight_smile:

With regards to finding more possible targets, are cores that have a frame count of 3 (or worse) with the frame advance method possible suspects?

Would it be useful for people to report them here if they find a core that does not go below a value of 3 when tested with say a suite of games?

#231

Just tested Megaman X2 / X3, US and Japanese, on bsnes / bsnes-mercury, balanced / accuracy, pre/post Brunnis patch:

nothing wrong here.

I passed the capcom logo, title screen, played half way in the level.

Win7 x64 nvidia opengl.

#234

Tested with the buildbot version through core updater, no issue with bsnes/bsnes-mercury on Rockman X2.

The MSU-1 chip is working.

I notice a big slow down with bsnes/bsnes-mercury balanced right on the title screen if GPU hard sync is ON @0. GPU hard sync OFF = no slow down. Brunnis patch makes no difference.

Then playing the game is OK with GPU hard sync @0. That’s strange there is that processing peak there on the title screen, perhaps a pre-loading?

#237

I’m going to assume this is all for games that are actually patched for MSU1 audio and not just MSU1 capable games.

#242

There have been a few committs today fixing a potential issue with the clearing of the frame event flag. For those of you who have seen issues with MSU-1 patched games, please compile bsnes (non-mercury) from source an retry.

I still haven’t been able to reproduce the issue, so no idea if the changes help.

#244

Would be interesting to confirm whether the same occurs in bsnes-mercury.

#246

Yeah it’s probably a serialization + MSU-1 issue that gets actually worse with the input lag hack. I don’t really understand why it happens when music playback starts, maybe the state size changes when music starts? that could explain why it works fine with super road blaster since that one is MSU-1 all the way.

#247

Yep, the state size was one of the first things that came to my mind as well. I’ll try to look some more into the issue as soon as I get some spare time for it. Probably not this week, though.

EDIT: Just a thought… What happens if you disable the lag fix, but leave the frame_event_performed declaration in the CPU as well as the serialization of said flag? That way you’ll just include the extra variable in the serialization, but it would be completely unused. Does Mega Man X with MSU-1 support work or not?

#248

I just confirmed that it’s not the lag fix itself that causes issues with MSU-1 games. Simply adding the frame_event_performed flag to the CPU Status struct (and disabling the other parts of the lag fix code) will make the core crash with Mega Man X with MSU-1.

Turning off the lag fix just seems to hide the problem, at least to a degree (MMX2 and MMX3 crash even without the lag fix). It’s probably the serialization code that needs to be fixed.

#249

Thank you for identifying the actual cause of the error. It would be worth verifying whether the serialization issue is occurring on both Windows and Linux (or other systems). Valgrind may also help identify problems where memory is overwritten, given the normal debugging methods are not helping.

Has memory alignment been ruled out for the struct issue? Are the other members of the struct exceeding their memory allocation?

It’s great work in fixing the input latency bug which was present (and apparently undetected) in the snes emulators for quite a long time.

#250

If the posted stack trace at github is pointing directly to the cause of the bug, then consider testing in non-x86 system. There is alternate SSE2 and memory alignment code in mismatch.c (of Retroarch project, not libetro-bsnes) which may be disabled if the non-x86 system works with Brunnis’ fix; and likewise worth verifying the compiler’s flags for any memory alignment set by default or by build environment.

#251

[QUOTE=bidinou;42111]Apparently there was some additional input lag specific to the (s)nes. There is one more frame of lag specific to opengl (you can work around it by using the dispmanx video driver but then you have no pixel shaders – btw, I’d love to know what good overlays you can use with it :slight_smile: I tried a few but the result was not satisfying – and there seems to be one more frame of lag which is not understood for the time being.

Also I read this, is it relevant ? “Also, Lakka has now been modified upstream to apply the rt-kernel patch for more consistent scheduling with 1Khz tick rate, preemptible etc. (frame delay setting can be increased further)”

Finally, there is this frame_delay_setting that can help you get back up to one other frame of lag but you need a very fast CPU and this is super annoying to tweak. (different optimal value per system or even per game)

/me enjoys reading this thread even if he is just a dumb user

Edit : brunnis made a summary here : http://libretro.com/forums/showthread.php?t=5428&page=18 Without frame delay, there is up to 5 frames of delay while he only measures about 2 on a real SNES. I clearly see a huge difference between RetroPie on a Pi and the NES/PCE/… FPGA core on my MiST for instance. But FPGA computers are 1) very expensive 2) support few systems.[/QUOTE] Does the windows version have the dispmanx video driver? Which, has no shaders and has to use the blurry bilinear filter, right? Does that shave 1 frame off 5 frames of delay and thus you get 4 frame delay?

#252

[QUOTE=MrPoe;42584]Does the windows version have the dispmanx video driver?[/quote]no. it’s raspberry pi only.

Which, has no shaders true

and has to use the blurry bilinear filter, right?

no

re: gl shader lag - if your shader is running at 60fps i don’t think the shaders will be causing lag? also i thought someone tested with shaders and said it didn’t cause (additional) lag?

#253

Great that Brunnis findings and work was mentioned in the 1.3.5 release!