Input Lag Compensation to compensate for game's internal lag?

RetroArch General
#185

Ok, I took the time to do some tests today and, once again, this feature is just eye-wateringly amazing! I never thought I’d see such an extreme responsiveness with such a small expense in terms of performance, so I’m really excited about this. Now, here are the results of my initial tests.

All of the following was observed on a Windows 10 x64 system (latest update), with an i7 4790K processor and an Nvidia GTX970 videocard with the latest drivers. I have Hard GPU Sync ON at all times and I kept the Secondary Instance option enabled anytime I would use Run-ahead functions.

MAME:

  • I tried both mame2010 and the latest version. Both show no change in frame reaction, but I expected this outcome since these cores do not seem to support load/save states yet.

NX ENGINE (Cave Story):

  • this core does not need the Run-ahead function, since it actually has next-frame reaction already. That’s neat!

FB Alpha (both the latest core and the 2012 variant): works really well, however it seems like there is something funky happening with the audio in some specific games. I have tested the following random sample of games:

  • Dodonpachi: works flawlessly with 2-3 frames;
  • Ketsui: works flawlessly with 2-3 frames;
  • Battle Garegga: frame reaction works great and video is smooth, BUT music is slowed down by 50%. Sound effects are perfectly fine, just the music is affected.
  • Salamander: frame reaction works great and video is smooth. Sound effects exhibit some minor artifacts, as if they were doubled.
  • Gunbird 2: frame reaction works great and video is smooth. Music is fine, but the voices are sped up and noticeably higher-pitched (Marion has a chipmunk-like voice!)

MGBA: works flawlessly, without any hitch or issue whatsoever. I have tested the following games quite thoroughly:

  • Metroid Fusion (with 2 frames maximum. If you select 3 frames, some graphics start disappearing as expected, like the flash that pops off Samus’ arm-cannon when you shoot);
  • Castlevania Akatsuki no Minuet (perfect with 1 frame);
  • Super Mario Advance (all games of the series with 2 frames maximum, especially for Super Mario World which has a slower reaction just like in the SNES version. Perfect once again);
  • Minish Cap (perfect with 1 frame). I have performed all of the tests above with both the latest 0.6.1 core and a 0.4.1 build. The difference between the two is that the latter is much less resource-intensive and I could even lower the audio latency to 20ms - this is perfect for Mother 3 by the way.

BSNES Mercury Balanced: it works well overall, however with certain games I get some slowdown. Super Mario World works fine, but when trying Kirby Super Star or other graphically intensive titles, I either get some desyncing, stuttering or just slowdown overall. It does not play nicely with Frame Delay as well. I wonder if the same optimizations and speedups that Dwedit committed to snes9x can be implemented on this core too?

SNES9X: The latest core is blazingly fast. Run-ahead seems to perform very well on all the games I tested, which include:

  • Super Mario World
  • Kirby Super Star
  • Aladdin
  • The Legend of Zelda: A Link to the Past

ParaLLelN64 with Angrylion GFX Plugin: this core does not seem to produce any discernible effect at all. Activating or disabling Run-ahead does not change anything (Mario 64 reacts on jumps after 5-6 frames).

BeetlePSX: frame reaction is improved, but as someone previously reported there seems to be some noticeable video and audio stutter anytime you press a button. Also, it does not play nicely with Frame Delay at all. I have tried the following games and all exhibit the same issue:

  • Final Fantasy from Final Fantasy Origins collection
  • Crash Bandicoot
  • Akumajou Dracula Gekka no Yasoukyoku (SOTN Japanese version).

I hope this information is useful somehow to further refine and improve this input lag compensation! Many thanks to everyone who contributed to this.

1 Like
#186

I can reliably trigger the controller bug in Higan, but still have no idea what’s going on.

Easiest way to trigger it is to play Super Mario World, get a Fire Flower, hold Y and mash directional arrows.

#187

Just tested this on Mario World. Color me impressed. Mario jumps at the exact next frame! It’s amazing and the game becomes more enjoyable as a result (i always thought the game feels a bit off because of the tiny amount of lag)

1 Like
#188

Genesis+GX crashes with Nuked audio and 2nd instance now.
Mame audio is fine.

#189

yep, it’s jumping to zero.

#190

With this new runahead feature, is the “Frame delay” option becoming obsolete ?

#191

No, this is entirely different as far as I understood.

All the latency-related options that are currently featured in RA improve the input lag that occurs between the button press and the corresponding on-video reaction, which derives from a lot of varying factors (including V-Sync and this is where Frame Delay comes into play mainly). The emulation of the inherent lag of a game is left completely untouched and accurately preserved.

Run-ahead does something “bolder” in a way: it eliminates a user-defined amount of frames of internal lag, allowing you to get next-frame response by manually tweaking the values according to the individual title you’re playing. I believe these two concepts of input lag reduction are meant to co-exist and serve entirely different purposes.

2 Likes
#192

I ended up removing the Hard Audio Disable feature entirely from the Genesis Plus GX core, until I can get it working properly.

#193

Does this come at a performance cost but it’s generally safer?

#194

Hard Audio Disable only applied for the secondary core. While it did speed it up, I don’t think it was worth it if there were crashes and stuff happening.

If we need a quick fix to re-enable the feature, I’d come up with a way to disable using the Nuke sound core in the secondary core.

That said, Genesis Plus GX works fine without using a secondary core.

Horribly Unrealistic Torture Test Comparison: (Run ahead 6 frames not using secondary core)

  • Emulator built always forcing hard audio disabled: 420FPS
  • Emulator not doing hard audio disable: 245FPS

More realistic comparison: (Run ahead 1 frame using secondary core, not touching any controls)

  • Emulator built forcing hard disable always: 580FPS
  • Emulator only using hard disable for secondary core: 570FPS
  • Emulator never using hard disable: 560FPS
#195

I would like to ask a question about the second instance option.

I understand it’s there for the cores that have issues with audio when using the run ahead feature. But does it cause any problems to the other cores? I mean, can i have it enabled all the time when i use the feature and forget about it?

#196

Question about this new feature. Is their any plans for Retroarch to automatically set the Number of Frames to Run ahead? Or will we have to just figure it out for each game and emulator? If so will this setting be tracked somewhere to have this zero input lag for every game and emulator? Let me know if I am misunderstanding the feature completely lol.

#197

There’s no way for RetroArch to figure it out and automatically set it.

#198

Retroarch does have code for identifying games. and does ship with a ROM Database.

It’s just that nobody has compiled a database of runahead values for every game.

Once you have a database with runahead values for every game, you can implement automatic selection of runahead values.

This is where crowdsourcing magic comes into play.

1 Like
#199

The feature is so new that there are no known problems from leaving Secondary Core on all the time.

The only side effects I know of is:

  • Always creates a new copy of the core’s DLL file in your temp folder and deletes it every time you run a game. (There’s no sane way to tell Windows to load a second instance of a DLL file without actually copying it elsewhere)
  • Uses more RAM due to running a second copy of the emulator. So not only is the game loaded into memory twice, but also the emulator core, the emulated system’s memory, etc.
  • When switching from the Main instance to the Secondary instance, the emulator code and data could have been pushed out of cache, depending on the processor’s cache memory. This could add latency.
  • Not using the secondary core gives more consistent performance numbers, because it is not tied to how rapidly you are pressing the buttons.
  • Audio is sampled from the executed frame rather than the shown frame, so there is small additional audio latency.
  • Feature is not available on operating systems without library files. (like the Wii)
1 Like
#200

Is it generally accepted that NES games have one built-in frame of latency, while SNES games have two, or is it not that simple (differences in programming methods back in the day, etc)?

#201

No, not that simple. For example, Mario World is two frames and Donkey Kong Country is one frame.

Then Kirby’s Adventure on the NES has two frames of input lag.

It just varies from game to game. It is more common to find two frames of latency on the SNES though.

2 Likes
#202

Thanks for the information regarding setting each game or not. Hopefully we can keep a running Google Doc or something once the feature is out in the wild. Its very promising feature and I look forward to it and I am sure the longer its out their the more information that will be gathered.

#203

Let’s start compiling lists of every game on the NES/SNES/Genesis/etc. and the amount of built-in lag frames they have.

I am willing to add new metadata to libretro-database for the purpose of adding lag frame information to game entries.

6 Likes
#204

What is the suggested way to test this? I have seen people use high speed cameras but it that the only way?