An input lag investigation

What? I’m an “input lag is in your head” guy? Really? Is that why I’ve lavished praise on Brunnis and just told you “Few things make me happier than the developments of this thread”? Is that why I’m critical of byuu’s rejection of the possibility of further improvements to input lag, and of his suggestion that an additional 16ms doesn’t matter?

I’m afraid your emulator lacks I/O accuracy. Nothing to be gained from this. Moving on…

"If anything, the idea, if true, that VC in Dolphin performs better than VC on the Wii is just testament to the Wii’s weakness. " This isn’t the idea at all. I’m sorry if this wasn’t clear enough for you. Yes, please stop discussing you are just confusing the issue. If you aren’t willing to help confirm that’s fine.

as stated in the op he’s using a retrolink usb controller. this is a basic usb hid device, not something you typically have to worry about. the controllers i’d be suspicious of would be ones that require specialized drivers or especially those cheapo console to usb adaptors. the latter are notorious for additional input lag.

also, before ppl bring up usb polling rates keep in mind that the 8ms default usb polling rate != 8ms of additional latency, that just means that your os only polls the controller for input every 8ms. so just ball parking it that’s more like ~4ms of additional latency on the avg (+ ~8ms worst case for just missing an input change, + ~0ms if just caught it). but compared to everything else this is kind of an insignificant factor.

according to the lead libretro/retroarch devs the video display stack (os wm / compositor, video drivers, etc) are responsible for nearly all additional latency on pc hardware. if the various software involved didn’t excessively buffer cmds and frames and gave us a universal way to control how/when the hardware draws and pushes frames to the display, then even with vsync & double buffering we’d be able to emulate a 2d raster based system with only ~1 frame of additional latency (that is step emu (guest vblank first, then active) to generate a frame, tell the gfx card to draw said frame immediately into the back buffer, then wait for vsync and flip front/back buffers). vulkan may very well allow us to have such control, but from what i understand at this point the drivers are still very much a wip.

@vookvook re nintendo’s vc emu’s: software running on the wii/u has pretty much direct control over the hardware, add to the fact that nintendo probably knows how to use that hardware better than anyone else and so i’d assume their emu’s probably have only ~1-2 frames of additional latency over the original hardware. i’ve never examined dolphin’s source code in depth so i have no idea how it’s setup to work, but in a very best case scenario gpu cmds from the guest would be translated 1-1 to equiv. cmds on the host resulting in no additional latency. however, even if this is the case as stated above the video display stack on the host is probably going to add a few frames.

while it’s possible i think it’s unlikely that emulation of games thru dolphin would result in less latency than retroarch + nestopia/fceumm. equal maybe, but even then… i think you’d really have to have retroarch configured sub-optimally in order for this to be true. according to tests done by Brunnis he’s been able to achieve as low as ~2 frames of additional latency over orig hardware (even lower when adding a small frame_delay) in retroarch using the following settings, try them and see if things improve at all:


audio_sync = "false"
video_threaded = "false"
video_fullscreen = "true"
video_driver = "gl"
video_vsync = "true"
video_max_swapchain_images = "1"
video_hard_sync = "true"
video_hard_sync_frames = "0"
video_disable_composition = "true"

(note video_max_swapchain_images probably doesn’t apply here, i’d have to double check the source to retroarch to be certain but i set it here anyway, figure it can’t hurt)

[QUOTE=e-tank;45019]a

@vookvook re nintendo’s vc emu’s: software running on the wii/u has pretty much direct control over the hardware, add to the fact that nintendo probably knows how to use that hardware better than anyone else and so i’d assume their emu’s probably have only ~1-2 frames of additional latency over the original hardware. i’ve never examined dolphin’s source code in depth so i have no idea how it’s setup to work, but in a very best case scenario gpu cmds from the guest would be translated 1-1 to equiv. cmds on the host resulting in no additional latency. however, even if this is the case as stated above the video display stack on the host is probably going to add a few frames.

while it’s possible i think it’s unlikely that emulation of games thru dolphin would result in less latency than retroarch + nestopia/fceumm. equal maybe, but even then… i think you’d really have to have retroarch configured sub-optimally in order for this to be true. according to tests done by Brunnis he’s been able to achieve as low as ~2 frames of additional latency over orig hardware (even lower when adding a small frame_delay) in retroarch using the following settings, try them and see if things improve at all:


audio_sync = "false"
video_threaded = "false"
video_fullscreen = "true"
video_driver = "gl"
video_vsync = "true"
video_max_swapchain_images = "1"
video_hard_sync = "true"
video_hard_sync_frames = "0"
video_disable_composition = "true"

(note video_max_swapchain_images probably doesn’t apply here, i’d have to double check the source to retroarch to be certain but i set it here anyway, figure it can’t hurt)[/QUOTE]

hey these settings worked great. I can’t tell the difference between them any more.

Thanks.

[QUOTE=wareya;44919]Just to note, this is expected when using “sync audio” on audio backends that take large chunks of audio at a time, which is most of them because everything is horrible.

On linux with OSS I’m capable of pushing the hardware audio buffer size to 256, which close enough to an integral dividend of 800 (=1/60th of a second at 48000hz) that it doesn’t act so erratically, but that’s a pretty unstable OS environment (oss on linux is a fourth-class citizen). On windows your best bet is normally the WASAPI backend, but on most machines the chunks that WASAPI shared mode takes out are ~10ms in size, so you’re still going to get noticable stutter even though the chunks are smaller than 1/60th of a second. This is one of the drawbacks of higan’s audio output chain being too simple.

It should be possible to make audio sync work better for input latency on audio backends that take more than 1/60th of a second at a time, but there’ll still be microstutter.[/QUOTE] Thanks for this explanation, wareya.

[QUOTE=e-tank;45019]as stated in the op he’s using a retrolink usb controller. this is a basic usb hid device, not something you typically have to worry about. the controllers i’d be suspicious of would be ones that require specialized drivers or especially those cheapo console to usb adaptors. the latter are notorious for additional input lag.

also, before ppl bring up usb polling rates keep in mind that the 8ms default usb polling rate != 8ms of additional latency, that just means that your os only polls the controller for input every 8ms. so just ball parking it that’s more like ~4ms of additional latency on the avg (+ ~8ms worst case for just missing an input change, + ~0ms if just caught it). but compared to everything else this is kind of an insignificant factor.

according to the lead libretro/retroarch devs the video display stack (os wm / compositor, video drivers, etc) are responsible for nearly all additional latency on pc hardware. if the various software involved didn’t excessively buffer cmds and frames and gave us a universal way to control how/when the hardware draws and pushes frames to the display, then even with vsync & double buffering we’d be able to emulate a 2d raster based system with only ~1 frame of additional latency (that is step emu (guest vblank first, then active) to generate a frame, tell the gfx card to draw said frame immediately into the back buffer, then wait for vsync and flip front/back buffers). vulkan may very well allow us to have such control, but from what i understand at this point the drivers are still very much a wip.[/QUOTE] Yep, while I don’t like to make assumptions, I wouldn’t expect this particular controller to add any significant delay in itself. I have been asked to look into the effect of de-bouncing circuitry in the controller and while that would be interesting, I don’t really plan on going into such depth. Besides, with my test configuration, there really isn’t much of the input lag that we can’t already account for given the known quantities. Please see this post where I made a summary:

http://libretro.com/forums/showthread.php?t=5428&p=41748&viewfull=1#post41748

OpenGL on this Radon R9 390 with the 16.5.2.1 drivers really seems to have a minimal amount of buffering. It seems to start scanning out the framebuffer immediately or almost immediately after swapping buffers, which means that there’s hardly any room for improvement. That said, my recent tests of the latest 16.7.3 driver shows how volatile the situation is. That new driver added rather siginficantly to the input lag, for no apparent reason.

Good to hear!

Hi Brunnis and others,

You can check the polling rate of your game controller by running the tool USBView: http://msdn.microsoft.com/en-us/library/windows/hardware/ff560019(v=vs.85).aspx

The Device Bus Speed and the bInterval value together determine the polling rate in milliseconds, as shown on the following page (second table is for full speed devices):

https://msdn.microsoft.com/en-us/library/windows/hardware/ff539317(v=vs.85).aspx I guess for minimal lag from the controller polling side you want it to be a Full Speed device with a bInterval value of 1 (resulting polling rate is 1ms).

[QUOTE=Dr.Venom;45113]Hi Brunnis and others,

You can check the polling rate of your game controller by running the tool USBView: http://msdn.microsoft.com/en-us/library/windows/hardware/ff560019(v=vs.85).aspx

The Device Bus Speed and the bInterval value together determine the polling rate in milliseconds, as shown on the following page (second table is for full speed devices):

https://msdn.microsoft.com/en-us/library/windows/hardware/ff539317(v=vs.85).aspx I guess for minimal lag from the controller polling side you want it to be a Full Speed device with a bInterval value of 1 (resulting polling rate is 1ms).[/QUOTE] Thanks for the tip! I just checked my LED rigged RetroLink controller and bus speed is “Low” and bInterval is 0x0A. This translates to a polling interval of 8 ms, which is what I expected. As far as I know, it’s what pretty much all gamepads use, but I’d be happy to see examples of ones using higher polling rate.

Either way, 8 ms polling rate leads to an average of 4 ms contribution to input lag. Hunting down a gamepad with higher polling rate should probably not be high on one’s list of priorities…

If I use Vsync ON and Frame Delay of let’s say 10… Will there be any improvement turning GPU Sync to 0 in input latency? It seems like using Frame delay and GPU Sync together is a little better but hard to tell. GPU Sync alone is not good enough, though. The reason I ask is because with CRT Royale shader I have to lower the Frame Delay if I have GPU Sync on, or performance drops.

yes, when used in conjunction with vsync frame_delay should remove that many ms worth of input latency.

that’s to be expected, many shaders are computationally expensive and will reduce the amount of time you’d otherwise be able to safely allocate via frame_delay.

[QUOTE=Brunnis;44578]Finally, an important observation which could very well compromise the Vulkan results in this post: The rendering does not work as expected in Vulkan mode. I first noticed this when playing around with Yoshi’s Island. As soon as I started moving, I noticed that scrolling was stuttery. It stuttered rapidly, seemingly a few times per second, in a quite obvious and distracting way. I then noticed the very same stuttering when scrolling around in the XMB menu. I tried changing pretty much every video setting, but the stuttering remains until I change back to OpenGL and restart RetroArch.

When anayzing the video clips from my tests, I noticed that the issue is that frames are skipped. When jumping with Mario in Yoshi’s Island, I can clearly see how, at certain points, a frame is skipped.

My system is pretty much “fresh” and with default driver settings. Triple buffering on/off makes no difference. The stuttering appears in both RetroArch 1.3.4 and the nighly build I tested. Same behavior with Radeon Software 16.5.2.1 and 16.7.3.

I’ve seen one other forum member mention the very same issue, but I couldn’t find that post again. I’ve seen no issues on GitHub for this. I doubt that this is a software configuration issue, but I guess it could be specific to the GPU (i.e. Radeon R9 390/390X). Would be great if we could try to get to the bottom of this, because it makes Vulkan unusable for actual gaming on my setup and could also skew the test results.[/QUOTE] I tested the new Nvidia driver version 372.54 which bumps up Vulkan to version 1.0.13 and it fixes the stuttering I was noticing in Symphony of the Night. I guess they fixed something with Vulkan’s Vsync. I still can’t fast forward though.

It tears now with vsync disabled as it should, but it ALSO stutters somehow. Stuttering and tearing are usually mutually exclusive.

So i have a Gsync monitor what settings should i be…setting to get the lowest input lag??

vsync OFF, audio sync ON, audio latency down as low as you can get without crackling.

Just tried the latest AMD driver (16.8.2) which includes VulkanRT 1.0.17.0. However, this VulkanRT version was already installed on my system. I tried anyway, but the rather severe stuttering remains. Max swapchain images set to 2 or 3 makes no difference.

That’s too bad. Hopefully AMD doesn’t end up too far behind on fixing that.

Hi ! Sorry for this dumb user question : as of now, using retropie, dispmanx with its 1 frame reduced delay makes a real difference to me and I cannot stand the delay in ogl mode. Believe it or not, my characters miss jumps in opengl mode and not in dispmanx mode in several games :). However I cannot stand the bilinear or rendering without any filter. It seems one cannot apply overlays in dispmanx mode. Or maybe I missed something ? Cheers

If you’re interested in N64 emulation, there is this Mupen + GlideN64 libretro core that’s quite great except for a really high input lag. When the frame buffer is ON we got about 80ms lag (probably more than 100ms in real conditions).

If you feel like checking where the input polling occurs (as you should be the expert for that :slight_smile: ) the github and issue are there: https://github.com/loganmc10/GLupeN64/issues/55

1 Like

No wonder why my lag test with Mupen64-Libretro and any plugins on both Project64 and Mupen64plus are different. Doom64 and Quake games are easier to play in Mupen64-Libretro with no vsync. It seems to add 4 frames on these standalone emulators the last time I tested.

[QUOTE=Tatsuya79;47205]If you’re interested in N64 emulation, there is this Mupen + GlideN64 libretro core that’s quite great except for a really high input lag. When the frame buffer is ON we got about 80ms lag (probably more than 100ms in real conditions).

If you feel like checking where the input polling occurs (as you should be the expert for that :slight_smile: ) the github and issue are there: https://github.com/loganmc10/GLupeN64/issues/55[/QUOTE] I’ll have to decline the offer for the time being. I really don’t have much time right now and I’ve already spent a lot of time on investigations like this lately. I haven’t posted in a while, but I have been busy with some stuff that I’m not ready to show just yet: I’m testing the new (and still experimental) VC4 OpenGL driver for Raspberry Pi to determine if it has any positive effects on input lag. While I have been able to do some tests, I have also run into what seems to be a bug in the driver. I’m currently awaiting feedback on that, but unfortunately there’s no timetable.

Ok, hope you’ll have success with that.