An input lag investigation

RetroArch General
#452

@vanfanel Sorry for the slow response, vanfanel. That’s great work getting this into the RPi kernel! I’ll try to test this, but it might be a while due to other things going on in my life at the moment. 500 Hz vs default 125 Hz should shave off another 3 ms (0.18 frames) from the average input lag and 6 ms (0.36 frames) at most. Every little bit counts.

@Nesguy Thanks! In my Windows tests (which is where Hard GPU Sync applies), I’ve always used a Hard GPU Sync Frames setting of 0, as you assumed. I haven’t tested with a setting of 1, but it “should” add a full frame period worth of input lag, i.e. 16.7 ms. I would personally invest in a machine that can handle Hard GPU Sync Frames = 0, but if the system is otherwise setup correctly and you have a very low input lag screen, it may not be worth it.

Regarding Apollo Lake, I made a quick test with Windows 10 on my Asrock J4205-ITX based system a long time ago and I believe I got good performance with Hard GPU Sync Frames = 0. On Linux, I use Max Swapchain Images = 2 (similar to using Hard GPU Sync), together with Frame Delay = 8 and things work great. However, I only use Nestopia and Snes9x2010.

There are several things that could cause your performance issue, for example:

  • Display driver differences (compared to when I tested)
  • Power saving settings (try setting “High Performance” power plan in Windows)
  • Maybe you’re using more demanding emulators?
  • Have you changed the Frame Delay setting to anything other than its default value (0)? This is a very demanding setting.
  • The NUC6CAYS uses a slightly slower Apollo Lake variant than the J4205-ITX. It’s worth mentioning, but I don’t think that’s what causes your issues.
#453

What about the new RawInput input driver introduced with RA 1.6?, is that based on the work you have done Brunnis?, or does your method produce less lag?

#454

Not based on anything I’ve done. It’s a different driver to handle input and doesn’t “conflict” with any of the stuff I’ve been doing/testing. If it does give lower input lag it will do so in addition to the stuff I’ve found. Nobody has tested the effect of this yet, though.

#455

Hey Brunnis, it was the frame delay setting- I had it set to 5 instead of 0. Switched it back to 0 and now SNES emulators run fine at hard gpu sync 0 frames. Everything is awesome now, thanks! :smiley:

I also set the intel built-in graphics settings to “high performance” under the power settings, just in case that makes a difference.

#456

Has any input lag testing been done with overlays? Do you think using overlays will increase input lag?

I’d also be interested in knowing if the CRT-Pi shader introduces any input latency, since it’s the only CRT shader that will run full speed on the NUC6CAYS. I tested it using Fudoh’s 240p Test Suite manual lag test, and found no difference in latency between using the crt-pi shader vs. no shader, with an average latency of less than one frame (16ms) in both cases. Sooo… that’s pretty awesome. It’d be nice to confirm this with a more scientific test, though.

Nope, not getting any work done today!

#457

Hi, im curious about 2 things

1- what is the diff between the retroarch/libreto snes cores input lag and the original snes9x/zsnes emulators input lag?

2- what about the input lag on pi3 using the gpio? Its lower than usb control?

#458

Time for another small update! I’ve just tested the impact on input lag from:

a) Shaders b) “raw” input driver

I used the same test procedure as always (see original post in this thread), using a Core i7-6700K @ 4.4 GHz and a GTX 1080. RetroArch 1.6.0 was used and testing was performed using Windows 10 and OpenGL. 25 samples were taken for each test case.

Shaders

[Input lag below reported as number of frames at 60 FPS]

No shaders: 5.21 avg / 4.25 min / 6.00 max crt-royale-kurozumi (Cg): 5.13 avg / 4.25 min / 6.00 max crt-geom (Cg): 5.22 avg / 4.00 min / 6.25 max crt-geom (GLSL): 5.08 avg / 4.00 min / 6.00 max

There was no difference at all in the amount of input lag between no shader and using shaders. The average, minimum and maximum measured input lag was the same (within measuring tolerances). This means you can use shaders without worrying about introducing extra input lag.

For another data point, I also tested the crt-aperture GLSL shader on my Pentium J4205 system running Ubuntu 16.10 in DRM/KMS mode, using the built-in Intel graphics. I measured input lag with my usual test routine and just like my tests of the other shaders on the GTX 1080 in Windows, input lag performance remained unchanged after activating the shader.

One thing to remember, though, is that running the shader passes takes additional time. In other words, the time required to generate each frame will increase. If you’re using the Frame Delay setting to reduce input lag, you will likely have to decrease the value in order for your computer/device to still be able to finish rendering the frame on time. With my i7-6700 @ 4.4GHz and GTX 1080, I had to turn frame delay down from 12 to 8 when using the crt-royale-kurozumi shader, therefore increasing input lag by 4 ms.

So, while shaders themselves don’t add any extra input lag, the increased processing time might force you to reduce the frame delay which will have a small impact on input lag. The good news is that you’ll know exactly by how much your input lag increases since it corresponds to the amount of milliseconds frame delay you have to remove in order to retain 60 FPS.

EDIT: There might be more to this than I initially thought. See post below by hunterk where he shows results from his own testing, clearly showing a negative impact on input lag with some shaders. I have not been able to reproduce this, despite running additional tests (this post has been updated with those additional results).

The “raw” input driver

The raw input driver was introduced in RetroArch 1.6.0 and the hope was that this driver would reduce input lag. Until today, however, no tests had been run comparing it to the default dinput driver.

Unfortunately, my tests show that the raw input driver provides zero difference in input lag. At least it’s not measurable with this test method and equipment.

By the way, on a completely unrelated note, why does the menu shader get deactivated whenever you load a shader preset? Seems strange that such a basic thing as using a shader disables the beautiful shader used for the menu background…

3 Likes
#459

In my testing, some shaders very much affected latency:

shader.png1033×452
CRT-Geom added a heap of latency in both RetroArch and Higan, but it doesn’t seem to be related to the number of passes. Perhaps it’s related to the number of registers being utilized, I dunno /shrug

As for the menu shader, it’s only when you apply Cg shaders, and that’s because the menu shader pipeline for the fancy ribbon, snow and bokeh all use GLSL shaders. There’s no Cg menu shader for the fancy ribbon because it uses derivatives in a way that doesn’t seem possible in Cg, while the others require drawing a fullscreen quad and the Cg pipeline lacks some stuff for that, as well, IIRC.

We’re deprecating the Cg shaders, anyway, though, so try to move to GLSL shaders when possible (Cg shaders aren’t going anywhere and will still be available as long as Nvidia keeps offering their Cg Toolkit, they’re just not a top priority anymore). I hand-converted almost all of them, so it shouldn’t be too painful to switch.

1 Like
#460

Hmm, I’m kind of surprised that you got 3-5 extra frames of input lag with some of those shaders… I was planning to test a few more shaders initially, but didn’t really have time. Seems I’ll have to go back and at least test crt-geom or crt-lottes to see if I can replicate your results. It’s unfortunate if there’s so much variation between shaders and there’s no way of knowing the exact impact it has on input lag without testing.

Thanks for the info regarding Cg/GLSL shaders.

#461

Crt-geom is my usual shader on many systems. No way it adds 100ms here. Some particular system/driver issue for sure.

1 Like
#462

I just finished testing the crt-geom shader. I tested both the Cg and the GLSL variants. Both perform exactly the same, input lag wise, as when running without shaders, just like the crt-royale-kurozumi shader. So, to give you some actual numbers (input lag as number of frames):

  • No shaders: 5.21 avg / 4.25 min / 6.00 max
  • crt-royale-kurozumi (Cg): 5.13 avg / 4.25 min / 6.00 max
  • crt-geom (Cg): 5.22 avg / 4.00 min / 6.25 max
  • crt-geom (GLSL): 5.08 avg / 4.00 min / 6.00 max

All of the results are well within measuring tolerances.

I can only speculate as to why you got such high input lag in your tests. Are you sure 60 FPS was maintained at all times? Although I don’t really know anything about shader pipelines, I do think it makes more sense that any shaders are run within the actual frame period. It seems strange that the GPU driver would pipeline the shader execution as indicated by your results.

3 Likes
#463

Yeah, 60 fps was maintained solid the whole time. I, too, assumed it could never push the latency outside of a single frame without reducing the framerate but that’s not what my results indicated. I was using an AMD card with an Eyefinity setup, so maybe there’s something weird at play with that.

Thanks for doing the additional check. I’m honestly relieved that my results weren’t universally applicable, since I like shaders so much :stuck_out_tongue:

1 Like
#464

Another data point regarding shaders and input lag:

I activated the crt-aperture GLSL shader on my Pentium J4205 system running Ubuntu 16.10 in DRM/KMS mode. I’m using the built-in Intel graphics. I measured input lag with my usual test routine and just like my tests of the other shaders in Windows, input lag performance remained unchanged after activating the shader.

3 Likes
#465

Thank you for your investigation on shaders and raw input lags ! Do you plan to test latency with the new Wasapi Windows audio driver too ?

#466

His testing setup is only for video latency.

I assume anyone could test audio latency just by putting a microphone and a gamepad next to their speaker and then recording a button press. Take that audio file into audacity or whatever and then look at the timing difference between the button click and the sound effect firing.

1 Like
#467

It seems like Mednafen Saturn has some input lag problems as well.

#468

I have a question to you brunnis, I have actually bought myself a J4205 (in an asrock beebox) as a dedicated retro arch box but I cannot run games reasonably without having threaded video enabled. Is that a limitation of the very small form factor of the beebox thingy or am I simply doing something wrong? I am using Ubuntu 16.10. and run Retroarch 1.6.1 in KMS Mode, video_swap_chain is set to 2 and frame delay to 8. Are you using threaded video mode yourself and were your input lags tests run with it enabled? If so, what latency change does the threaded mode cause - Thanks for your help!

#469

Unfortunately, I have not tested the difference between having threaded video disabled/enabled. I always have it disabled. I only run NES and SNES emulation currently and without using shaders and using a 1080p display, I can use max_swapchain_images=2 and a frame delay of ~6. Using a higher resolution display, such as a 4K TV, will increase the system load noticeably, although I haven’t quantified it. If you’re using a 4K screen, I would suggest actually running RetroArch in 1920x1080 instead.

There’s one more thing: A while back I updated my Ubuntu 16.10 installation. This updated the Linux kernel (don’t remember the version) and Intel GPU driver. After this, I noticed significantly worse GPU performance and I had to downgrade to restore performance. So, if everything else fails, try a completely fresh Ubuntu 16.10 installation with no updates.

2 Likes
#470

Thanks for your answer! I now found out that I simply had to set frame delay down to 4 and I was able to not use video_threading! :slight_smile:

#471

Frame delay should be the last thing you tweak. Get your other settings the way you want, and then dial in the amount of frame delay that your system can handle. This frame delay amount will vary for every emulator, in my experience.