Measuring latency

[denilsonsa]

(I think this has been discussed to death already, but I’m gonna post anyway…)

I’d like to be able to measure the latency of my setup. Because I cannot improve what I cannot measure.

The problem to be measured

I know we have:

• input latency (how long between the physical button press until the computer detects it)
• display latency (how long between outputting the video and the image showing on the screen)
• emulation latency (which can vary per core and per host CPU performance, with many settings and tweaks to minimize it)
• game engine latency (some games may take a non-zero amount of frames before reacting on-screen)
• post-processing latency (adding shaders and stuff can take time)
• operating system latency (which is outside of RetroArch’s control)

Some of these I can reduce by changing some settings. Others I can reduce by replacing my hardware (controllers, display), which can get expensive and isn’t always feasible (e.g. although a CRT has the lowest latency, I’m not getting one).

So, I’d like to be able to measure it. Either the total latency, or the individual parts.

Existing research and tools

There is an official Libretro Doc talking about latency, which suggests several settings that can be tweaked. But it doesn’t explain how to measure the latency.

There is a blog post from Recalbox about input lag and latency, where they even created a tiny one-off gadget called Latency Bro.

I’ve seem MiSTer Laggy project for MiSTer FPGA. It requires a (cheap) dedicated hardware sensor and it can measure the output+display latency. I wonder if RetroArch has anything similar.

I’ve seen An input lag investigation, which modifies a USB game controller to add an LED to a button press. Then it uses a 240fps video recording to manually measure the total latency.

I’ve seen an iOS app called Is It Snappy? and an Android app called Frameskip that are able to measure the time between two frames in a video.

I’ve seen MiSTer Controller Latency website and MiSTer Input Latency sheet and Gamepadla databases of controller latency (both wired and wireless). I’ve also seen software to measure controller latency, but that also requires modifying the controller.

I know there are two lag tests in 240p, from the 240p test suite which is available for most emulated systems.

But so far I’ve seen nothing measuring the audio latency. And I wonder if audio output could also be used to help measuring the other kinds of latency.

Game engine latency

The release notes for RetroArch 1.7.2 include the following:

How to check the amount of lag frames a game has

RetroArch has the ability to pause a core and advance it frame by frame. Perform the following steps to determine the amount of lag frames of a game:

1. Pause emulation (press ‘p’ button on keyboard).
2. Press and hold the jump button on the controller.
3. Advance emulation frame by frame (press ‘k’ button on keyboard) until the character jumps.

The number of k presses before you get a reaction should be the number of lag frames you can safely remove with run ahead.

These steps will eliminate the input and output latency, allowing us to isolate the game engine latency. This is similar to how people create Tool-Assisted Speedruns.

Unfortunately, stochastic parrots are outputting these steps without context and claiming it measures something else.

Objective

I want to know:

• How much lag does my TV introduce?
• Does my LCD monitor have less latency than my TV?
• Does the amount of lag change over different video signals? (e.g. HDMI, DisplayPort, VGA, SCART, …)
• How much latency do I get in RetroArch on Linux launched from Steam game mode?
• How much latency do I get in RetroArch on Linux launched from desktop mode?
• How much latency do I get in RetroArch on Lakka?
• How much does the latency change on different computers?
• Can a cheap and underpowered old mini-PC deliver acceptable latency?
• How much does the latency vary per RetroArch core? Should I have different settings for each core?

Question

Do you know anything else regarding measuring latency? Any other tool? Any other approach? Anything not yet mentioned in this topic? Anything that helps achieving the objective of measuring the actual latency of each setup?

Bonus points for easy/fast/precise solutions, because they let us iterate quicker. While recording a video and then analyzing frame-by-frame is doable, it is also tedious and time-consuming. Having quicker measurements makes it easier to observe the effect of each variable.

[sonninnos]

Fastest and simplest comparison: Make sure mouse is enabled in menu and then simply toggle mouse grab so that you see both menu and OS cursor, and then feel and observe how far behind the menu cursor is.

Another similar method instead of menu is using cores that draw a lightgun/pointer crosshair on the next frame just like the menu. Remote RetroPad in pointer mode will achieve the same results.

Still might be preferable to have a core running so that you can toggle menu between config changes, since some options still have issues coming to life when running menu only…

Mouse movement is superior compared to button press timing and reaction which will fool the brain more easily. Of course this is measuring presentation only, so any lag caused by the display is out of this loop, and so is frame stepping, which is why tests should be done with next frame response cases only.

[kokoko3k]

When I need to, I use a phone camera with slow motion record and capture a short clip of the game running while my finger presses the “fire” key in foreground.

the time difference between the fire hit and the game reacting to it is the total latency.

[alexb3d]

I saw two hardware options there; they look good; for emulation purposes, it would be interesting to capture the differences between the consoles and the emulators/cores.
The LED option is effective, although recording the 16 ms of each reloc cycle at 240 fps may be inaccurate.

Your questions:

• Flat-screen displays, the response time indicated—12 ms—would be the minimum required to achieve acceptable results. Avoid unknown brands; their response time is usually double or triple that.
  A CRT has no latency unless it has filters, or some other effect.

• LCD monitor or TV? We usually refer to all flat-screen displays as “LCDs.”

• The port you use doesn’t cause latency.

• I don’t know about Steam, but I imagine the game mode works the same way as it does in RetroArch on the desktop or Lakka—they all have it, and it’s enabled.

• It doesn’t depend on an old computer or mini-PC, but rather on the processor not exceeding the system’s processing capacity.

• There is no list of minimum requirements, but if the emulator is high-precision, it will require more hardware resources. For example, PCSX ReARMed runs very well on a Pentium Dual Core, whereas Beetle PSX does not run well on a first-generation i7. In these cases, it is necessary to choose an emulator based on the hardware’s capabilities.

I don’t think there’s a way to measure exact latency without dedicated hardware.

Predictive downsampling is used to measure the latency of the original hardware, not the latency caused by emulation on your PC. In most cases, this latency is part of the game’s design; in others, it’s a disaster. It helps eliminate the frames that the console generates natively and improves performance on LCD monitors.

I can give you some tips on how to visually measure input lag (?).
Use the Stella core. The Atari 2600 was the only console that didn’t have input lag.
Use a brand-name controller; generic ones usually cause lag.
If your LCD has a game mode, turn it on; if not, try turning off any automatic processing, contrast, color, noise reduction, or fine-tuning.
No actives shaders.
Use RetroArch with the default settings; don’t enable anything unusual.

[simplyetchedsite]

The slow motion phone camera idea sounds like the most practical way to check this at home. I also found that turning on game mode and keeping shaders off made the delay feel noticeably smaller in fast games.