An input lag investigation


#748

as i understand it beam racing/chasing is basically a very clever and more “the right way” solution / replacement for what frame delay currently provides in retroarch, its main purpose would be to cut down on that last frames worth of latency that’s inherent when pushing full frames with vsync on the host via standard api’s (opengl, vulkan, drm, etc. and when i say last frame i mean compared to methods like using hard sync/fences in opengl, which can achieve approximately just 1 frame additional latency over original hardware)

however, wouldn’t it be a lot of work for relatively little pay off? the libretro api, retroarch and its shader pipeline, and all the libretro modules of emulators are all currently based on full frames. not only that but the core logic in many of these emulators are too. (well, to varying degrees. the point being most emulators would need their core logic tweaked as well as the libretro implementation)

it’s also not compatible with run ahead, which while not a silver bullet, provides at least as good if not better results where supported. (run ahead requires that the emulator has reasonably efficient, fully complete, and side affect free serialization support, which not all emulators do or even can provide. and it’s also game dependent) I get that there are pros and cons to each approach, but run ahead at least made a lot of sense for retroarch since it was mostly built upon features that already existed.

personally i think medanfen would be a much better testing ground for this than retroarch. for one thing it would most likely be easier and less disruptive to implement there, and for another doing so would lay the ground work for getting all the libretro mednafen cores ready to support this if and when beam chasing ever gets added to retroarch + libretro api. (note: i would advise against requesting/pestering mednafen author to implement this)

edit: fixed some terminology

edit 2: nevermind, don’t bother responding to this post, mdrejhon has already addressed pretty much everything i’ve brought up at some point or another in one of his many posts here, there were just sooo many about this i didn’t catch all of it at first. anyway, i still stand by my statement that mednafen is more suited for this


#749

I’m sure there are a lot of people like me lurking in this thread who are very interested in seeing beamracing more widely available (for example by adding these features to the libretro API :heart_eyes_cat:). Now that things are getting closer to working code it does seem like a good time to start up a bounty.


#750

I’ll create a github issue on this & then create a BountySource

How I start with a $100 bounty if someone(s) else can pledge to throw in a total of matching $100?

= $200 starter bounty for libretro raster poll API addition + 1 emulator module made compatible (e.g. NES or other)

I’ll match dollar for dollar for anything less.

Negotiable: Can be willing to match more funds beyond $100, this is just a starting point.


#751

Thanks for the tip about mednafen. However, RetroArch as far as I know, is much more well known (including variants like RetroPie, etc).

Yes, addressed all concerns:

  • The practical payoff is actually at least slightly bigger than you think. …Beamracing make it workable on Android/PI devices too underpowered for RunAhead. It also saves more lag than a VRR display does (yet beamracing is also compatible with VRR mode too). Slow emulators on slow platforms will often take 1/60 sec to render, then 1/60sec of VSYNC ON buffer delay – two frames of lag on slow platforms (VSYNC ON backpressure latency, unless you do tricks to achieve next-refresh-cycle latency reliably and consistently). Or allows performance-intensive cycle-exact emulators to reduce latency . Streaming pixels realtime to the display like the original machine did, without all the attendant pre-delays. …It’s also more preservationist friendly latency-wise, and much more faithful to original machine’s latency behavior.

  • Frameslice beamracing is still compatible with RunAhead, I posted some diagrams. Although it is not as lag-saving as I thought, it can still reduce the CPU requirements of RunAhead by roughly 1 frame’s worth per 1/60sec, since the last frame no longer needs to be surge-executed, and the visible frame (last frame) can simply be realtime streamed to display at original emulator speed (aka beamracing)

  • Despite the “concept” complexity, it takes surprisingly little code, the difficulty is understanding the black box between Present()-to-photons on a per-pixel screen-scanout basis – nearly undocumented everywhere else except at Blur Busters. But the 18-point Best Practices list (that all of us have built up, myself, Calamity, Toni, etc) will save a lot of grief.


#752

Added GitHub Issue:

EDIT: Added $120 Cash to BountySource:

I’ll dollar-match your donations

I will dollar-match all your future donations (between now and end of September 2018). I donate another $1 everytime anyone donates $1 until the BountySource hits the $360 level.

EDIT: That was FAST! BountySource now $1050 and my share maxed at $360


#753

Cool that you made a bounty for this. Hope it goes well.


#754

Awesome. What’s the difference with this and Runahead? As far as i know Runahead cuts down “internal game” lag frames that exist in the original hardware. Can this do the same?


#755

it’s completely different. It reduces display lag to very low levels, potentially similar to original hardware and/or vsync OFF without tearing.


#756

BountySource now $140 for lagless VSYNC

Someone added $10, so I also added $10.

NOTE: I am currently dollar-matching all donations (thru the $360 level) until end of September. Contribute to the Bounty pot: https://www.bountysource.com/issues/60853960-lagless-vsync-support-add-beam-racing-to-retroarch

For every $1 you donate, I will donate another $1 – until end of September – or until $360 bounty is built up.

BountySource now $200 for lagless VSYNC

Twinaphex added $30, which I matched with $30.


#758

Big contribution from bparker :open_mouth:


#759

$1050 !!! !!! !!!

Wow, that maxes out my dollar-doubling commitment.
I’ve now donated $360 total – in addition to bparker06’s donation of $650.

I’m going to try to reach out to bparker06 to personally thank him for the generosity to the bounty prize pot.

This is now currently the #32 biggest bounty on BountySource.


#760

Amazing!
Hope that it comes to fruition in Retroarch.


#761

Huh how does that work. The site shows donations from 2017.


#762

He transferred them from another bounty that wasn’t getting any interest.


#763

its realy helpful information.


#764

I’ve been reading the new tests made in the last year and I’d like to thank again Brunnis and the rest for them and for this thread in general. Hopefully someone with the knowledge gets interested enough in the subject and tries to “lag-fix” other cores such as the infamous PPSSPP one. I think this is one of the main reasons that make RA unique and it’s becoming one of main concerns no matter the level of expertise.

After all this reading though, I’m not sure what’s ultimately recommended for Windows 7, and old ATI card (good CPU power, though) and CRT 15khz usage (so no shaders): GL? D3D 9? DRD 11? Is still D3D as “laggy” as it used to be against GL? As I understand, hard gpu sync is not yet implemented in D3D 11, right?

Also, is there any guide to try the new Wasapi Windows audio driver?

Thank you!


#765

EPIC thread.

Since the 1 frame of additional input lag in Linux vs Windows 10 has been removed, is it reasonable to conclude that the Raspberry Pi 3B is the lowest-cost solution for zero input latency emulation?


#766

Last I checked (which admittedly was a good while ago), the Raspberry Pi’s default closed-source video driver added a frame of input lag that you couldn’t get rid of. The solution to that is to use the open source VC4 video driver instead and build RetroArch with DRM support (and set video_max_swapchain_images to 2). I’ve tried it and it seemed to work decently, with an important caveat: The Raspberry Pi is too slow to run even pretty easy stuff like SNES via snes9x2010 at full speed (some games will work great, but some will have slowdown). That’s without fiddling with the frame delay setting, which adds additional computational requirements.

So, I’d say x86 is still the way to go for a reasonably trouble free and well working solution, with decent performance. I’m personally running a very carefully setup system based on an Intel NUC7PJYH (latest “Atom” based CPU based on the Gemini Lake design). Unfortunately, it’s much more expensive than a Pi, but it’s also at least 4 times faster. Until we get a significantly updated Pi (I’d guess that would be the next one they release), low input lag emulation is still mostly an x86 thing.


#767

Thanks for the info, and your contribution to the retro gaming community! :grinning: Is the VC4 video driver the one where you can’t use any shaders? It’s been a while since I played with a Raspberry Pi. That might be another significant drawback for many users that would warrant more expensive hardware. Not being able to run SNES9x2010 fullspeed is definitely the biggest deal-breaker, though.

I’m using a similarly-powered system, the NUC6CAYS, running Windows 10. Are you running Linux on the NUC7PJYH? Any tips for optimal system set-up?


#768

Thanks!

No, that’s Dispmanx.

Yes, running Ubuntu 18.04. Below are my notes for an optimal setup. Replace any occurrences of {username} and {user_password} with your Linux user name/password.

EDIT: The “guide” below is based on using Ubuntu 18.04. It will get you a Linux based RetroArch system that runs automatically on boot (auto-launched from the terminal using DRM/KMS for video output). When exiting RetroArch, the system will also shut down automatically in a controlled way. Using this guide, you can pretty easily set up a RetroArch console-like system, similar to RetroPie. Combined with RetroArch’s kiosk mode, you also get a system that cannot (easily) be corrupted by kids and other non-tech-savvy people. Great for creating a locked-down emulation box.

Set Ubuntu to start in console mode (you can skip this if using Ubuntu server flavor):

1. Find GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub and change it to GRUB_CMDLINE_LINUX_DEFAULT="text"
2. Update grub with sudo update-grub
3. Tell systemd to not load the desktop:
   - sudo systemctl enable multi-user.target --force
   - sudo systemctl set-default multi-user.target

Auto-login user:

1. sudo systemctl edit getty@tty1
2. Add these lines and exchange "username" with the real user name:
   [Service]
   ExecStart=
   ExecStart=-/sbin/agetty -a username --noclear %I $TERM

On Ubuntu Server, set network timeout so that it doesn't wait 5 minutes during boot for network to come up if no network is connected:

1. sudo systemctl disable systemd-networkd-wait-online.service
2. sudo systemctl mask systemd-networkd-wait-online.service

On Ubuntu Server, set ifup timeout so that it doesn't wait a long time during system shutdown if no network is connected:

1. sudo systemctl edit ifup@enp1s0.service
2. Add these lines:
   [Service]
   TimeoutStopSec=5sec

Make input in RetroArch work by setting correct permissions:

1. Add to /etc/udev/rules.d/99-evdev.rules:
   KERNEL=="event*", NAME="input/%k", MODE="666"
2. Then reload rules with sudo udevadm control --reload-rules.

Install cpufrequtils for controlling CPU governor:

1. sudo apt install cpufrequtils

To force the CPU to max turbo frequency at all times, we want to disable processor C states. If the motherboard/system supports disabling C-states via BIOS/UEFI, use this option. If not, do the following:

1. sudo nano /etc/default/grub
2. Find the line that says GRUB_CMDLINE_LINUX="". Add the following between the quotes: intel_idle.max_cstate=1
   NOTE: If there's already text between the quotation marks, just add the text at the end of the quote, with a space separating it from the existing text.
3. sudo update-grub
4. Reboot and check that the setting has had effect. The following command shall print '1': sudo cat /sys/module/intel_idle/parameters/max_cstate

Add RetroArch Ubuntu PPA and install RetroArch and cores:

1. sudo add-apt-repository ppa:libretro/testing
2. sudo apt-get update
3. sudo apt install retroarch
4. Install desired cores like this (snes9x as example): sudo apt install libretro-snes9x

Autostart RetroArch after auto-login, set CPU governor to performance, limit RetroArch to a maximum resolution (1080p in this example) and make the system shutdown when RetroArch is shutdown (with a 5 sec window to press key to abort and go to command line):

[Note: For maximum CPU performance, also disable C states in UEFI. However, leave SpeedStep enabled, since Turbo Boost otherwise seems to stop working. Note that these UEFI settings should be combined with using the "performance" CPU governor, otherwise the CPU will still try to lower its clocks. Disabling C states did seem to provide a small but measurable increase in performance over just setting the CPU governor to "performance".]

1. Install fbset: sudo apt install fbset
2. Go to the user's home directory.
3. sudo nano .profile
4. Add the lines below to the end of the file. Substitute {username} with the actual username. Substitute {user_password} with the user's login password.

# Set performance CPU governor for all four cores.
echo "{user_password}" | sudo -S cpufreq-set -c 0 -g performance
echo "{user_password}" | sudo -S cpufreq-set -c 1 -g performance
echo "{user_password}" | sudo -S cpufreq-set -c 2 -g performance
echo "{user_password}" | sudo -S cpufreq-set -c 3 -g performance

maxWidth=1920
maxHeight=1080

width=$(echo "{user_password}" | sudo -S fbset | awk 'NR==2{sub(/.*mode "/,"");sub(/x.*/,"");print;}')
height=$(echo "{user_password}" | sudo -S fbset | awk 'NR==2{sub(/.*x/,"");sub(/"/,"");print;}')

if [ $((width * height)) -gt $((maxWidth * maxHeight)) ]
then
    echo "Current resolution is ${width}x${height}, which is higher than the specified maximum of ${maxWidth}x${maxHeight}. Starting RetroArch in ${maxWidth}x${maxHeight}."
    sed -i "s/video_fullscreen_x = \"[0-9]\+\"/video_fullscreen_x = \"${maxWidth}\"/g" /home/{username}/.config/retroarch/retroarch.cfg
    sed -i "s/video_fullscreen_y = \"[0-9]\+\"/video_fullscreen_y = \"${maxHeight}\"/g" /home/{username}/.config/retroarch/retroarch.cfg
else
    echo "Starting RetroArch in ${width}x${height}."
    sed -i "s/video_fullscreen_x = \"[0-9]\+\"/video_fullscreen_x = \"${width}\"/g" /home/{username}/.config/retroarch/retroarch.cfg
    sed -i "s/video_fullscreen_y = \"[0-9]\+\"/video_fullscreen_y = \"${height}\"/g" /home/{username}/.config/retroarch/retroarch.cfg
fi

retroarch

if read -r -s -n 1 -t 5 -p "Press any key to abort system shutdown and return to the command line..."
then
    echo " Shutdown aborted."
else
    shutdown now
fi