Mupen64Plus-Next: High Frame Time Deviation When Upscaling ParaLLEl

[Strangeguy]

Greetings. I’ve semi-recently started using RetroArch for most of my emulation needs up through 5th-gen consoles. More recently, I’ve updated to RetroArch version 1.19.1. I initially had issues with stuttering and audio popping, but for the vast majority of cores this stopped being an issue after setting the vertical refresh rate to the figure RetroArch settles at in measurement (turns out my antique “60Hz” monitor only outputs around 50Hz, boo).

The sole exception, as far as my testing has shown, has been Mupen64Plus-Next. Specifically, when using the ParaLLEl RDP plugin and upscaling past native resolution. When set to any upscaling factor, the following problems occur:

• Frame time deviation increases dramatically. In other cores, or in Mupen with other plugins/settings, frame time deviation tends to hang around 27-28% for me. With ParaLLEl and upscaling enabled, average frame time deviation is upwards of 90%
• Maximum stated framerate stabilizes at less than the core-requested 60 FPS, usually about 5% slower at 56-57 FPS. Visually, games run at a smooth framerate dotted by small, evenly-spaced stutters where individual frames frame hang on-screen long enough to notice.
• Both audio underrun and blocking settle at figures above 0%. Underrun in particular tends to settle in upper single digits. Audio also pops in sync with visual stutter mentioned above. This behavior is consistent across all audio drivers, though Wasapi seems to be the least bad in terms of underrun/blocking.

I’ve attached a picture that displays the typical core AV information when running Mupen64Plus-next as above. It should be noted that this behavior is the same regardless of upscaling factor: 2X upscale posts the same statistics, and has the same audio-visual issues, as 8X upscale. As well, any upscaling factor when using Glide64 does not result in abnormal frame time delay or audio issues.

In troubleshooting so far, I found that increasing the audio buffer size eliminated audio popping and reduced (but did not eliminate) measured underrun and blocking. Doing so also restores nominal framerate counts to 60. However, aside from introducing increasingly noticable delay in the audio, this doesn’t actually improve frame time delay or reduce visible stutter.

I’m running a rather new PC build with an Intel i7-14700K and Nvidia RTX 4070 Ti SUPER, as well as 48 GB of DDR5-7200. All drivers are up to date. When audio and visual syncing are turned off, Mupen emulation, even at maximum ParaLLEl upscaling, runs cartoonishly fast-forwarded, often upwards of 180 FPS. I do not have any reason to suspect my hardware is inadequate.

I’ve done my best to look for other RetroArch users with similar problems. So far, I haven’t found any instances of syncing/stuttering issues quite as specific as mine, or if they are lack details about how similar the issue really is. I’m therefore submitting my own experiences in hopes that it helps point towards a solution, both for myself and others.

[hunterk]

I don’t suppose your monitor supports variable refresh rate (gsync/freesync)? If so, try turning on ‘sync to exact content framerate’, which would likely solve all of your sync issues in one fell swoop.

[Strangeguy]

Unfortunately not. It’s a rather old monitor that I was simply using as an interim solution until I can relocate the computer to a home theater setup.

To be honest, I wouldn’t be surprised if the monitor itself was creating issues by having a true refresh rate that’s quite a bit lower than advertised. Though, you would think that would create issues in other cores/emulators, not just a specific Mupen64Plus plugin/setting combination.

[hunterk]

Yeah, that is indeed weird. You might try that setting anyway. It’ll result in occasional video stutters, but it should clear up your audio issues.

[Strangeguy]

I gave VRR a try and ran a few more tests in Mario Tennis (A game that I often test plugin setups on due to being particularly sensitive, and also the game I most want to use ParaLLEl RDP’s full feature set for). This first image shows figures with VRR on and 2X upscale (other scaling factors performed similarly). As you can see, compared to baseline having VRR introduced massive amounts of audio underrun and blocking, as well as droping listed framerate. Stutter/popping happened in about the same intervals as before, but each stopping point was noticably longer, about in proportion to the loss of framerate.

In order to isolate the effects of VRR, I reduced scaling factor to native resolution and tried again with only VRR enabled.Audio underrun and blocking were higher than with VRR off, but the game ran at 60 FPS with only slightly higher frame time deviation than without VRR. There was no noticable stuttering or popping, but this is already normal behavior when not upscaling with ParaLLEl.

I also did a few tests on Diddy Kong Racing, but didn’t save pictures of those due to the results being rather different from other games in my collection. In particular, native resolution and VRR on resulted in high frame time deviations of 90%+, but also unusually short frame times of only around 1ms. Perceived audio/video performance was the same as other native resolution tests.

[hunterk]

Hmm, I see you’re using WASAPI. Have you tried with any other audio drivers?

[Strangeguy]

I had settled on Wasapi as that seemed to be the best performing one. However, I tried them again on Mario Tennis with VRR disabled and 2x upscale in order to get some numbers. Unless stated, all of them ran at around 56-57 FPS (with regular stutter/popping proportional to not making 60 FPS) with frame time deviations in between 90-95%. Otherwise, figures were as follows:

• Wasapi: 8% underrun, 1.8% blocking
• Xaudio: 4% underrun, 90% blocking
• Dsound: 9% underrun, 44% blocking
• SDL2: No underrun or blocking, but framerate dropped to 54-55 FPS and stutter/pop size became proportionally longer.

As mentioned in the first post, increasing the audio buffer size in Wasapi eliminates audio popping (but also introduces noticable audio delay) while video continues to micro-stutter despite showing 60 FPS. Underrun goes down, but isn’t eliminated entirely, when doing this.

[Buccaneer]

I’ve solved all audio popping and video stuttering with the settings below (using wasapi and vulcan/glcore).

I add Preemptive Frames/Runahead in cores when it is possible.

All runs super smooth and without audio/input lags (with wired headphones and with aptx adaptive Bluetooth speaker).

I turned V-sync in Retroarch off (in NVidia driver settings it’s turned on to “fast vsync”)

• audio_latency = “64”
• video_hard_sync = "true (in glcores)
• video_frame_rest = "true
• video_max_swapchain_images = “2”
• video_refresh_rate = “60.000000”
• video_swap_interval = “0”
• vrr_runloop_enable = “true”

I also set the dynamic audio rate in audio-syncro-setting to 0.00 and wasapi to non-exclusive. Turning dynamic audio thing off (despite the fear mongering label text that says not to do so) was the most important thing.

Hope that helps.

[Strangeguy]

@Buccaneer

• audio_latency = “64”
• video_hard_sync = "true (in glcores)
• video_frame_rest = "true
• video_max_swapchain_images = “2”
• video_refresh_rate = “60.000000”
• video_swap_interval = “0”
• vrr_runloop_enable = “true”

Giving this combination of settings a try produced similar results to previous tests, including loss of framerate/increased stutter duration with VRR enabled. I will say, however, that it was a relative improvement from previous attempts to handle Vsync via Nvidia control panel: Usually the Nvidia software has done a worse job controlling RetroArch’s Vsync than RetroArch itself.

Setting frame rest on had it show up in the overlay, but what few tests I did never actually got around to any rest frames.It doesn’t seem to have any influence on the outcome.

[Buccaneer]

With frame rest I got a little bit lower CPU Load (0-5% depending on the core) I play on a notebook so every percent counts

[Buccaneer]

Just looking at your stats screenshot again and wondering why mario Tennis gets a Frame rate of 60fps. As far as i know most N64 games have 25fps (PAL) / 30 fps (NTSC) cap.I do not use mupen because it has no preemtive frames support but when I run it with Parallei core (glcore) it shows me around 25fps here. Is there an option to double fps in mupen? If yes try to turn that off.

One addition to wasapi: I’ve set the wasapi buffer to 64

[Strangeguy]

There’s no such option for Mupen64Plus-Next. My understanding is that Mupen updates at only half the stated framerate: 60 FPS simply indicates the emulator is running at full NTSC speed.

So far, the closest I’ve gotten to optimal running while unscaling is setting the base audio buffer to at least 96ms (64ms with 64ms set in the wasapi buffer doesn’t produce any better result than just having the base set to 64ms alone). This minimizes-elminates pop, but leaves very faint, barely noticable “stuck” frame every second or so. From the look of the overlay, there seems to be a fixed interval where there’s one frame that takes much longer than the rest, with a frame time of 30-60ms depending on how tuned in the settings are.

For now, my hypothesis is that there’s some combination of my old/outdated monitor being unable to maintain a 60Hz refresh rate and some interaction between ParaLLEl and the Vulkan driver that prevents me from synching audio correctly, which is why only this core/plugin seems to be affected. I should probably try on a more modern monitor/TV and see if it changes the horizon of refresh rates I can time audio to.

[GemaH]

Wait, if i understand correctly, your monitor only runs at 50hz?

[Strangeguy]

My monitor is set to 60Hz, but RetroArch’s own estimate of the true refresh rate hovers around 50.8Hz. It being an old monitor doesn’t help, but I’m wondering if the cheap DisplayPort to HDMI cable I have to use to get it hooked up is limiting it somehow. It was only an interim solution while I set up a permanent home theater setup for the PC build, but I may try a more premium cable to see if it makes a difference.

It’s strange, though. Forcing RetroArch’s refresh rate to 60Hz causes stutter/popping in all my other cores, but Mupen64Plus-Next is the only core that has issues when set to 50.8Hz, and then only when upscaling using the ParaLLEl RDP plugin. Meanwhile, my other, non RetroArch emulators don’t stutter like this: I checked PCSX2 to make sure I wasn’t going crazy and it ran Gran Turismo 3 and 4 smoothly and without any stuttering, also using Vulkan.

You would think if it were my monitor’s refresh rate, it would affect all my content, not just an N64 RetroArch core. That being said, I also tried recreating the settings as similarly as possible in a standalone N64 emulator (RMG, in this case) and it also stutters; even worse than Mupen64Plus.

[GemaH]

Some emulators behave differently than others to such things. In my experience, Mupen64+Parallel RDP always behaves differently than most other cores when it comes to syncing/frame rate whenever i test different settings. Maybe it’s a vulkan thing.

Anyway, there’s definitely something wrong with the monitor here so i’m not sure if its worth the time trying to troubleshoot this from the emulator side. But if you really want to play these games now, have you tried maybe using PAL roms that also run at 50hz natively?

[Strangeguy]

I think for now I’m going to try a faster/more premium cable (I’m going to need one for the later setup anyway) to see if I can get the full rated refresh rate out of what I have. if not, I’m not in an enormous hurry to play my N64 right now, and if I absolutely need a smooth framerate I can just play it in native resolution. I just wanted to provide as much documentation about this experience in case something else is at play. Anyway, it wouldn’t be the first time some funny Vulkan thing messed up.

Shame, though. I was running 2x upscale on an ancient 1050Ti just last year and didn’t encounter any syncing issues like this. Of course, that was on a laptop whose monitor refreshed at the right speed.

[Strangeguy]

Okay: So this was kind of silly of me.

That issue of my 60Hz monitor struggling to hold 50Hz? Yeah, I forgot having multiple displays uses up available bandwidth. I turned off my second monitor and Retroarch’s measurement on the first shot up to 60Hz and held it like a vicegrip. Everything ran perfectly from there, even the most extreme upscaling.

I have some higher quality cables on the way to make sure my graphics card can put out a steady signal to both screens. Worst case scenario, I just turn off the secondary when I’m gaming for now. Still, lesson learned. Thanks again for everyone’s help.