I am using RetroArch 1.6.4. I installed the Mednafen PSX Core and tried to run a few games on it, at all of the base settings the games ran fine, however, if I changed the internal resolution to anything other than its base x1 resolution then the game would lag. Oddly enough games would still toggle to fast forward perfectly fine, also when I quickly switched between x1 internal resolution to x8 resolution the game would run at full speed for about 10-30 seconds before it lagged again. This did not happen in the older versions of RetroArch for me.
Are you using the hardware version? The software renderer is very demanding.
No, I am using the HW version.
It sounds to me like the video is choppy rather than you are losing frames.
Reduce frame delay if set. Try threaded video. Enable vysnc if you haven’t already
If your computer was running out of horses you wouldn’t be able to fast forward much faster than you can play.
Also, insure you are using the vulkan render for the best performance.