I’m not sure there is a RetroArch general option that will make the game match the speed of your monitor.
However if, like me, you don’t have a G-Sync/FreeSync monitor/GFXcard-capable setup you can just add custom refresh rates/resolutions to your system and run the game at its native refresh rate.
So instead of speeding up the game to 60hz to avoid jerky video, it runs at its native (or near) speed while avoiding video jerkiness.
For example, Mortal Kombat II (and other games of the same board) runs at ~54.70. So I created a custom 1282x722 resolution/refresh rate with Nvidia control panel of 55hz (not exact but pretty close).
Then added these lines to MAME’s mk2.cfg
video_autoswitch_refresh_rate = "2"
video_windowed_fullscreen = "false"
video_fullscreen_x = "1282"
video_fullscreen_y = "722"
video_refresh_rate = "55.000000"
so whenever the game starts the monitor will automatically switch to that resolution and refresh rate. No stuttering at all. Now since it’s at 55hz and not 54.70hz you might expect video stuttering or frame skip/drop every ~1-2 sec or so. But it doesn’t happen. If however I try to run the game at a say, 58hz then you can clearly see a consistent frame drop every second or so. So it seems MAME or even RA itself will speed up (or down) the game/content if it’s within a certain threshold. Not sure exactly how it all works tbh, video/audio sync is probably one of the most complex aspect of RA. It also seem to be heavily core-dependent.
If you don’t have an nvidia card, I think there are other programs that can do the same job for your system. I’m on Windows for reference.