There are 3 override options. “Save Core Overrides” saves those settings for all the games running by the core and “Save Game Overrides” only saves them for the current game. However there is also “Save Directory Overrides”. This one saves the settings for all the games that are in the same folder as the one you running.
This way you can save different settings per system, (assuming you organize your roms per system) since some cores are used for multiple systems, like GenesisPlusGX for instance.
Control options (like shaders) also have their own separate overrides.