What operating system are you on? I regrettably don’t have any experience with RetroArch on Linux yet (Hell, I barely have any experience but I’ve been working like a crazy person to understand and configure this on my PC), but I think I can you help you out if you’re on Windows. Or, should everything be uniform, I may be able to help you out regardless.
Assuming you already have specific configuration files for each core (by setting core_specific_config = “true” in RetroArch.cfg), load the configuration file of the core you’d like to change in your frontend (named core_libretro.dll.cfg for me). From there, configure the controller and save the configuration. That will go to the /config folder in the main retroarch folder. From here, all other settings should be equal to the core’s original configuration file, so just delete the original cfg file (but back it up first in the admittedly possible possibility that this messes your stuff up) and rename the one you just saved to match the name of the newly deleted config file.
Alternatively, you can mess with the internals of the original cfg file (namely copy and paste the newly saved file’s input_player1_ line block to the original config file [and the input_player1_joypad_index if different controllers register as different joypads with a different index, but I kinda doubt that would be the case in a USB adapter]), but that’s a bit more tedious and the aforementioned way worked perfectly fine for me.
I’m sure there are easier ways as I’ve yet to explore autoconf or core overrides, but again I’ve gotten acceptable results from this process. The only problem I encounter is that once you close content, you’ll have to continue using that controller (or risk weird controls on the new gamepad should they be under the same joypad index) until you load content from a different core (or reload retroarch’s config), where then that core’s configuration file will load and leave you with the desired gamepad newly configured to use.