Wireless Xbox 360 controller, index and user numbers keep changing

First, I love Retroarch and all the work going in to it is greatly appreciated. I did see similar questions, but the threads were old so hopefully making a new one is ok.

I have a pretty annoying problem with wireless Xbox 360 controllers. Seemingly randomly when launching RA, controller index and user numbers change, and the gamepad currently connected does not work, input screen shows N/A and I have to reconfigure it using keyboard. Every time this happens, I select the controller again and save settings and it works for a time, but eventually messes up again.

Sometimes I use 2 controllers for multilayer, and this does not help the issue. Sometimes it also messes up bindings(Well of course, if they are saved for a different user/device index).

So is there any way to fix this outside of binding everything to keyboard and emulating keys with xpadder or something similar? Stand alone emulators do not usually suffer from this issue.

Windows 10 Retroarch 1.5