Xbox Series X Controller only works via Steam

Hi everyone, as I have recently recognized, my new Xbox Series X controller doesn’t work with Retroarch on Windows 10 when I directly start Retroarch. I am not able to do any mapping…I have already tried different drivers, only sdl2 seems to work somehow in the beginning but when I start a game a popup comes saying the device has been disconnected… Since I am using Steam a lot I have added Retroarch as non-steam game to start it (yes, I am aware there’s a dedicated Steam version around, but it is beta and limited in the number of emulators). Anyway, when I start Retroarch via Steam and I have the XBox controller support enabled, I can use the controller without any problem in Retroarch! So question for me is what Steam does with the controller actually to make it working. Does anyone understand this behavior or could tell me how to use the controller in Retroarch even without Steam? Thanks in advance.

Id stick to Xinput, id also disable steams controller support for desktop to see if that helps.