I’m not really familiar with the codebase of this project however I have a XP 32 system at hand which I use the emulator on so I decided to look at what is causing the crash, I don’t know whenever this is helpful for fixing the crash at the source level, but as far as the faulty ASM code in the compiled PE goes…
First of all, the newest retroarch-phoenix builds crash at the following instruction:
00404D05 |. 83EC 30 SUB ESP,30
00404D08 |. B9 00000000 MOV ECX,0 ; |
00404D0D |. 8B7424 40 MOV ESI,DWORD PTR SS:[ESP+40] ; |
00404D11 |. 0F2805 B0944700 MOVAPS XMM0,DQWORD PTR DS:[4794B0] ; |
00404D18 |. 8B46 0C MOV EAX,DWORD PTR DS:[ESI+C] ; |
00404D1B |. 8B50 10 MOV EDX,DWORD PTR DS:[EAX+10] ; |
00404D1E |. 807A 35 01 CMP BYTE PTR DS:[EDX+35],1 ; |
00404D22 |. 19C0 SBB EAX,EAX ; |
00404D24 |. 25 0000FBFF AND EAX,FFFB0000 ; |
00404D29 |. 05 0000CF00 ADD EAX,0CF0000 ; |
00404D2E |. 807A 05 00 CMP BYTE PTR DS:[EDX+5],0 ; |
00404D32 |. 0FB652 34 MOVZX EDX,BYTE PTR DS:[EDX+34] ; |
00404D36 |. 0F294424 10 MOVAPS DQWORD PTR SS:[ESP+10],XMM0 ; | <- crashes on this instruction
00404D3B |. 0F45C1 CMOVNE EAX,ECX ; |
00404D3E |. 894424 04 MOV DWORD PTR SS:[ESP+4],EAX ; |
00404D42 |. 8D4424 10 LEA EAX,DWORD PTR SS:[ESP+10] ; |
00404D46 |. 895424 08 MOV DWORD PTR SS:[ESP+8],EDX ; |
00404D4A |. 890424 MOV DWORD PTR SS:[ESP],EAX ; |
00404D4D |. FF15 8CD64800 CALL DWORD PTR DS:[<&USER32.AdjustWindowRect>] ; \AdjustWindowRect
A wild guess is that this could be some alignment issue. Older versions (the last version without this crash I’m aware of was WIP3, which I’m using now in conjunction with the newer retroarch EXE) do not include those instructions, perhaps they were compiled with different optimizations? The newer versions appear to initialize the window rectangle with 0, 0, 640, 480 using a single SSE instruction to copy the data from a global, which is the source of the crash, older versions initialize it inline as is visible below.
004034D0 /$ 57 PUSH EDI
004034D1 |. 56 PUSH ESI
004034D2 |. 53 PUSH EBX
004034D3 |. 89CB MOV EBX,ECX
004034D5 |. 83EC 30 SUB ESP,30
004034D8 |. B9 00000000 MOV ECX,0 ; |
004034DD |. 8B7424 40 MOV ESI,DWORD PTR SS:[ESP+40] ; |
004034E1 |. C74424 10 00000000 MOV DWORD PTR SS:[ESP+10],0 ; |
004034E9 |. 8B46 0C MOV EAX,DWORD PTR DS:[ESI+C] ; |
004034EC |. 8B50 10 MOV EDX,DWORD PTR DS:[EAX+10] ; |
004034EF |. 807A 35 01 CMP BYTE PTR DS:[EDX+35],1 ; |
004034F3 |. 19C0 SBB EAX,EAX ; |
004034F5 |. 25 0000FBFF AND EAX,FFFB0000 ; |
004034FA |. 05 0000CF00 ADD EAX,0CF0000 ; |
004034FF |. 807A 05 00 CMP BYTE PTR DS:[EDX+5],0 ; |
00403503 |. 0FB652 34 MOVZX EDX,BYTE PTR DS:[EDX+34] ; |
00403507 |. C74424 14 00000000 MOV DWORD PTR SS:[ESP+14],0 ; |
0040350F |. 0F45C1 CMOVNE EAX,ECX ; |
00403512 |. 894424 04 MOV DWORD PTR SS:[ESP+4],EAX ; |
00403516 |. 8D4424 10 LEA EAX,DWORD PTR SS:[ESP+10] ; |
0040351A |. C74424 18 80020000 MOV DWORD PTR SS:[ESP+18],280 ; |
00403522 |. C74424 1C E0010000 MOV DWORD PTR SS:[ESP+1C],1E0 ; |
0040352A |. 895424 08 MOV DWORD PTR SS:[ESP+8],EDX ; |
0040352E |. 890424 MOV DWORD PTR SS:[ESP],EAX ; |
00403531 |. FF15 C8764A00 CALL DWORD PTR DS:[<&USER32.AdjustWindowRect>] ; \AdjustWindowRect
I couldn’t find any other references to AdjustWindowRect in the executable so I guess the culprit location in the source could be deducted from that.
Nopping out the offending XMM0 move “”“fixes”"" the crash, but it leaves the UI in a malformed state.