Environment issue? Simple or not so simple?

[NoCforMe]

I certainly don't want to preempt sinsi or anyone else here who knows more than I do about X64 programming. However, I ran across an explanation of spill space (aka shadow space) which may be helpful. Apparently, there are two main reasons for providing this memory space: debugging and handling functions with variable-length argument lists (varargs):

The Shadow space (also sometimes called Spill space or Home space) is 32 bytes above the return address which the called function owns (and can use as scratch space), below stack args if any. The caller has to reserve space for their callee's shadow space before running a call instruction.

It is meant to be used to make debugging x64 easier.

Recall that the first 4 parameters are passed in registers. If you break into the debugger and inspect the call stack for a thread, you won't be able to see any parameters passed to functions. The values stored in registers are transient and cannot be reconstructed when moving up the call stack.

This is where the Home space comes into play: It can be used by compilers to leave a copy of the register values on the stack for later inspection in the debugger. This usually happens for unoptimized builds. When optimizations are enabled, however, compilers generally treat the Home space as available for scratch use. No copies are left on the stack, and debugging a crash dump turns into a nightmare.

Of course, the space can be used by a compiler, but we're not using one (this is assembly language), so the spill space just sits there unused, even though it must be "allocated" (by moving the stack pointer before calling the function).

Any better explanations welcomed here.

[jj2007]

below stack args

Really?

[sinsi]

The Shadow space (also sometimes called Spill space or Home space) is 32 bytes above the return address

It's the 32 bytes, not 32 bytes above.

below stack args

Really?

Actually, yes. On entry

Code Select Expand

RSP+0     return address
RSP+8     space for 1st arg
RSP+16    space for 2nd arg
RSP+24    space for 3rd arg
RSP+32    space for 4th arg
RSP+40    actual 5th arg - first stack arg

I've never heard the debugging reason though, the main use for me is saving the args for later use, since the first WinAPI call will probably trash RCX etc.
If you don't use it then there's 32 bytes for any small locals for you to use.
I've seen code that uses it to save registers instead of pushing them.

[jj2007]

Actually, yes. On entry
Code Select Expand

Code Select Expand

RSP+0     return address
RSP+8     space for 1st arg
RSP+16    space for 2nd arg
RSP+24    space for 3rd arg
RSP+32    space for 4th arg
RSP+40    actual 5th arg - first stack arg

You are right, as usual

[Nate523]

Thank you for the detailed explanation, I happen to have come across something interesting that has confused me further, because what we have discussed I have attempted on my personal computer at home computer by installing Visual Studio 2022 Express edition, I today got my work computer setup with Visual Studio 2022 Professional, and when I setup a project the same way for MASM as I did before, the code seems to work without the addition of:

Code Select Expand

sub rsp, 28h
When I say work, I mean that I get an exit code other than : code -1073741819 since from other examples I have been doing that seems to be a universal you messed something up. Further more if I add the:

Code Select Expand

mov rcx, 78
instruction the exit program will end with : code 78

Any ideas to what the difference is? Makes my head spin a bit that in one computer it works and the other it doesn't! However, the code might not be working properly because even though it runs, I'm learning to debug by stepping through watching the registers, and everything seems to work but I've noticed that no matter which way with or without the "sub rsp, 28h" instruction, the line :

Code Select Expand

call ExitProcess

Is throwing an exception :

Code Select Expand

Exception thrown at 0x00007FF6A28C1031 in TestASM.exe: 0xC0000005: Access violation writing location 0xFFFFFFFFFFFFFFF8
So me using the term "worked" might be incorrect. So two things that interest me at this point:

1. One computer I need to add the "sub rsp, 28h" in order for the program to work at all, but another computer environment works without (mind you I haven't explored to many other programs so perhaps spill space and such will become more important later as I program something more complex)

2. Why is "call ExitProcess" throwing an exception? Stepping through everything seems to work until that point, and the program will even run if I just run it giving me a exit code as expected, but I don't like getting an exception if it truly is an error.

Once again, thanks for the help, it feels like I am at least making baby steps toward something that seems like a new world to me!

[sinsi]

Post the non-working code, my mind reading skills are pretty poor

[Nate523]

Sorry for that!

Its a code to just step down and watch the registers change as I use the debugging feature:

Code Select Expand

INCLUDELIB kernel32.lib
ExitProcess PROTO
.data

var QWORD 100                            ; initialize variable mem

.code
main PROC

SUB RSP, 28H                            ; Byte align the top of the stack to 16 bytes

XOR RCX, RCX                            ; Clear registry
XOR RDX, RDX                            ; Clear registry
MOV RCX, 33                             ; Assign reg/imm
MOV RDX, RCX                            ; Assign reg/reg
MOV RCX, var                            ; Assign reg/mem
MOV var, RDX                            ; assign mem/reg

MOV RCX, 78                             ; Did it exit afterwards?

call ExitProcess
main ENDP
END

When I step down in the debugger of my work computer I can see everything transpiring between var, RCX, and RDX, but as soon as I hit call ExitProcess it throws an exception. I will note, that the same exact program does not throw the exception on my personal home computer, it exits as expected with no errors. Perhaps my work computer has a rights issue when 'ExitProcess' is called? The program seems to execute as expected and return an exit code 78 as expected, just throws an exception at the end :

Exception thrown at 0x00007FF691751039 in TestASM.exe: 0xC0000005: Access violation writing location 0x00000000542FF947.

[zedd151]

asm file

Code Select Expand

include \masm64\include64\masm64rt.inc

.data

var QWORD 100                            ; initialize variable mem

.code
main PROC

SUB RSP, 28H                            ; Byte align the top of the stack to 16 bytes

XOR RCX, RCX                            ; Clear registry
XOR RDX, RDX                            ; Clear registry
MOV RCX, 33                             ; Assign reg/imm
MOV RDX, RCX                            ; Assign reg/reg
MOV RCX, var                            ; Assign reg/mem
MOV var, RDX                            ; assign mem/reg

MOV RCX, 78                             ; Did it exit afterwards?

call ExitProcess
main ENDP
END
batch file to build it with

Code Select Expand

@echo off

set appname=test2

if exist %appname%.obj del %appname%.obj
if exist %appname%.exe del %appname%.exe

\masm64\bin64\ml64.exe /c /nologo %appname%.asm

\masm64\bin64\link.exe /SUBSYSTEM:WINDOWS /ENTRY:main /nologo %appname%.obj

dir %appname%.*

pauseran in x64dbg debugger, no issues while running from the debugger, or while stepping the code in the debugger.
Something wrong on your end, it seems.

[sinsi]

Yeah no problem here building with VS.

[sinsi]

Interestingly the fault address (ending in 1039) is the address of the CALL ExitProcess line.
Could you go to Project>Properties and copy the ML64 and LINK command lines?

[NoCforMe]

One thing, and pardon my ignorance of things 64-bit, but is this right in a 64-bit program?

Code Select Expand

INCLUDELIB kernel32.lib

[Nate523]

I went to Project->Properties->Microsoft Macro Assembler->Command Line:

Code Select Expand

ml64.exe /c /nologo /Zi /Fo"TestASM\x64\Debug\%(FileName).obj" /W3 /errorReport:prompt  /Ta

And in ...->Linker->Command Line:

Code Select Expand

/OUT:"C:\C++\TestASM\x64\Debug\TestASM.exe" /MANIFEST /NXCOMPAT /PDB:"C:\C++\TestASM\x64\Debug\TestASM.pdb" /DYNAMICBASE "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "comdlg32.lib" "advapi32.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "odbc32.lib" "odbccp32.lib" /DEBUG /MACHINE:X64 /ENTRY:"main" /INCREMENTAL /PGD:"C:\C++\TestASM\x64\Debug\TestASM.pgd" /SUBSYSTEM:CONSOLE /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /ManifestFile:"TestASM\x64\Debug\TestASM.exe.intermediate.manifest" /LTCGOUT:"TestASM\x64\Debug\TestASM.iobj" /ERRORREPORT:PROMPT /ILK:"TestASM\x64\Debug\TestASM.ilk" /NOLOGO /TLBID:1


[sinsi]

Apart from file paths/names they're my command lines too.
Can you zip the exe and pdb (or better still the whole solution) and attach it here?

[Nate523]

Attached the whole project folder.

[sinsi]

No problems running the exe you built and the exe the project built.

Exception thrown at 0x00007FF6A28C1031 in TestASM.exe: 0xC0000005: Access violation writing location 0xFFFFFFFFFFFFFFF8
Exception thrown at 0x00007FF691751039 in TestASM.exe: 0xC0000005: Access violation writing location 0x00000000542FF947.

Interesting the differences. Did you change any code?
Also why would ExitProcess write to memory?

One thing to try, add two lines, but I don't see how it will fix the problem

Code Select Expand

call ExitProcess
add rsp,28h
ret
main ENDP