.MODEL flat

[CCurl]

I was able to get something to build and run without installing MASM.

In this, VirtualAlloc returns EAX=0, so of course VirtualProtect will also not be successful.

Code: [Select]

; TEST program

; ---------------------------------------------------------------------------------------------------------
.686P
.model flat, stdcall
.stack 4096

; ---------------------------------------------------------------------------------------------------------
; Constants

memorySize EQU 1024*8

PAGE_READWRITE                  EQU 0004h
PAGE_EXECUTE                       EQU 0010h
PAGE_EXECUTE_READWRITE EQU 0040h

MEM_COMMIT                         EQU 010000h

; ---------------------------------------------------------------------------------------------------------
.data

hHeap DWORD ?
theMem DWORD ?

; ---------------------------------------------------------------------------------------------------------
.code

ExitProcess PROTO, dwExitCode:dword

GetProcessHeap PROTO    ; Get the current process heap handle

HeapAlloc PROTO,
hHeap:DWORD, ; handle to private heap block
dwFlags:DWORD, ; heap allocation control flags
dwBytes:DWORD ; number of bytes to allocate

HeapFree PROTO,
hHeap:DWORD, ; handle to heap with memory block
dwFlags:DWORD, ; heap free options
lpMem:DWORD ; pointer to block to be freed

VirtualProtect PROTO,
pMem:DWORD, ; pointer to the memory
dwSize:DWORD, ; size of area to protect
fNewProtect:DWORD, ; new protection
pOldProtect:DWORD ; pointer to storage for old protection value

VirtualAlloc PROTO,
pMem:DWORD, ; optional, NULL lets the system figure it out
dwSize:DWORD, ; requested size
fAllocType:DWORD, ; allocation type
fProtect:DWORD ; protection flags


; ---------------------------------------------------------------------------------------------------------
main proc

; call GetProcessHeap
; mov hHeap, eax
; invoke HeapAlloc, hHeap, 0, memorySize
; invoke HeapFree, hHeap, 0, theMem

invoke VirtualAlloc, 0, memorySize, MEM_COMMIT, PAGE_READWRITE
mov theMem, eax
invoke VirtualProtect, theMem, memorySize, PAGE_EXECUTE, 0

invoke ExitProcess,0

main endp
end main

; ---------------------------------------------------------------------------------------------------------


[dedndave]

when you set the attribute for VirtualProtect, use PAGE_EXECUTE_READWRITE so that you can write the code in there

Code: [Select]

PAGE_EXECUTE_READWRITE EQU 40h
VirtualAlloc will probably handle that attribute constant, so that you don't have to use VirtualProtect

also - we typically list PROTO's first in the source - they don't need to be in an opened section

[CCurl]

New ASM file ... do you get a value back in EAX from your VirtualAlloc call? Mine comes back with ZERO.

Code: [Select]

; TEST program

; ---------------------------------------------------------------------------------------------------------
.686P
.model flat, stdcall
.stack 4096

; ---------------------------------------------------------------------------------------------------------
ExitProcess PROTO,
dwExitCode:DWORD

GetProcessHeap PROTO    ; Get the current process heap handle

HeapAlloc PROTO,
hHeap:DWORD, ; handle to private heap block
dwFlags:DWORD, ; heap allocation control flags
dwBytes:DWORD ; number of bytes to allocate

HeapFree PROTO,
hHeap:DWORD, ; handle to heap with memory block
dwFlags:DWORD, ; heap free options
lpMem:DWORD ; pointer to block to be freed

VirtualProtect PROTO,
pMem:DWORD, ; pointer to the memory
dwSize:DWORD, ; size of area to protect
fNewProtect:DWORD, ; new protection
pOldProtect:DWORD ; pointer to storage for old protection value

VirtualAlloc PROTO,
pMem:DWORD, ; optional, NULL lets the system figure it out
dwSize:DWORD, ; requested size
fAllocType:DWORD, ; allocation type
fProtect:DWORD ; protection flags

; ---------------------------------------------------------------------------------------------------------
; Constants

memorySize EQU 1024*8

PAGE_READWRITE EQU 0004h
PAGE_EXECUTE EQU 0010h
PAGE_EXECUTE_READWRITE EQU 0040h

MEM_COMMIT EQU 010000h

; ---------------------------------------------------------------------------------------------------------
.data

hHeap DWORD ?
theMem DWORD ?

; ---------------------------------------------------------------------------------------------------------
.code

main proc

; call GetProcessHeap
; mov hHeap, eax
; invoke HeapAlloc, hHeap, 0, memorySize
; invoke HeapFree, hHeap, 0, theMem

invoke VirtualAlloc, 0, memorySize, MEM_COMMIT, PAGE_EXECUTE_READWRITE
mov theMem, eax
invoke VirtualProtect, theMem, memorySize, PAGE_EXECUTE_READWRITE, 0

invoke ExitProcess,0

main endp
end main

; ---------------------------------------------------------------------------------------------------------


[dedndave]

VirtualAlloc returns the address of the allocated block
if it returns 0, it means there is an error; call GetLastError to get the error code

https://msdn.microsoft.com/en-us/library/windows/desktop/aa366887%28v=vs.85%29.aspx

[dedndave]

To reserve and commit pages in one step, call VirtualAlloc with MEM_COMMIT | MEM_RESERVE.

Attempting to commit a specific address range by specifying MEM_COMMIT without MEM_RESERVE and a non-NULL lpAddress fails unless the entire range has already been reserved. The resulting error code is ERROR_INVALID_ADDRESS.

in ASM, we use the OR operator
MEM_COMMIT or MEM_RESERVE

[dedndave]

by the way, 8KB allocation is very small
you could put that much (and more) in the .DATA? section, easily

[CCurl]

Whoops ... one too many ZEROs on the MEM_COMMIT constant. Now I get a non-ZERO number back from VirtualAlloc.

Yeah, 8K is just for the test, trying 8MB now.

[CCurl]

It WORKS!!!

Code: [Select]

; TEST program

; ---------------------------------------------------------------------------------------------------------
.686P
.model flat, stdcall
.stack 4096

; ---------------------------------------------------------------------------------------------------------
ExitProcess PROTO,
dwExitCode:DWORD

GetLastError PROTO

VirtualAlloc PROTO,
pMem:DWORD, ; optional, NULL lets the system figure it out
dwSize:DWORD, ; requested size
fAllocType:DWORD, ; allocation type
fProtect:DWORD ; protection flags

; ---------------------------------------------------------------------------------------------------------
; Constants

memorySize EQU 1024*1024*8

PAGE_READWRITE EQU 0004h
PAGE_EXECUTE EQU 0010h
PAGE_EXECUTE_READWRITE EQU 0040h

MEM_COMMIT EQU 01000h
MEM_RESERVE EQU 02000h
; ---------------------------------------------------------------------------------------------------------
.data

hHeap DWORD ?
theMem DWORD ?

; ---------------------------------------------------------------------------------------------------------
.code

main proc

invoke VirtualAlloc, 0, memorySize, MEM_COMMIT or MEM_RESERVE, PAGE_EXECUTE_READWRITE
mov theMem, eax
test eax, eax
jnz doTest
call GetLastError
jmp allDone

doTest:
mov ebp, theMem
inc ebp
mov byte ptr [ebp], 00bah
mov DWORD ptr [ebp][1], 11223344h
mov byte ptr [ebp][5], 00c3h
call ebp
; EDX should get 11223344h in it

allDone:
invoke ExitProcess,0
ret

main endp
end main

; ---------------------------------------------------------------------------------------------------------