Hi guys
i need a bit help on virtual alloc functions. I´m looking for a routine or library that can be able to handle huge virtual memory, because the ones i´m implementing from RosAsm are crashing badly due to fragmented memory blocks, i guess.
The problem is for the newer pdbdumper i´m making. When it loads a file with let´s say 6 Mb, the virtualloc is ok even if i double the size (or mul by 4) with:
mov eax D@FileSize | shl eax 2 | Align_On MEM_ALIGNMENT eax
call 'RosMem.VMemAlloc' D@pOutput, eax
This is ok, since internally, VMemAlloc uses VirtualAlloc to reserve and commit the necessary pages.
But...when i load a file pdb with 22 Mb, the app simply crashes, because VMemAlloc allocated the necessary size, but as far i saw, the allocated memory is not entirely commited, causing the parser for pdb crash when it tries to access a certain address in virtualmemory.
If needed, i can post here, the functions used internally in VMemAlloc, but i would like to know 1st, if it do exists a library or routines that can handle the necessary memory for huge apps (Allocate, free and reallocate huge memory blocks). Something like what hexworkshop does, i mean, you can open a file with whatever size and it will still load it without crash.
Btw, i don´t need a growable memory. All i was thinking in something simple. I mean, allocate XXX Mb of memory at once and use it according and later free it when it is no longer needed.
The virtual alloc is a bad choice to use memory.
global or heap memory are surely a good soluce.
Tks...but, is there a example of using heap memory to allocate huge files, without having to make it growable ? I mean, simply 1 function at the start of the app like "createheap" or allocateheapmem (or something..) from where i can allocate enough memory to handle the whole file, and then, once i finish to use the memory, free it.
I don´t use heapmemory very often, so i´m not really use to it. If there is already a function/routines that can handle huge files (With heapalloc/heapcreate etc), it would be better then doing it from scratch.
No problem with this one, allocating 20 x 200 MB with HeapAlloc (and writing to it...):
include \masm32\MasmBasic\MasmBasic.inc ; download (http://masm32.com/board/index.php?topic=94.0)
Init
msize=200000000
invoke SetLastError, 0
REPEAT 20
Let esi=New$(msize) ; uses HeapAlloc
PrintLine "Last error = ", Err$()
mov edi, esi
mov al, "x"
mov ecx, msize
rep stosb
ENDM
Inkey "ok?"
Exit
end start
Even after a REPEAT 200, it just shows me "ok?".
Many tks JJ. I´ll give a try.
I should have masmbasic here in another HD to test.
Btw....to free the memory i presume you did something like
Delete$(mem)
right?
Quote from: guga on August 02, 2014, 07:06:09 AM
Btw....to free the memory i presume you did something like
Delete$(mem)
right?
The Let esi=... takes care of that inside the loop (Let (http://www.webalice.it/jj2006/MasmBasicQuickReference.htm#Mb1132) checks in a table whether esi was already allocated, and if yes, deletes the previous one). One might use Clr$ (http://www.webalice.it/jj2006/MasmBasicQuickReference.htm#Mb1150) esi after the ENDM, although Exit aka ExitProcess takes care of freeing everything.
Other example:
include \masm32\MasmBasic\MasmBasic.inc ; download (http://masm32.com/board/index.php?topic=94.0)
Init
xor ecx, ecx
; xor esi, esi not needed, Init zeroes esi edi ebx
.Repeat
Let esi=Str$(ecx)+" "+esi+" "+Chr$(ecx+"a")
inc ecx
.Until ecx>9
PrintLine esi
Clr$ esi
Inkey "[", esi, "]"
Exit
end start
Output:9 8 7 6 5 4 3 2 1 0 a b c d e f g h i j
[]
My point was to show that HeapAlloc works, even for very large sizes. Your problem seems to be elsewhere...
JJ many many thanks :)
You gave an increadible and simple solution. the heapalloc function i already made on rosmem.dll, but i forgot where i was using it. After testing the heapalloc/heapfree, pdbdumper could completelly load a 22Mb file without crashing.
I´ll use heapalloc for this app and wil try to see why Virtualmem is crashing when loading huge files. If i couldn´t find a viable solution, i´ll recreate the memory allocation routines on rosasm (and rosmem.dll) only with heapalloc instead virtualalloc.
It seems pretty stable.
For VirtualAlloc 22MB should be a very small allocation.
;===============================================================================
include \masm32\include\masm32rt.inc
.686
.xmm
;===============================================================================
.data
sysinfo SYSTEM_INFO <>
.code
;===============================================================================
start:
;===============================================================================
invoke GetSystemInfo, ADDR sysinfo
mov ebx, sysinfo.dwPageSize
printf("Page Size : %d\n", ebx)
@@:
printf("Allocation Size : %u\n", ebx)
invoke VirtualAlloc, NULL, ebx, MEM_COMMIT or MEM_RESERVE, PAGE_READWRITE
mov esi, eax
.IF esi == 0
printf("%s\n", LastError$())
jmp @F
.ENDIF
mov eax, 0ffh
mov edi, esi
mov ecx, ebx
rep stosb
invoke VirtualFree, esi, 0, MEM_RELEASE
shl ebx, 1
jmp @B
@@:
inkey
exit
end start
Result on a system with 8GB of physical memory:
Page Size : 4096
Allocation Size : 4096
Allocation Size : 8192
Allocation Size : 16384
Allocation Size : 32768
Allocation Size : 65536
Allocation Size : 131072
Allocation Size : 262144
Allocation Size : 524288
Allocation Size : 1048576
Allocation Size : 2097152
Allocation Size : 4194304
Allocation Size : 8388608
Allocation Size : 16777216
Allocation Size : 33554432
Allocation Size : 67108864
Allocation Size : 134217728
Allocation Size : 268435456
Allocation Size : 536870912
Allocation Size : 1073741824
Allocation Size : 2147483648
Not enough storage is available to process this command.
Hi Michael
indeed...I tested it again, and after a reboot, the crash stopped. I don´t understand what was happening.
sounds like a memory leak of some kind - commonly with GDI objects :P