A comparison of the speed of HeapAlloc/HeapFree vs HeapReAlloc (full code attached, plain Masm32).
First, the alloc/free loop:
push [esi+4*ebx] ; old pointer
push rv(nrandom, 1000) ; get a random number
mov [esi+4*ebx], rv(HeapAlloc, procHeap, HEAP_GENERATE_EXCEPTIONS, eax)
pop edx
invoke lstrcpyn, [esi+4*ebx], offset somestring, edx ; fill the string
pop ecx
jecxz @F
invoke HeapFree, procHeap, 0, ecx
@@: inc ebx
Second, the realloc loop: push rv(nrandom, 1000) ; get a random number
mov [esi+4*ebx], rv(HeapReAlloc, procHeap, HEAP_GENERATE_EXCEPTIONS, [esi+4*ebx], eax)
pop edx
invoke lstrcpyn, [esi+4*ebx], offset somestring, edx ; fill the string
inc ebx
alloc#
+94 ms
+468 ms
+826 ms
+390 ms
+624 ms
+297 ms
+468 ms
+437 ms
+406 ms
+265 ms
realloc
+94 ms
+78 ms
+78 ms
+78 ms
+78 ms
+78 ms
+78 ms
+78 ms
+78 ms
+78 ms
ok?
alloc
+47 ms
+47 ms
+47 ms
+47 ms
+31 ms
+31 ms
+47 ms
+31 ms
+47 ms
+32 ms
realloc
+46 ms
+32 ms
+47 ms
+31 ms
+47 ms
+47 ms
+31 ms
+47 ms
+31 ms
+47 ms
ok?
Interesting, Marinus. Which Windows version? My results above are Win7-64, but testing it now on Win10 (and a much slower machine) delivers something different; so it seems M$ has improved HeapAlloc:alloc
+125 ms
+109 ms
+109 ms
+94 ms
+94 ms
+94 ms
+93 ms
+110 ms
+94 ms
+93 ms
realloc
+110 ms
+109 ms
+156 ms
+141 ms
+125 ms
+125 ms
+109 ms
+110 ms
+125 ms
+109 ms
Hi Jochen,
Win8.1 - 64
Hi,
In honor of April 1st, here are some results.
Redirect to file, P-III, W2k
alloc
+311 ms
+751 ms
+300 ms
+300 ms
+301 ms
+561 ms
+300 ms
+301 ms
+600 ms
+301 ms
realloc
+691 ms
+360 ms
+641 ms
+341 ms
+651 ms
+340 ms
+661 ms
+341 ms
+651 ms
+340 ms
ok?
Screen capture
F:\TEMP\TEST>heaptest
alloc
+321 ms
+310 ms
+301 ms
+300 ms
+301 ms
+300 ms
+300 ms
+301 ms
+310 ms
+301 ms
realloc
+380 ms
+361 ms
+350 ms
+341 ms
+340 ms
+341 ms
+340 ms
+341 ms
+340 ms
+341 ms
ok?
Screen capture P-II MMX, W98
alloc
+731 ms
+1015 ms
+901 ms
+959 ms
+937 ms
+842 ms
+880 ms
+880 ms
+875 ms
+900 ms
realloc
+5747 ms
+16980 ms
+22631 ms
+24967 ms
+26686 ms
+27869 ms
+28271 ms
+28124 ms
+28596 ms
+28670 ms
ok?
Screen capture, Pentium M, WinXP
A:\>heaptest
alloc
+200 ms
+140 ms
+190 ms
+181 ms
+190 ms
+190 ms
+180 ms
+191 ms
+190 ms
+190 ms
realloc
+211 ms
+250 ms
+210 ms
+211 ms
+200 ms
+200 ms
+211 ms
+210 ms
+210 ms
+210 ms
ok?
Cheers,
Steve
W7-64bits
alloc
+63 ms
+62 ms
+63 ms
+62 ms
+62 ms
+63 ms
+62 ms
+47 ms
+78 ms
+62 ms
realloc
+63 ms
+62 ms
+78 ms
+63 ms
+62 ms
+62 ms
+78 ms
+63 ms
+62 ms
+63 ms
ok?
These results are pretty confusing :(
Thanks, folks :icon14:
Hi,
win10-64
alloc
+31 ms
+47 ms
+31 ms
+31 ms
+31 ms
+32 ms
+31 ms
+31 ms
+31 ms
+47 ms
realloc
+32 ms
+31 ms
+47 ms
+31 ms
+31 ms
+47 ms
+31 ms
+32 ms
+31 ms
+31 ms
ok?
Here it is Jochen
alloc
+438 ms
+219 ms
+187 ms
+203 ms
+157 ms
+171 ms
+141 ms
+203 ms
+187 ms
+157 ms
realloc
+156 ms
+187 ms
+172 ms
+172 ms
+188 ms
+265 ms
+266 ms
+187 ms
+250 ms
+203 ms
ok?
Thanks, Rui & LiaoMi. I start wondering if something is wrong with my Core i5/Win7 machine ::)
This was a LAB class thread so I moved it here. :biggrin:
Well done, Hutch. I thought it would be a Campus demo showing that reallocating is more efficient, but it turns out that it is pretty unclear what's happening there.
Theory would suggest that allocating a new buffer, then deleting the old one would be much slower; simply because with HeapReAlloc, the OS can often reuse the existing buffer, either because the new string is shorter, or because there were some unused bytes at the tail.
But most posts here show that actually there is no speed difference. Except on my machine :(
win 10 64 home:
alloc
+47 ms
+31 ms
+31 ms
+31 ms
+32 ms
+47 ms
+31 ms
+31 ms
+31 ms
+32 ms
realloc
+31 ms
+31 ms
+31 ms
+32 ms
+46 ms
+32 ms
+31 ms
+31 ms
+31 ms
+32 ms
ok?
alloc
+47 ms
+31 ms
+47 ms
+31 ms
+32 ms
+47 ms
+31 ms
+31 ms
+47 ms
+31 ms
realloc
+47 ms
+31 ms
+47 ms
+47 ms
+31 ms
+47 ms
+31 ms
+47 ms
+32 ms
+46 ms
ok?
Win 8.1 64 bits. I5. 8gb of ram.
alloc
+62 ms
+47 ms
+47 ms
+46 ms
+47 ms
+47 ms
+62 ms
+47 ms
+47 ms
+47 ms
realloc
+62 ms
+63 ms
+46 ms
+63 ms
+62 ms
+47 ms
+62 ms
+63 ms
+47 ms
+93 ms
ok?
Windows 7 x64, Pentium Dual Core E5700 3.0GHz, 4gb of RAM
This is really weird. I attach a new version that calculates a) the ratio of timings and b) the reuse rate in the HeapReAllocate case. For short strings (<=8 bytes), reuse is 100%, for <=32 bytes it's still 84%.
Results for Core i5 on Win7-64:alloc --------------------------------------
16 ms 234 ms 62 ms 31 ms 47 ms 94 ms 31 ms 62 ms 63 ms 46 ms 686 ms total
realloc
16 ms 16 ms 15 ms 0 ms 16 ms 15 ms 16 ms 16 ms 15 ms 16 ms
reuse: 60% - 141 ms total
alloc : realloc = 4.87 for len=640
alloc --------------------------------------
15 ms 468 ms 63 ms 93 ms 359 ms 63 ms 62 ms 78 ms 109 ms 265 ms 1575 ms total
realloc
16 ms 15 ms 16 ms 15 ms 16 ms 16 ms 15 ms 16 ms 31 ms 16 ms
reuse: 59% - 172 ms total
alloc : realloc = 9.16 for len=1280
alloc --------------------------------------
46 ms 640 ms 156 ms 328 ms 171 ms 156 ms 234 ms 187 ms 188 ms 171 ms 2277 ms total
realloc
203 ms 468 ms 468 ms 468 ms 452 ms 453 ms 452 ms 468 ms 468 ms 468 ms
reuse: 59% - 4368 ms total
alloc : realloc = 0.52 for len=2560
alloc --------------------------------------
406 ms 405 ms 297 ms 218 ms 265 ms 188 ms 202 ms 188 ms 187 ms 140 ms 2496 ms total
realloc
749 ms 1560 ms 1716 ms 1653 ms 1654 ms 1732 ms 1747 ms 1763 ms 1731 ms 1732 ms
reuse: 59% - 16037 ms total
alloc : realloc = 0.16 for len=5120
Note that the speed advantage for realloc rises to a factor 9 until string len rand(1280), then it drops dramatically, i.e. realloc becomes much slower than alloc ::)
I7 Win8.1 64
alloc --------------------------------------
0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 15 ms total
realloc
0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 16 ms
reuse: 0% - 31 ms total
alloc : realloc = 0.48 for len=80
alloc --------------------------------------
0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 16 ms total
realloc
0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 0 ms
reuse: 0% - 15 ms total
alloc : realloc = 1.07 for len=160
alloc --------------------------------------
16 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 32 ms total
realloc
0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms
reuse: 0% - 31 ms total
alloc : realloc = 1.03 for len=320
alloc --------------------------------------
15 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 15 ms total
realloc
0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms
reuse: 0% - 31 ms total
alloc : realloc = 0.48 for len=640
alloc --------------------------------------
0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 32 ms total
realloc
0 ms 15 ms 0 ms 0 ms 16 ms 0 ms 0 ms 15 ms 0 ms 0 ms
reuse: 0% - 46 ms total
alloc : realloc = 0.70 for len=1280
alloc --------------------------------------
16 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 15 ms 47 ms total
realloc
0 ms 16 ms 0 ms 16 ms 0 ms 15 ms 0 ms 16 ms 0 ms 0 ms
reuse: 0% - 63 ms total
alloc : realloc = 0.75 for len=2560
alloc --------------------------------------
16 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 47 ms total
realloc
0 ms 16 ms 16 ms 15 ms 0 ms 16 ms 15 ms 16 ms 0 ms 16 ms
reuse: 0% - 110 ms total
alloc : realloc = 0.43 for len=5120
ok?
i7-4gen win10 64
alloc --------------------------------------
0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 0 ms 15 ms total
realloc
0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 0 ms 16 ms
reuse: 0% - 32 ms total
alloc : realloc = 80
alloc --------------------------------------
0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 15 ms 15 ms total
realloc
0 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms
reuse: 0% - 16 ms total
alloc : realloc = 160
alloc --------------------------------------
0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 15 ms total
realloc
0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms
reuse: 0% - 32 ms total
alloc : realloc = 320
alloc --------------------------------------
0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 15 ms total
realloc
16 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 15 ms 0 ms
reuse: 0% - 47 ms total
alloc : realloc = 640
alloc --------------------------------------
0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms total
realloc
15 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 16 ms 0 ms 15 ms
reuse: 0% - 62 ms total
alloc : realloc = 1280
alloc --------------------------------------
0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 0 ms 16 ms total
realloc
16 ms 0 ms 15 ms 0 ms 0 ms 16 ms 0 ms 15 ms 0 ms 16 ms
reuse: 0% - 78 ms total
alloc : realloc = 2560
alloc --------------------------------------
0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 31 ms total
realloc
16 ms 0 ms 16 ms 0 ms 15 ms 16 ms 0 ms 15 ms 16 ms 0 ms
reuse: 0% - 94 ms total
alloc : realloc = 5120
ok?
0.47 for len=0.94 for len=0.47 for len=0.32 for len=0.00 for len=0.21 for len=0.33 for len=
JJ, how much ram do you have? :idea:
first incarnation of heaptest under linux mint 18 .. wine .. AMD 1 GHz dual core
alloc ...approx 130 ... realloc ... approx 1500
regards mikeb
alloc --------------------------------------
0 ms 0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 15 ms total
realloc
0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms
reuse: 0% - 32 ms total
alloc : realloc = 0.47 for len=80
alloc --------------------------------------
0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 0 ms 15 ms total
realloc
0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms 0 ms
reuse: 0% - 31 ms total
alloc : realloc = 0.48 for len=160
alloc --------------------------------------
0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 32 ms total
realloc
0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms
reuse: 0% - 15 ms total
alloc : realloc = 2.13 for len=320
alloc --------------------------------------
16 ms 0 ms 0 ms 0 ms 0 ms 15 ms 0 ms 0 ms 0 ms 0 ms 31 ms total
realloc
16 ms 0 ms 0 ms 15 ms 0 ms 16 ms 0 ms 0 ms 16 ms 0 ms
reuse: 0% - 63 ms total
alloc : realloc = 0.49 for len=640
alloc --------------------------------------
15 ms 0 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 47 ms total
realloc
0 ms 15 ms 0 ms 16 ms 0 ms 15 ms 0 ms 16 ms 0 ms 16 ms
reuse: 0% - 78 ms total
alloc : realloc = 0.60 for len=1280
alloc --------------------------------------
15 ms 0 ms 16 ms 0 ms 0 ms 0 ms 0 ms 16 ms 0 ms 0 ms 47 ms total
realloc
15 ms 16 ms 0 ms 15 ms 16 ms 0 ms 16 ms 15 ms 0 ms 16 ms
reuse: 0% - 109 ms total
alloc : realloc = 0.43 for len=2560
alloc --------------------------------------
16 ms 15 ms 16 ms 0 ms 0 ms 0 ms 0 ms 0 ms 0 ms 16 ms 63 ms total
realloc
16 ms 15 ms 16 ms 16 ms 15 ms 16 ms 15 ms 32 ms 15 ms 16 ms
reuse: 0% - 172 ms total
alloc : realloc = 0.37 for len=5120
ok?
Same machine as before.
My results for Win10 on a slow machine. I really don't know how to interpret these results: damn slow on Win7, no reuse of heap memory on Win10, pretty fast on XP in a virtual machine ::)
Win10, Celeron cpu:
alloc --------------------------------------
0 ms 16 ms 0 ms 0 ms 16 ms 0 ms 15 ms 16 ms 0 ms 16 ms 79 ms total
realloc
15 ms 16 ms 0 ms 15 ms 16 ms 16 ms 0 ms 15 ms 0 ms 16 ms
reuse: 0% - 109 ms total
alloc : realloc = 0.72 for len=80
alloc --------------------------------------
0 ms 15 ms 0 ms 16 ms 0 ms 16 ms 0 ms 15 ms 0 ms 16 ms 78 ms total
realloc
0 ms 16 ms 15 ms 0 ms 16 ms 32 ms 15 ms 0 ms 16 ms 0 ms
reuse: 0% - 110 ms total
alloc : realloc = 0.71 for len=160
alloc --------------------------------------
15 ms 0 ms 16 ms 0 ms 15 ms 0 ms 16 ms 0 ms 0 ms 15 ms 77 ms total
realloc
0 ms 31 ms 0 ms 15 ms 0 ms 16 ms 16 ms 16 ms 16 ms 15 ms
reuse: 0% - 125 ms total
alloc : realloc = 0.62 for len=320
alloc --------------------------------------
16 ms 15 ms 16 ms 16 ms 15 ms 16 ms 16 ms 0 ms 15 ms 16 ms 141 ms total
realloc
15 ms 32 ms 15 ms 16 ms 16 ms 15 ms 16 ms 15 ms 16 ms 0 ms
reuse: 0% - 156 ms total
alloc : realloc = 0.90 for len=640
alloc --------------------------------------
15 ms 16 ms 16 ms 15 ms 16 ms 0 ms 0 ms 16 ms 16 ms 15 ms 125 ms total
realloc
32 ms 31 ms 31 ms 16 ms 15 ms 32 ms 31 ms 15 ms 32 ms 15 ms
reuse: 0% - 250 ms total
alloc : realloc = 0.50 for len=1280
alloc --------------------------------------
32 ms 15 ms 16 ms 15 ms 16 ms 0 ms 16 ms 15 ms 16 ms 16 ms 157 ms total
realloc
15 ms 31 ms 32 ms 31 ms 31 ms 31 ms 32 ms 31 ms 16 ms 31 ms
reuse: 0% - 281 ms total
alloc : realloc = 0.56 for len=2560
alloc --------------------------------------
47 ms 63 ms 15 ms 0 ms 16 ms 15 ms 16 ms 16 ms 15 ms 16 ms 219 ms total
realloc
47 ms 47 ms 46 ms 47 ms 47 ms 47 ms 47 ms 47 ms 47 ms 47 ms
reuse: 0% - 469 ms total
alloc : realloc = 0.47 for len=5120
ok?WinXP VM with Core i5:
alloc --------------------------------------
10 ms 20 ms 20 ms 10 ms 10 ms 10 ms 10 ms 20 ms 10 ms 10 ms 130 ms total
realloc
20 ms 10 ms 10 ms 20 ms 10 ms 10 ms 10 ms 10 ms 10 ms 10 ms
reuse: 60% - 120 ms total
alloc : realloc = 1.08 for len=320
alloc --------------------------------------
20 ms 70 ms 70 ms 50 ms 40 ms 30 ms 50 ms 41 ms 40 ms 40 ms 451 ms total
realloc
20 ms 10 ms 20 ms 10 ms 20 ms 10 ms 10 ms 20 ms 10 ms 20 ms
reuse: 59% - 150 ms total
alloc : realloc = 3.01 for len=640
alloc --------------------------------------
30 ms 370 ms 101 ms 70 ms 100 ms 90 ms 50 ms 60 ms 60 ms 50 ms 981 ms total
realloc
20 ms 20 ms 20 ms 10 ms 20 ms 20 ms 20 ms 20 ms 11 ms 20 ms
reuse: 58% - 181 ms total
alloc : realloc = 5.42 for len=1280
alloc --------------------------------------
50 ms 500 ms 151 ms 90 ms 80 ms 80 ms 60 ms 70 ms 50 ms 60 ms 1191 ms total
realloc
30 ms 30 ms 20 ms 30 ms 20 ms 20 ms 20 ms 31 ms 20 ms 20 ms
reuse: 59% - 241 ms total
alloc : realloc = 4.94 for len=2560
alloc --------------------------------------
80 ms 90 ms 90 ms 80 ms 70 ms 50 ms 70 ms 50 ms 61 ms 50 ms 691 ms total
realloc
50 ms 30 ms 40 ms 40 ms 40 ms 30 ms 40 ms 40 ms 40 ms 40 ms
reuse: 59% - 390 ms total
alloc : realloc = 1.77 for len=5120Quote from: felipe on April 03, 2018, 04:02:13 AM
JJ, how much ram do you have? :idea:
8GB
Don't know how HeapAlloc and HeapReAlloc work internally...
Maybe windows does reserve much more memory, in case a realloc is needed, using some kind of priority system and giving the extra unused reserved memory back when needed?
Looking at the time intervals between allocations this could be plausible, just a thought.
Thanks, Marinus.
There is a thread at SOF: Reason for ~100x slowdown with heap memory functions using HEAP_NO_SERIALIZE (https://stackoverflow.com/questions/1983563/reason-for-100x-slowdown-with-heap-memory-functions-using-heap-no-serialize-on), and M$ writes about Low-fragmentation Heap (https://msdn.microsoft.com/en-us/library/aa366750%28VS.85%29.aspx)
Both are good reads, and almost unrelated to my problem, but the second one contains a phrase that I found interesting:
QuoteThe LFH also cannot be enabled if you are using the heap debugging tools in Debugging Tools for Windows
So I checked my source and found
OPT_Debug 1, which tells RichMasm to use... guess what... Debugging Tools for Windows. Mystery solved, the heap gets extra fast again when disabling that option :icon_redface:
I added a warning to the old Detecting and debugging heap corruption (http://masm32.com/board/index.php?topic=5314.msg67040#msg67040) thread.
Quote from: Siekmanski on April 03, 2018, 07:21:37 AMMaybe windows does reserve much more memory, in case a realloc is needed, using some kind of priority system and giving the extra unused reserved memory back when needed?
It used to work that way, see the reuse figures above, but one interesting result is that on Win10, there is no reuse of the same buffer with HeapReAlloc.
Thanks to everybody, and apologies that I wasted your time :icon14:
Quote from: jj2007 on April 03, 2018, 07:42:12 AM
apologies that I wasted your time :icon14:
What are you talking about? Just by doing some research we learn. i like to do some research and helping to do it too. 8)
+1