I am trying to speed up file searches but no luck so far:
AMD Athlon(tm) Dual Core Processor 4450B (MMX, SSE, SSE2, SSE3)
Windows 7 Professional
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
21369 µs for 100 loops, BasicInfo=1, LargeFetch=1
23.7.12 14:24:04 \Masm32\Examples\exampl01
20.2.12 18:15:47 \Masm32\Examples\enumerate
19708 µs for 100 loops, BasicInfo=0, LargeFetch=0
23.7.12 14:24:04 \Masm32\Examples\exampl01
20.2.12 18:15:47 \Masm32\Examples\enumerate
19834 µs for 100 loops, BasicInfo=1, LargeFetch=1
23.7.12 14:24:04 \Masm32\Examples\exampl01
20.2.12 18:15:47 \Masm32\Examples\enumerate
20297 µs for 100 loops, BasicInfo=0, LargeFetch=0
23.7.12 14:24:04 \Masm32\Examples\exampl01
20.2.12 18:15:47 \Masm32\Examples\enumerate
16150 µs for 100 loops, BasicInfo=1, LargeFetch=1
23.7.12 14:24:04 \Masm32\Examples\exampl01
20.2.12 18:15:47 \Masm32\Examples\enumerate
18467 µs for 100 loops, BasicInfo=0, LargeFetch=0
23.7.12 14:24:04 \Masm32\Examples\exampl01
20.2.12 18:15:47 \Masm32\Examples\enumerate
I've seen claims for miraculous improvements but my OS fails to honour these flags :(
Grateful for tests and ideas.
P.S.: I am aware of Lingo's stuff (http://www.masmforum.com/board/index.php?topic=16421.msg143124#msg143124) but it crashes. Probably too many GetProcAddress with ordinals ;)
Intel(R) Pentium(R) 4 CPU 3.00GHz (MMX, SSE, SSE2, SSE3)
Microsoft Windows XP
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
190 µs for 100 loops, BasicInfo=1, LargeFetch=1
3100 µs for 100 loops, BasicInfo=0, LargeFetch=0
251 µs for 100 loops, BasicInfo=1, LargeFetch=1
3045 µs for 100 loops, BasicInfo=0, LargeFetch=0
254 µs for 100 loops, BasicInfo=1, LargeFetch=1
3052 µs for 100 loops, BasicInfo=0, LargeFetch=0
Thanks, Dave.
This is really messy (MSDN FindFirstFileEx (http://msdn.microsoft.com/en-us/library/windows/desktop/aa364419%28v=vs.85%29.aspx)):
QuoteFIND_FIRST_EX_LARGE_FETCH .. and Windows XP: This value is not supported until Windows Server 2008 R2 and Windows 7."
You just demonstrated the contrary :bgrin:EDIT: No, that was just a bug in the first version.
Now why is it slow on my Win7-32 and -64 systems but fast on XP? ::)
To get the computer name, I added
TryRTE RegX2, con
Let edi=GetRegVal("HKLM\SYSTEM\CurrentControlSet\Control\ComputerName\ComputerName", "ComputerName")
RegX2:
Works fine on W7-32 but requires admin privileges on W7-64; however, when I run it as admin, it does no longer read the network drive... oh my dear friends in Redmond... :icon_mrgreen:
Jochen,
the results:
Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SS
E4.2, AVX)
Windows 7 Professional
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
1112 µs for 100 loops, BasicInfo=1, LargeFetch=1
01.01.1601 02:00:00
01.01.1601 02:00:00
804 µs for 100 loops, BasicInfo=0, LargeFetch=0
01.01.1601 02:00:00
01.01.1601 02:00:00
1042 µs for 100 loops, BasicInfo=1, LargeFetch=1
01.01.1601 02:00:00
01.01.1601 02:00:00
753 µs for 100 loops, BasicInfo=0, LargeFetch=0
01.01.1601 02:00:00
01.01.1601 02:00:00
1002 µs for 100 loops, BasicInfo=1, LargeFetch=1
01.01.1601 02:00:00
01.01.1601 02:00:00
919 µs for 100 loops, BasicInfo=0, LargeFetch=0
01.01.1601 02:00:00
01.01.1601 02:00:00
-- ok --
Gunther
Thanxalot, Dave and Gunther. In the meantime, I fixed the bug that made it so fast on XP :icon_mrgreen:
; perform a dummy search to find out if the OS supports the fast flags:
FindExInfoBasic=1
invoke FindFirstFileEx, chr$("C:\NoFFx.Yet"),
FindExInfoBasic, offset wfd, 0, 0, FIND_FIRST_EX_LARGE_FETCH
invoke GetLastError <<<<<<<< I had forgotten this one :redface:
.if eax!=ERROR_INVALID_PARAMETER
add dword ptr [esi], FindExInfoBasic ; ffxInfoLevel
add dword ptr [esi+4], FIND_FIRST_EX_LARGE_FETCH ; ffxAdditionalFlags
.endif
New version attached at top of thread.
@Gunther: It searches \Masm32\Examples\* - since you had no results, that was probably the wrong drive or puter. The new version complains with a MsgBox if it doesn't find anything.
XP results:
Intel(R) Celeron(R) M CPU 420 @ 1.60GHz (MMX, SSE, SSE2, SSE3)
Microsoft Windows XP
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
MSI, 21:47:41
253 ms for 1000 loops, BasicInfo=0, LargeFetch=0
02.03.2013 00:51:53 \Masm32\Examples\exampl01
03.11.2012 12:16:41 \Masm32\Examples\Bill_Cravener
288 ms for 1000 loops, BasicInfo=0, LargeFetch=0
02.03.2013 00:51:53 \Masm32\Examples\exampl01
03.11.2012 12:16:41 \Masm32\Examples\Bill_Cravener
130 ms for 1000 loops, BasicInfo=0, LargeFetch=0
02.03.2013 00:51:53 \Masm32\Examples\exampl01
03.11.2012 12:16:41 \Masm32\Examples\Bill_Cravener
127 ms for 1000 loops, BasicInfo=0, LargeFetch=0
02.03.2013 00:51:53 \Masm32\Examples\exampl01
03.11.2012 12:16:41 \Masm32\Examples\Bill_Cravener
Flags are never set, i.e. it fails on XP as documented.
Jochen,
here are the results from my virtual machine:
Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SS
E4.2, AVX)
Microsoft Windows XP
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
VIRTUALXP-36801, 23:12:39
128 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12.01.2013 22:03:43 \Masm32\Examples\timer_demos
12.01.2013 22:03:43 \Masm32\Examples\poasm
121 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12.01.2013 22:03:43 \Masm32\Examples\timer_demos
12.01.2013 22:03:43 \Masm32\Examples\poasm
55 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12.01.2013 22:03:43 \Masm32\Examples\timer_demos
12.01.2013 22:03:43 \Masm32\Examples\poasm
52 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12.01.2013 22:03:43 \Masm32\Examples\timer_demos
12.01.2013 22:03:43 \Masm32\Examples\poasm
53 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12.01.2013 22:03:43 \Masm32\Examples\timer_demos
12.01.2013 22:03:43 \Masm32\Examples\poasm
52 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12.01.2013 22:03:43 \Masm32\Examples\timer_demos
12.01.2013 22:03:43 \Masm32\Examples\poasm
...
889 bytes in LogJJ.txt created 23:12:39
-- ok --
Is that better?
Gunther
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
C:\Documents and Settings\Dave\Desktop\GetFilesEx => getfilesex
Intel(R) Pentium(R) 4 CPU 3.00GHz (MMX, SSE, SSE2, SSE3)
Microsoft Windows XP
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
K7NL, 14:45:42
258 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12/19/2012 8:25:23 AM \Masm32\Examples\static
12/10/2012 10:30:42 AM \Masm32\Examples\poasm
253 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12/19/2012 8:25:23 AM \Masm32\Examples\static
12/10/2012 10:30:42 AM \Masm32\Examples\poasm
121 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12/19/2012 8:25:23 AM \Masm32\Examples\static
12/10/2012 10:30:42 AM \Masm32\Examples\poasm
123 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12/19/2012 8:25:23 AM \Masm32\Examples\static
12/10/2012 10:30:42 AM \Masm32\Examples\poasm
119 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12/19/2012 8:25:23 AM \Masm32\Examples\static
12/10/2012 10:30:42 AM \Masm32\Examples\poasm
122 ms for 1000 loops, BasicInfo=0, LargeFetch=0
12/19/2012 8:25:23 AM \Masm32\Examples\static
12/10/2012 10:30:42 AM \Masm32\Examples\poasm
...
882 bytes in LogJJ.txt created 2:45:43 PM
Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz (MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1)
Microsoft Windows XP
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
21-7, 17:33:57
218 ms for 1000 loops, BasicInfo=0, LargeFetch=0
5/28/2013 5:08:33 PM \Masm32\Examples\timer_demos
5/28/2013 5:08:33 PM \Masm32\Examples\Bill_Cravener
124 ms for 1000 loops, BasicInfo=0, LargeFetch=0
5/28/2013 5:08:33 PM \Masm32\Examples\timer_demos
5/28/2013 5:08:33 PM \Masm32\Examples\Bill_Cravener
58 ms for 1000 loops, BasicInfo=0, LargeFetch=0
5/28/2013 5:08:33 PM \Masm32\Examples\timer_demos
5/28/2013 5:08:33 PM \Masm32\Examples\Bill_Cravener
59 ms for 1000 loops, BasicInfo=0, LargeFetch=0
5/28/2013 5:08:33 PM \Masm32\Examples\timer_demos
5/28/2013 5:08:33 PM \Masm32\Examples\Bill_Cravener
58 ms for 1000 loops, BasicInfo=0, LargeFetch=0
5/28/2013 5:08:33 PM \Masm32\Examples\timer_demos
5/28/2013 5:08:33 PM \Masm32\Examples\Bill_Cravener
59 ms for 1000 loops, BasicInfo=0, LargeFetch=0
5/28/2013 5:08:33 PM \Masm32\Examples\timer_demos
5/28/2013 5:08:33 PM \Masm32\Examples\Bill_Cravener
...
1876 bytes in LogJJ.txt created 5:33:57 PM
-- ok --
Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz (MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX)
Windows 8 Pro with Media Center
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
Line 12: Get/SetRegVal failed
, 13:23:33
59 ms for 1000 loops, BasicInfo=1, LargeFetch=2
19/12/2012 4:48:12 PM \Masm32\Examples\timer_demos
19/12/2012 4:48:12 PM \Masm32\Examples\poasm
45 ms for 1000 loops, BasicInfo=0, LargeFetch=0
19/12/2012 4:48:12 PM \Masm32\Examples\timer_demos
19/12/2012 4:48:12 PM \Masm32\Examples\poasm
51 ms for 1000 loops, BasicInfo=1, LargeFetch=2
19/12/2012 4:48:12 PM \Masm32\Examples\timer_demos
19/12/2012 4:48:12 PM \Masm32\Examples\poasm
37 ms for 1000 loops, BasicInfo=0, LargeFetch=0
19/12/2012 4:48:12 PM \Masm32\Examples\timer_demos
19/12/2012 4:48:12 PM \Masm32\Examples\poasm
36 ms for 1000 loops, BasicInfo=1, LargeFetch=2
19/12/2012 4:48:12 PM \Masm32\Examples\timer_demos
19/12/2012 4:48:12 PM \Masm32\Examples\poasm
34 ms for 1000 loops, BasicInfo=0, LargeFetch=0
19/12/2012 4:48:12 PM \Masm32\Examples\timer_demos
19/12/2012 4:48:12 PM \Masm32\Examples\poasm
Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (MMX, SSE, SSE2, SSE3, SSSE3)
Microsoft Windows XP
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
SIEKMANSKIPC, 08:25:58
147 ms for 1000 loops, BasicInfo=0, LargeFetch=0
24-12-2007 10:47:34 \Masm32\Examples\bcraven
24-12-2007 10:47:24 \Masm32\Examples\exampl11
146 ms for 1000 loops, BasicInfo=0, LargeFetch=0
24-12-2007 10:47:34 \Masm32\Examples\bcraven
24-12-2007 10:47:24 \Masm32\Examples\exampl11
74 ms for 1000 loops, BasicInfo=0, LargeFetch=0
24-12-2007 10:47:34 \Masm32\Examples\bcraven
24-12-2007 10:47:24 \Masm32\Examples\exampl11
74 ms for 1000 loops, BasicInfo=0, LargeFetch=0
24-12-2007 10:47:34 \Masm32\Examples\bcraven
24-12-2007 10:47:24 \Masm32\Examples\exampl11
75 ms for 1000 loops, BasicInfo=0, LargeFetch=0
24-12-2007 10:47:34 \Masm32\Examples\bcraven
24-12-2007 10:47:24 \Masm32\Examples\exampl11
74 ms for 1000 loops, BasicInfo=0, LargeFetch=0
24-12-2007 10:47:34 \Masm32\Examples\bcraven
24-12-2007 10:47:24 \Masm32\Examples\exampl11
...
880 bytes in LogJJ.txt created 8:25:59
-- ok --
I've done some more testing - see new attachment on top of thread, now allowing to drag a folder over the exe to get any folder tested.
The pattern that emerges is that for small folders FindFirstFileEx performs better without the two flags, but for large folders and network drives it can be some 20% or so faster with the flags set. Not on XP, though, this is Win7 only.
AMD Athlon(tm) Dual Core Processor 4450B (MMX, SSE, SSE2, SSE3)
Windows 7 Professional
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
Commandline=[C:\Masm32\RichMasm], testing [C:\Masm32\RichMasm\*]
...
last #folders=18
Total ms by flag setting, ignoring first 2 runs:
25692 ms for flags OFF
19988 ms for flags ON
Commandline=[?], testing [C:\Masm32\Examples\*]
...
last #folders=18
Total ms by flag setting, ignoring first 2 runs:
1068 ms for flags OFF
1476 ms for flags ON
Commandline=["C:\Program Files"], testing [C:\Program Files\*]
...
last #folders=51
Total ms by flag setting, ignoring first 2 runs:
1870 ms for flags OFF
1681 ms for flags ON
From a command prompt:
F:\>GetFilesEx.exe
Commandline=[?], testing [m:\Masm32\Examples\*]
However, it works OK double-clicking from Explorer.
Worked:
F:\>GetFilesEx.exe c:\
Commandline=[c:\], testing [c:\\*]
2 ms for 10 loops, BasicInfo=1, LargeFetch=2 29/05/2013 2:08:35 PM ..formation
1 ms for 10 loops, BasicInfo=0, LargeFetch=0 29/05/2013 2:08:35 PM ..formation
2 ms for 10 loops, BasicInfo=1, LargeFetch=2 29/05/2013 2:08:35 PM ..formation
1 ms for 10 loops, BasicInfo=0, LargeFetch=0 29/05/2013 2:08:35 PM ..formation
1 ms for 10 loops, BasicInfo=1, LargeFetch=2 29/05/2013 2:08:35 PM ..formation
1 ms for 10 loops, BasicInfo=0, LargeFetch=0 29/05/2013 2:08:35 PM ..formation
...
4220429 bytes in created 9:30:00 AM, last #folders=9
Total ms by flag setting, ignoring first 2 runs:
9 ms for flags OFF
7 ms for flags ON
Failed:
F:\>GetFilesEx.exe c:
Commandline=[c:], testing [m:\Masm32\Examples\*]
it tries to run on my G: drive, when masm32 is on my C: drive
i have an older version of masm32 on G:
Thanks, Sinsi & Dave. I attached a corrected version above.
(I had assumed the zero arg of the commandline was a full path to the exe, but it's just what you type in :()
my bad - i did not put a command line arg :redface:
now, it says no file for C:\masm32\examples, which is incorrect
Quote from: dedndave on May 29, 2013, 08:40:23 PM
now, it says no file for C:\masm32\examples, which is incorrect
sh.... what's wrong today ::)
Can you post the header. e.g.
Commandline=[?], testing [C:\Masm32\Examples\*] ?
Intel(R) Pentium(R) 4 CPU 3.00GHz (MMX, SSE, SSE2, SSE3)
Microsoft Windows XP
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
K7NL, 04:00:55
Commandline=[?], testing [C:\Masm32\Examples\*]
by the way, i have examples from different versions in the Masm32 folder
Examples_v7, Examples_v10, Examples_v11 :biggrin:
to run your test, i have renamed Examples_v11 to Examples
also, i have added some of my own examples - lol
(which i have temporarily moved elsewhere)
Jochen,
here the results from your latest version:
c:\scratch02>GetFilesEx.exe C:\Program Files
Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SS
E4.2, AVX)
Windows 7 Professional
Test FindExInfoBasic and FIND_FIRST_EX_LARGE_FETCH flags
Line 18: Get/SetRegVal failed , 15:31:09
Commandline=[C:\Program Files], testing [C:\Program Files\*]
112 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
79 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
120 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
80 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
97 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
78 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
116 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
78 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
99 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
103 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
100 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
102 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
99 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
103 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
102 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
102 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
99 ms for 1000 loops, BasicInfo=1, LargeFetch=2 25.05.2013 16:35:58 .. Explorer
102 ms for 1000 loops, BasicInfo=0, LargeFetch=0 25.05.2013 16:35:58 .. Explorer
...
1914 bytes in LogJJ.txt created 15:31:11, last #folders=27
Total ms by flag setting, ignoring first 2 runs:
668 ms for flags OFF
712 ms for flags ON
c:\scratch02>
Gunther
Where did u get those counting macros? that's michael w's right? i searched but no luck. anyone has link?
tyx
Quote from: xandaz on May 30, 2013, 08:27:57 AM
Where did u get those counting macros?
NanoTimer() is part of MasmBasic (http://masm32.com/board/index.php?topic=94.0). It's a standalone macro, you'll find it in \Masm32\MasmBasic\MasmBasic.inc (you can search the file for QueryPerformanceCounter).
ty jj