Inspired by fearless (https://masm32.com/board/index.php?topic=11371.0), I added a ZipFiles macro to the latest MasmBasic version (http://masm32.com/board/index.php?topic=94.0):
include \masm32\MasmBasic\MasmBasic.inc
Init
GfNoRecurse=1 ; current level only
GetFolders "\Masm32\examples\exampl01\qikpad"
AddFiles "\Masm32\examples\exampl01\qikpad\*.asm|*.inc|*.rc|*.ico|*.bmp"
ZipFiles "MyQikpad.zip"
ShEx "MyQikpad.zip" ; have a look at the result... it's ShellExecute under the hood
EndOfCode
ZipFiles takes exactly one argument: the name of the archive. Use GetFiles (https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1056) & friends (GetFolders, AddFiles) to define which files and folders are being added. As shown above, you can limit the files to be included by extension.
Note GfNoRecurse=1 is needed here because by default GetFolders and GetFiles include all folders/files below the start directory; and the built-in Windows has the bad habit to do the same, i.e. it zips sub-folders, too - so you would end up with duplicates, no good ;-)
Recall (https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1172) "MyFiles.lst", Files$() is another, much simpler option to define what to zip.
Demo source & exe attached (builds with MasmBasic version 27.10.2023 (http://masm32.com/board/index.php?topic=94.0)).
Nice.
Can we non-MasmBasic users get a peek behind the scenes at how exactly that mechanism works, or would that be proprietary info? As the person suggested, it would be nice to be able to programmatically zip up files.
Are you simply EXECing the zip program, or are you doing the zipping yourself? (I suspect the former.)
Hi Jochen,
I am not sure but does the ZipFiles macro require FreeArc? Thanks.
QuoteZipFiles ; requires a FreeArc installation
GetFiles \masm32\m32lib\*.asm ; fill the Files$() array with the desired files
ZipFiles "The Masm32 lib" ; the minimum: just the name of the *.arc archive
; with archive name, file count, show, Launch console mode:
ZipFiles "The_Masm32_lib", 10, SW_MINIMIZE, CREATE_NEW_CONSOLE ; use first 10 files and a minimised new console
ZipFiles "The_Masm32_lib", 0, SW_MINIMIZE, CREATE_NEW_CONSOLE ; use all files in Files$() and a minimised new console
Rem - requires FreeArc in its default location (e.g. C:\Program Files\FreeArc\bin\Arc.exe)
- creates archive in *.arc format
- return value: see Launch
- use MbZipLog = 1 to see the command lines in ZipLog.txt
https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1073
Quote from: NoCforMe on October 27, 2023, 11:45:05 AMNice.
Can we non-MasmBasic users get a peek behind the scenes at how exactly that mechanism works, or would that be proprietary info? As the person suggested, it would be nice to be able to programmatically zip up files.
See attachment.
QuoteAre you simply EXECing the zip program
No.
Quote from: Vortex on October 27, 2023, 07:57:46 PMI am not sure but does the ZipFiles macro require FreeArc?
Hi Erol,
I have not updated yet the online manual. Indeed, the FreeArc thing still exists, but I've renamed it ArcFiles (the first time ever that I rename a macro :biggrin: ).
Note that FreeArc is a much better archiver, but, well, it means that the end user must install another program. The new
ZipFiles "myfiles.zip" works
out of the box on every recent Windows version. I've just tested it in my WinXP VM, and it works - very, very slow but the archive is ok.
P.S.: Online manual updated (https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1073).
Another usage example:
include \masm32\MasmBasic\MasmBasic.inc
$Data \Masm64\Examples\Simple
$Data \Masm64\Examples\Advanced
Init
Read Files$()
ZipFiles "Masm64_examples.zip"
ShEx "Masm64_examples.zip" ; have a look at the result (it's ShellExecute)
EndOfCode
This one requires the text file Masm64_examples.lst with folder entries as above (see Recall (https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1172)):
include \masm32\MasmBasic\MasmBasic.inc
Init
Recall "Masm64_examples.lst", Files$()
ZipFiles "Masm64_examples.zip"
EndOfCode
It might be buried in one of the macros but it looks like you forgot to release this interface
invoke CoCreateInstance, addr CLSID_Shell, NULL, CLSCTX_INPROC_SERVER, addr IID_IShellDispatch, edi
Hi adeyblue,
Thanks for mentioning this - I am quite insecure how to handle this. So far I've done the following:
invoke CoCreateInstance, addr CLSID_Shell, NULL, CLSCTX_INPROC_SERVER, addr IID_IShellDispatch, edi ; addr pISD
.if !eax
...
invoke SysFreeString, vDest.bstrVal
CoInvoke pFolder, IShellDispatch.Release ; <<<<<< ok?
; deb 4, "folder released", eax
.endif
invoke CoUninitialize
mov eax, fileCtOk
ret
MbZip endp
The IShellDispatch.Release should be a Folder.Release, but that doesn't really matter because the code is identical. Still, I'll correct that in the source.
It seems to work fine, I can't see any leaked handles in Task Manager even if I call it a hundred times in a loop. Should it rather be like this?
CoInvoke pFolder, Folder.Release
CoInvoke pInterface, IShellDispatch.Release
Are both needed, or would the Folder.Release be enough, given that CoUnitialize follows suit? I've made a test with a hundred calls: there is clearly a leak when I forget the Folder.Release, but omitting IShellDispatch.Release has no consequences.
Quote from: adeyblue on October 28, 2023, 08:11:45 AMIt might be buried in one of the macros but it looks like you forgot to release this interface
invoke CoCreateInstance, addr CLSID_Shell, NULL, CLSCTX_INPROC_SERVER, addr IID_IShellDispatch, edi
Found this by guru Raymond Chen (https://devblogs.microsoft.com/oldnewthing/20040520-00/?p=39243):
QuoteYou need to take care even with local objects. Consider:
void Sample()
{
if (SUCCEEDED(CoInitialize(NULL))) {
CComPtr<IXMLDOMDocument> p;
if (SUCCEEDED(p.CoCreateInstance(CLSID_IXMLDOMDocument))) {
...
}
CoUninitialize();
}
}
Easy as pie. And there's a bug here.
When does the destructor for that smart-pointer run?
Answer: When the object goes out of scope
So the question is here, will the destructor be called in CoUninitialize?
Quote from: jj2007 on October 28, 2023, 09:41:19 AMI've made a test with a hundred calls: there is clearly a leak when I forget the Folder.Release, but omitting IShellDispatch.Release has no consequences
Then your test was incomplete, and it's pretty simple to prove so
Do
CoInit
CoCreate
Print pInterface
pInterface.Release
CoUnint
Loop
Do that 2000 times, pInterface is always the same address or one of a very few - so the same few blocks of memory are continuously recycled
Do
CoInit
CoCreate
Print pInterface
CoUnint
Loop
Do that 2000 times, pInterface is now always a different address.
If you're feeling particularly spicy, put a HeapWalk loop after those loops and count how many times it, er, loops. Put the Release loop first so you can count the difference.
Thanks for looking into this, adeyblue.
Quote from: adeyblue on October 31, 2023, 12:44:54 PMDo that 2000 times, pInterface is now always a different address.
It is, indeed. However, that is the case with or without the IShellDispatch.Release - here are the differences between the old and the new interface:
mov eax, pInterface
push eax
sub eax, DiffInterface
pop DiffInterface
fdeb 4, "out", eax
without release:
out eax 528
out eax 440
out eax 176
out eax 264
out eax 792
out eax 1320
out eax -1672
out eax 1760
out eax -2376
out eax 1496
out eax -1848
out eax 704
out eax 1408
out eax -1936
out eax 1408
out eax 1232
out eax -2992
out eax 1056with release:
out eax 528
out eax 440
out eax 176
out eax 264
out eax 792
out eax 1320
out eax -1672
out eax 1760
out eax -2376
out eax 1496
out eax -1848
out eax 704
out eax 1408
out eax -1936
out eax 1408
out eax 1232
out eax -2992
out eax 1056No errors in both cases, both pInterface.Release (if used) and CoUninitialize return S_OK, i.e. eax==0.
I tried it with a thousand iterations. Occasional big jumps but the overall pattern is somebody grabbing memory from the heap. The working set stays put at 10MB, with or without pInterface.Release
Note I
do release the folder interface in both runs; the difference is that in one of them I release pISD, i.e. the pointer to IShellDispatch, in the other run I don't. My best guess is that CoUninitialize does decrement the counter and releases it automagically. What is your explanation? You seem to know much more about COM than I do...
Perhaps only LCOM gives same address every call.
LCOM: lightweight component object model
COM is dynamic and LCOM is often static
It's the first time I hear about LCOM, Timo. It exists but apparently not on Windows: a search for "LCOM" lightweight component object model yields 25,000 hits but no official M$ docs on top.
A search for "LCOM" "lightweight component object model" yields only one hit here (https://programming.vip/keywords/c?page=8), and it's dead. Where did you find LCOM mentioned?
It is actually client side feature, but used many ways.
I can't remember nor find any good in from it either.
Some COM interfaces in html and richedit use it.
Why MS don't mentioned it :
https://learn.microsoft.com/en-us/windows/win32/com/the-lightweight-client-side-handler
Basically in server-side C++ isn't good for it, only languages like C and asm.
Interesting, thanks Timo :thumbsup:
I did some more testing, and while omitting CoInvoke pInterface, IShellDispatch.Release has no impact at all, doing the same with CoInvoke pFolder, Folder.Release is not a good idea: if it's disabled, there is a 100k leak per call.
What bothers me here is that it's so badly documented. Of course, I could add the IShellDispatch.Release, apparently it does no harm, but I'd like to know what's happening under the hood: is it needed, yes or no? If yes, why?
Globally shared memory is always problem for that kind of interfaces.
Memory is released, when usage count become zero, so release is very important.
If C++ compiler fail, code is just bad.
Quote from: TimoVJL on November 01, 2023, 11:26:07 PMGlobally shared memory is always problem for that kind of interfaces.
Do you have a link explaining the problem? My suspicion is that with CLSCTX_INPROC_SERVER the memory is local to the process; and I do not see
any increase of the working set when not releasing the IShellDispatch interface.
Is IShellDispatch interface capable to serve CLSCTX_INPROC_SERVER at all ?
Just too big interface with too many features ?
Quote from: TimoVJL on November 02, 2023, 12:31:49 AMIs IShellDispatch interface capable to serve CLSCTX_INPROC_SERVER at all ?
Well, zipping works fine, so obviously it is capable ;-)
Google groups, Dec 4, 2007 (https://groups.google.com/g/microsoft.public.scripting.vbscript/c/A2ySZ4fmQiE):
QuoteHowever, if there is an empty subdirectory I receive the following error
"The specified directory c:\temp\vbe is empty, so compressed (zipped) folders
cannot add it to the archive.
I tested this by adding an empty folder and using this code:
include \masm32\MasmBasic\MasmBasic.inc
Init
Dim Files$() ; redim to empty the Files$() array
Let Files$(0)="\Masm32\examples\threads"
ZipFiles "ThreadsTest.zip" ; create zip archive
ShEx "ThreadsTest.zip" ; open it
EndOfCodeResult: no problem,
ZipFiles (https://www.jj2007.eu/MasmBasicQuickReference.htm#Mb1073) doesn't complain but you'll see the empty folder in the archive.
Windows internals from the same post:
QuoteWenYuan Wang [MSFT]
unread,
Dec 7, 2007, 8:49:09 AM
to
Hello Rds,
Thanks for your reply.
I consulted shell team. Windows XP or above handles Zip files by
zipfldr.dll (which is located in %windir%\system32\zipfldr.dll). It's a
thrid party product. I'm sorry to say Microsoft doesn't provide any API to
compress Zip via zipfldr.dll.
The MSDN documentation you referenced about CopyHere method only applies to
normal file system path.It doesn't work for a zip file folder. That's why
you find the option 4 to disable the progress dialog doesn't help.It's by
desgin.
By the way, currently the zip file functionality is only meant to be used
with User Interaction. In other words, programmatically access to the zip
file NSE is not officially supported.
That was almost 16 years ago, and it seems it's still only unofficially supported. It works fine :biggrin: