Problems trying to use ReadDirectoryChangesW()

[NoCforMe]

Hopping over here from another thread in the Workshop on how to synchronize a database under Windows, I've written a testbed program to try out the ReadDirectoryChangesW() function that sinsi posted about. Looks like it might be able to detect the changes to the filesystem that I'd been asking about.

But I can't get the damn thing to work. And there are some confusing things in the Micro$oft documentation that are definitely not helping.

I'm first attempting to use it in synchronous mode to keep things simple. (Later I'd like to use it asynchronously to be able to detect changes on the fly.)


Here's how I'm setting up the function call:

Code Select Expand

INVOKE ReadDirectoryChangesW,
TestDirHandle,
ADDR rdcBufferPtr,
$RDCbufferSize, ;?????
TRUE, ;bWatchSubtree
FILE_NOTIFY_CHANGE_FILE_NAME,
ADDR rdcBytesReturned,
NULL, ;lpOverlapped
NULL ;lpCompletionRoutine

Now, the documentation confusion is over the usage of 2nd and 3rd parameters. Here's what Microsoft Learn has to say about them:

[out] lpBuffer

A pointer to the DWORD-aligned formatted buffer in which the read results are to be returned. The structure of this buffer is defined by the FILE_NOTIFY_INFORMATION structure. This buffer is filled either synchronously or asynchronously, depending on how the directory is opened and what value is given to the lpOverlapped parameter. For more information, see the Remarks section.

[in] nBufferLength

The size of the buffer that is pointed to by the lpBuffer parameter, in bytes.

Now that is very strange: the pointer to the buffer pointer is an "out" parameter, meaning that the function sets this parameter, not the caller. But the buffer size is an "in" parameter, meaning that the caller must set that.

WTF???? That doesn't make any sense. Furthermore, the "remarks" for the function tell us

When you first call ReadDirectoryChangesW, the system allocates a buffer to store change information. This buffer is associated with the directory handle until it is closed and its size does not change during its lifetime.

So if the OS allocates this buffer and sets its size, how the hell am I supposed to know its size?

Based on what I've found elsewhere online, this may be a mistake in the documentation, since some code I found (which presumably works) allocates its own buffer and passes that as the buffer-pointer parameter. (That code is here.)

Anyway, I've tried various things:
If I pass a pointer to a DWORD variable that isn't initialized to anything, AND I set the buffer size to any nonzero value, I get the error "Invalid access to memory location" (obtained through GetLastError()).

If I set the buffer size parameter to zero, the program hangs (goes unresponsive).

If I set the buffer pointer parameter to an allocated buffer (using HeapAlloc()), the program hangs regardless of the buffer-size parameter.

I checked the directory handle I'm passing in to the function, and it's valid, so far as I can tell (CreateFile() returns with a valid file handle).

Anyone have any experience with this function? I can post some code later if that'll help.

[zedd151]

I found this while searching for ReadDirectoryChangesW...
https://masm32.com/board/index.php?msg=38448

Not exactly what you were asking but might help you achieve desired result?

[NoCforMe]

Thanks, but those functions (FindFirstChangeNotification() and FindNextChangeNotification()) have some limitations that ReadDirectoryChangesW() doesn't.

[sinsi]

The code you posted works. The reason it "hangs" is because it is waiting for a directory change to occur.
Once you e.g. add a file to the directory the function will return.

Synchronous, get it 

[TimoVJL]

Using it in thread might be easier for GUI.

Also using CreateIoCompletionPort() and GetQueuedCompletionStatus() with it works well.

A small program using TEC label printer worked well over decade (from 2004 - 2022).
development started with Pelles C 2.80

[NoCforMe]

The code you posted works. The reason it "hangs" is because it is waiting for a directory change to occur.
Once you e.g. add a file to the directory the function will return.

Synchronous, get it 

You're right. Duh! (on me, not you)

Now to go async, which should be much more useful.

Oh, and it definitely wants an allocated buffer, so that part of the docs is totally bogus.

[NoCforMe]

Also using CreateIoCompletionPort() and GetQueuedCompletionStatus() with it works well.

That program I linked to above uses WaitForMultipleObjects() and GetOverlappedResult().
Which ones should I use? Yours or theirs?
(I've only used WaitForMultipleObjects() once in my programming life, never used any of the other ones.)

[NoCforMe]

The code you posted works. The reason it "hangs" is because it is waiting for a directory change to occur.
Once you e.g. add a file to the directory the function will return.

OK, question:
If I call it in async mode, it'll return immediately, right?
So it will only return whatever directory changes happened from when? from the call? or from the the previous call?
And if I use a completion routine, does that then become active so that it will capture any directory changes as they occur?

My hope was that this could function as a kind of continuous monitor. But maybe it doesn't work that way, and I'd need to call it at regular intervals (every minute or so)?

[TimoVJL]

Also using CreateIoCompletionPort() and GetQueuedCompletionStatus() with it works well.

That program I linked to above uses WaitForMultipleObjects() and GetOverlappedResult().
Which ones should I use? Yours or theirs?
(I've only used WaitForMultipleObjects() once in my programming life, never used any of the other ones.)

Something like this, as it use thread

Code Select Expand

hCompPort = CreateIoCompletionPort(hDir, NULL, dwCompKey, 0);
while (bRun) {
  if (ReadDirectoryChangesW(hDir, &fni, sizeof(fni), FALSE, FILE_NOTIFY_CHANGE_LAST_WRITE, &dwRC, &ol, NULL)) {
    GetQueuedCompletionStatus(hCompPort, &dwWrite, &dwCompKey, &pOL, INFINITE);
    ...
  }
}
PostQueuedCompletionStatus(hCompPort, 0, 0, NULL);

EDIT: with events
ReadDirectoryChangesW_craziness.cpp

[jj2007]

Using it in synchronous mode in a thread might be the easiest solution. Before returning from the thread, just send yourself a message...

[NoCforMe]

OK. Looks like I have a few choices here.

[sinsi]

I couldn't get async to work, probably because I don't know enough about overlapped etc.
Sync in a thread worked much better.

What I found when I used every filter in dwNotifyFilter
 - rename always returned two entries, old name and new name, as expected
 - create new returned one entry
 - delete file returned one entry
 - create via copy returned three entries, but one at a time, one for creation and two for modified (size and time?), probably because of the filter
You always get the name but there's no indication whether it's a file or directory.

That's as far as I got before bedtime

Here's another example of rubbish documentation

If there is both a short and long name for the file, the function will return one of these names, but it is unspecified which one.

[NoCforMe]

Here's another example of rubbish documentation

If there is both a short and long name for the file, the function will return one of these names, but it is unspecified which one.

Well, honest, I guess.
But definitely rubbish.

"Duh, we just don't know how that code behaves under those circumstances ..."

[TimoVJL]

If you opened the file using the short name, you can receive change notifications for the short name.

EDIT:
ReadDirectoryChangesW function (winbase.h)

[sinsi]

If you opened the file using the short name, you can receive change notifications for the short name.

1. Searching for that quote gives me an "AI Overview"  .
2. We aren't opening files (unless the AI means directories as well).
3. "can" receive, not "will".

"Duh, we just don't know how that code behaves under those circumstances ..."

"The intern who wrote the code in the early 90s also documented it but he now works for Google, so..."