Watch directory for changes

[Biterider]

Hi
I had a problem with some test equipment that generates csv-files and I needed to monitor these files once they were generated on a network share.

I remembered some discussions here about using FindFirstChangeNotification/FindNextChangeNotification or ReadDirectoryChangesW and read some information on the net about the shortcomings of these APIs, so I decided to try it without using an IOCP.  Btw, a good reading on this topic https://medium.com/tresorit-engineering/how-to-get-notifications-about-file-system-changes-on-windows-519dd8c4fb01.

The problem with ReadDirectoryChangesW is that it is prone to dropping changes if the information is not fetched fast enough. This returned information is stored in a linked list by one thread (hWatcherThread) and a second thread (hProcessThread) processes it by calling a callback procedure defined at initialisation.
Synchronisation is implemented using events and SRWLock. Slim Reader/Writer (SRW) locks were introduced in Win7 and are very easy to use.

In my tests with this code under heavy load, moving ~4000 files to SSD, no change notification is missing! 

Regards, Biterider

[ognil]

Maybe the  detour technique for patching the original ReadDirectoryChangesW  function as an idea can help

Q: Patch ReadDirectoryChangesW via retour in masm64 assembly for windows 10

A: Changing or hooking the ReadDirectoryChangesW or/and other functions in Windows using native code (MASM64 assembly) involves understanding several aspects of Windows internals, including:

Windows API Hooking : This is the process of intercepting calls to a specific function and redirecting them to your own implementation.

Function Pointer Manipulation : To modify the behavior of ReadDirectoryChangesW, you need to manipulate its address.

64-bit Assembly Programming : MASM64 has specific calling conventions and register usage for x64 architecture.
Steps to Hook ReadDirectoryChangesW

1. Understand the Function Signature
The ReadDirectoryChangesW function has the following signature:

Code Select Expand

BOOL ReadDirectoryChangesW(
  HANDLE                          hDirectory,
  LPVOID                          lpBuffer,
  DWORD                           nBufferLength,
  BOOL                            bWatchSubtree,
  DWORD                           dwNotifyFilter,
  LPDWORD                         lpBytesReturned,
  LPOVERLAPPED                    lpOverlapped,
  LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine
);
This function is part of the Windows API and is exported by kernel32.dll. In x64, it follows the fastcall calling convention, where the first four arguments are passed in registers (RCX, RDX, R8, R9) and the rest are pushed onto the stack.

2. Locate the Function Address
To hook ReadDirectoryChangesW, you need to find its address in memory. You can do this using GetProcAddress from the Windows API.

Code Select Expand

; Example: Get the address of ReadDirectoryChangesW
extrn GetModuleHandleA : proc
extrn GetProcAddress : proc

.data
kernel32_dll db "kernel32.dll", 0
read_directory_changes db "ReadDirectoryChangesW", 0

.code
GetReadDirectoryChangesWAddress proc
    sub rsp, 28h                     ; Allocate shadow space for x64 calling convention
    lea rcx, kernel32_dll            ; Load the name of the DLL
    call GetModuleHandleA            ; Get the handle to kernel32.dll
    lea rdx, read_directory_changes  ; Load the name of the function
    mov rcx, rax                     ; Move the module handle into RCX
    call GetProcAddress              ; Get the address of ReadDirectoryChangesW
    add rsp, 28h                     ; Restore the stack
    ret
GetReadDirectoryChangesWAddress endp
3. Create a Trampoline Function
Once you have the address of ReadDirectoryChangesW, you can create a trampoline function that will redirect calls to your custom implementation.

Code Select Expand

; Original function address (to be filled later)
OriginalReadDirectoryChangesW dq 0

; Custom implementation of ReadDirectoryChangesW
MyReadDirectoryChangesW proc
    ; Save original registers
    push rbx
    push rsi
    push rdi

    ; Log or modify parameters here
    ; For example, you could log the directory handle (RCX)
    ; or modify the buffer (RDX)

    ; Call the original function
    mov rbx, [OriginalReadDirectoryChangesW]
    sub rsp, 20h                     ; Shadow space for original function
    call rbx                         ; Call the original ReadDirectoryChangesW
    add rsp, 20h                     ; Restore the stack

    ; Restore registers
    pop rdi
    pop rsi
    pop rbx

    ret
MyReadDirectoryChangesW endp
4. Patch the Original Function
To redirect calls to ReadDirectoryChangesW, you need to overwrite the first few bytes of the original function with a jump to your custom implementation. This is typically done using a detour technique.
In x64, you can use a relative jump (JMP) to redirect execution. The JMP instruction takes 5 bytes: 1 byte for the opcode (E9) and 4 bytes for the relative offset.

Code Select Expand

; Function to patch the original ReadDirectoryChangesW
PatchReadDirectoryChangesW proc
    sub rsp, 28h                     ; Allocate shadow space

    ; Get the address of the original function
    call GetReadDirectoryChangesWAddress
    mov [OriginalReadDirectoryChangesW], rax

    ; Calculate the relative offset for the JMP instruction
    lea rdx, MyReadDirectoryChangesW ; Address of our custom function
    mov rcx, rax                     ; Address of the original function
    sub rdx, rcx                     ; Calculate relative offset
    sub rdx, 5                       ; Subtract 5 bytes for the JMP instruction

    ; Write the JMP instruction to the original function
    mov byte ptr [rax], 0E9h         ; Write the JMP opcode
    mov dword ptr [rax + 1], edx     ; Write the relative offset

    add rsp, 28h                     ; Restore the stack
    ret
PatchReadDirectoryChangesW endp
5. Restore the Original Function (Optional)
If you want to restore the original function after hooking it, you need to save the original bytes before overwriting them. You can then restore these bytes when needed.

Code Select Expand

; Save the original bytes
OriginalBytes db 5 dup(0)

SaveOriginalBytes proc
    mov rax, [OriginalReadDirectoryChangesW]
    mov rsi, rax
    lea rdi, OriginalBytes
    movsd                             ; Save the first 4 bytes
    movsb                             ; Save the 5th byte
    ret
SaveOriginalBytes endp

RestoreOriginalBytes proc
    mov rax, [OriginalReadDirectoryChangesW]
    lea rsi, OriginalBytes
    mov rdi, rax
    movsd                             ; Restore the first 4 bytes
    movsb                             ; Restore the 5th byte
    ret
RestoreOriginalBytes endp
6. Putting It All Together
Finally, you can call the PatchReadDirectoryChangesW function to apply the hook. Once the hook is applied, any calls to ReadDirectoryChangesW will be redirected to your custom implementation.

Code Select Expand

main proc
    ; Apply the hook
    call PatchReadDirectoryChangesW

    ; Your program logic here

    ; Optionally restore the original function
    call RestoreOriginalBytes
    ret
main endp
Important Considerations
Permissions : Modifying code in memory requires appropriate permissions. You may need to adjust memory protection using VirtualProtect.
Thread Safety : Ensure that no other threads are executing the function while you're patching it.
Compatibility : Be cautious when modifying system functions, as this can lead to instability or crashes if not done correctly.

Conclusion
Hooking ReadDirectoryChangesW in MASM64 involves finding the function's address, creating a custom implementation, and patching the original function with a jump to your code. This technique allows you to intercept and modify the behavior of the function at runtime. However, it should be used with caution, as improper handling can lead to system instability.

[Biterider]

Hi ognil
Thanks for your post, but there is no need to patch or detour anything at all. 
ReadDirectoryChangesW works perfectly well if you know how to use it properly.

Biterider

[Biterider]

Hi
Today I tested the code for what I wrote it and found that the file server packs the data much more tightly than running the same code on my PC.
This shows a bug that I only noticed when the unpacking loop was needed.

Another fix was the shutdown sequence.
First, the exit event is signalled to cut the data feed from the watch thread, and when that thread exists, it gets signalled to trigger the exit of the process thread.
This sequence now works as intended.

First post updated.

Regards, Biterider