News:

Masm32 SDK description, downloads and other helpful links
Message to All Guests
NB: Posting URL's See here: Posted URL Change

Main Menu

Clear File Cache

Started by aw27, August 25, 2018, 02:14:45 AM

Previous topic - Next topic

aw27

The attachment contains an unofficial method to clear the file cache (It is related to this topic).
Automatically runs as administrator.

Possible output:

Cache: Current Size 32248 KB PeakSize 521748 KB MinimumWorkingSet 256 B MaximumWorkingSet 4294967296 B
Flushed FileCache WorkingSet
Emptied Memory Working Sets
Cache: Current Size 0 KB PeakSize 521748 KB MinimumWorkingSet 256 B MaximumWorkingSet 4294967296 B





jj2007

With some modifications, it builds:
\masm32\bin64\rc.exe res.rc
\masm32\bin64\ml64  -c -Zp8 filecache.asm
\masm32\bin64\link /ENTRY:main /SUBSYSTEM:console /FIXED /MACHINE:X64 filecache.obj res.res


But which is the unofficial method? And why is it better than ClearFileCache? If it works, that is - what about a testbed, as in the other thread?

six_L

Hi,AW
1 runas normally.
QuoteCacheInfo:
Current Size: 14188 KB
PeakSize: 473860 KB
MinimumWorkingSet: 256 B
MaximumWorkingSet: 0 B

Flushed FileCache WorkingSet

Emptied Memory Working Sets

CacheInfo:
Current Size: 14188 KB
PeakSize: 473860 KB
MinimumWorkingSet: 256 B
MaximumWorkingSet: 0 B
2 runas administrator
Quoteerror on NtSetSystemInformation
option casemap:none
option win64:3

include \UASM64\include\windows.inc

includelib \UASM64\Lib\kernel32.lib
includelib \UASM64\Lib\user32.lib
includelib \UASM64\Lib\advapi32.lib
includelib \UASM64\Lib\gdi32.lib
includelib \UASM64\Lib\ntdll.lib

ICO_MAIN equ 1000h
DLG_MAIN equ 100
IDA_EDIT equ 102
IDC_CLEAR equ 103
NULL EQU 0
FALSE EQU 0
TRUE EQU 1
SystemFileCacheInformation EQU 21
SystemMemoryListInformation EQU 80
MemoryEmptyWorkingSets EQU 2
SE_PRIVILEGE_ENABLED EQU 2h
TOKEN_ADJUST_PRIVILEGES EQU 20h
TOKEN_QUERY EQU 8h
HANDLE TYPEDEF PTR

LUID STRUCT
LowPart  DWORD ?
HighPart SDWORD ?
LUID ENDS
 
LUID_AND_ATTRIBUTES STRUCT
Luid LUID <>
Attributes DWORD ?
LUID_AND_ATTRIBUTES ENDS

PLUID_AND_ATTRIBUTES TYPEDEF PTR LUID_AND_ATTRIBUTES

TOKEN_PRIVILEGES STRUCT
PrivilegeCount DWORD ?
Privileges LUID_AND_ATTRIBUTES{}
TOKEN_PRIVILEGES ENDS

SYSTEM_FILECACHE_INFORMATION STRUCT
CurrentSize QWORD ?
PeakSize QWORD ?
PageFaultCount DWORD ?
MinimumWorkingSet QWORD ?
MaximumWorkingSet QWORD ?
CurrentSizeIncludingTransitionInPages QWORD ?
PeakSizeIncludingTransitionInPages QWORD ?
TransitionRePurposeCount DWORD ?
Flags DWORD ?
SYSTEM_FILECACHE_INFORMATION ENDS

NtSetSystemInformation PROTO :DWORD, :PTR, :DWORD
NtQuerySystemInformation PROTO :DWORD, :PTR, :DWORD, :PTR

.data
szBuffer db 4096 dup(0)
CacheInfo SYSTEM_FILECACHE_INFORMATION <>
fmt db "CacheInfo:",13,10
db "Current Size: %d KB ",13,10
DB "PeakSize: %d KB ",13,10
DB "MinimumWorkingSet: %d B ",13,10
DB "MaximumWorkingSet: %d B ",0
SeIncreaseQuotaPrivilege db 'SeIncreaseQuotaPrivilege',0
SeProfileSingleProcessPrivilege db 'SeProfileSingleProcessPrivilege',0
flushedWS db 'Flushed FileCache WorkingSet',13,10,0
emptiedWS db 'Emptied Memory Working Sets',13,10,0

.data?
hInstance dq ?
hMainhWnd dq ?
hEdithWnd dq ?

.code

ErrorMessage Proc USES RBX lpCaption:qword
Local lpErrorMessage:QWORD

call GetLastError
lea rbx,lpErrorMessage
invoke FormatMessage, FORMAT_MESSAGE_ALLOCATE_BUFFER or FORMAT_MESSAGE_FROM_SYSTEM, NULL, EAX, LANG_NEUTRAL,Rbx,0,NULL
invoke MessageBox, 0, lpErrorMessage, lpCaption, MB_OK
invoke LocalFree, lpErrorMessage
ret   

ErrorMessage EndP

SetPrivilege proc hToken: QWORD, lpszPrivilege : PTR, bEnablePrivilege : DWORD
LOCAL luid : LUID
LOCAL tp : TOKEN_PRIVILEGES

invoke LookupPrivilegeValueA,NULL,lpszPrivilege,ADDR luid
.if rax==0
invoke ErrorMessage,CStr("LookupPrivilegeValueA")
jz @exit
.endif
mov tp.PrivilegeCount, 1
mov eax,luid.LowPart
mov tp.Privileges.Luid.LowPart, eax
mov eax,luid.HighPart
mov tp.Privileges.Luid.HighPart, eax
cmp bEnablePrivilege, 0
jz @F
mov tp.Privileges.Attributes, SE_PRIVILEGE_ENABLED
jmp short @L1
@@:
mov tp.Privileges.Attributes,0
@L1:
invoke AdjustTokenPrivileges,hToken,FALSE,ADDR tp,sizeof TOKEN_PRIVILEGES,NULL,NULL
@exit:
ret
SetPrivilege endp

DisplayCacheInfo proc
Local tmpBuf[512]:BYTE

  invoke RtlZeroMemory,ADDR tmpBuf, sizeof tmpBuf
invoke NtQuerySystemInformation,SystemFileCacheInformation,offset CacheInfo,sizeof SYSTEM_FILECACHE_INFORMATION,NULL
mov rdx, CacheInfo.CurrentSize
shr rdx, 10 ; divide by 1024
mov r8, CacheInfo.PeakSize
shr r8, 10 ; divide by 1024
invoke wsprintf,addr tmpBuf,offset fmt,RDX,R8,CacheInfo.MinimumWorkingSet,CacheInfo.MaximumWorkingSet
invoke lstrcat, addr szBuffer,addr tmpBuf
invoke lstrcat, addr szBuffer, CStr(13,10)
invoke SetWindowText,hEdithWnd,addr szBuffer
ret
DisplayCacheInfo endp

_filecache proc
LOCAL processToken : HANDLE
LOCAL command : DWORD

invoke DisplayCacheInfo

invoke GetCurrentProcess
invoke OpenProcessToken,RAX,TOKEN_ADJUST_PRIVILEGES or TOKEN_QUERY,ADDR processToken
.if rax==0
invoke ErrorMessage,CStr("OpenProcessToken")
jz @exit
.endif
; // Clear FileCache WorkingSet
invoke SetPrivilege,processToken,CStr("SeIncreaseQuotaPrivilege"),TRUE
.if rax==0
invoke ErrorMessage,CStr("SetPrivilege")
jz @exit
.endif
invoke memset,offset CacheInfo,0,sizeof SYSTEM_FILECACHE_INFORMATION
mov CacheInfo.MinimumWorkingSet, -1
mov CacheInfo.MaximumWorkingSet, -1
invoke NtSetSystemInformation,SystemFileCacheInformation,offset CacheInfo,sizeof SYSTEM_FILECACHE_INFORMATION
.if rax==0
invoke ErrorMessage,CStr("NtSetSystemInformation")
jnz @exit
.endif

invoke lstrcat, addr szBuffer, CStr(13,10)
invoke lstrcat, addr szBuffer, offset flushedWS
invoke SetWindowText,hEdithWnd,addr szBuffer

; Purge Memory Standby
invoke SetPrivilege,processToken,offset SeProfileSingleProcessPrivilege,TRUE
.if rax==0
invoke ErrorMessage,CStr("SetPrivilege")
jz @exit
.endif
mov command, MemoryEmptyWorkingSets
invoke NtSetSystemInformation,SystemMemoryListInformation,addr command,sizeof DWORD
.if rax==0
invoke ErrorMessage,CStr("NtSetSystemInformation")
jnz @exit
.endif
invoke lstrcat, addr szBuffer, CStr(13,10)
invoke lstrcat, addr szBuffer, offset emptiedWS
invoke SetWindowText,hEdithWnd,addr szBuffer
invoke lstrcat, addr szBuffer, CStr(13,10)
invoke DisplayCacheInfo
@exit:
ret
_filecache endp

_ProcDlgMain Proc USES rbx hWnd:qword,wMsg:dword,wParam:qword,lParam:qword

mov eax,wMsg
.if eax == WM_INITDIALOG
mov rax,hWnd
mov hMainhWnd,rax

invoke LoadIcon,hInstance,ICO_MAIN
invoke SendMessage,hWnd,WM_SETICON,ICON_BIG,rax
invoke GetDlgItem,hWnd,IDA_EDIT
mov hEdithWnd,rax

.elseif eax == WM_COMMAND
mov rax,wParam
.if ax == IDCANCEL
invoke EndDialog,hWnd,NULL
.elseif ax == IDOK
invoke RtlZeroMemory,ADDR szBuffer, sizeof szBuffer
invoke CreateThread,NULL,NULL,offset _filecache,NULL,NULL,NULL
invoke CloseHandle,rax
.elseif ax == IDC_CLEAR
invoke SetWindowText,hEdithWnd,NULL
.endif
.elseif eax == WM_CLOSE
invoke EndDialog,hWnd,NULL
.else
mov rax,FALSE
ret
.endif
mov rax,TRUE
ret

_ProcDlgMain endp

WinMainCRTStartup Proc 

invoke GetModuleHandle,NULL
mov hInstance,rax
invoke DialogBoxParam,hInstance,DLG_MAIN,NULL,offset _ProcDlgMain,NULL
invoke ExitProcess,NULL
ret

WinMainCRTStartup  Endp


end



//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
#include <\UASM64\include\resource.h>
//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
#define ICO_MAIN 0x1000
#define DLG_MAIN 100
#define IDA_LOOPF 101
#define IDA_EDIT 102
#define IDC_CLEAR 103
//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
ICO_MAIN ICON "main_1.ico"
//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
DLG_MAIN DIALOG 293, 180, 300, 144
STYLE DS_MODALFRAME | DS_CENTER | WS_POPUP | WS_VISIBLE | WS_CAPTION | WS_SYSMENU | WS_THICKFRAME
CAPTION "filecache"
FONT 10, "Cambria"
{
EDITTEXT IDA_EDIT, 2, 2, 295, 121,ES_WANTRETURN | ES_MULTILINE | ES_AUTOHSCROLL | ES_AUTOVSCROLL | WS_BORDER | WS_VSCROLL | WS_HSCROLL | WS_TABSTOP
PUSHBUTTON "Clear(&C)", IDC_CLEAR, 2, 126, 32, 14
PUSHBUTTON "Filecache(&F)", IDOK, 203, 126, 45, 14
PUSHBUTTON "Exit(&X)", IDCANCEL, 250, 126, 45, 14
}
//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

the attachments are exe and Mark Russinovich's Cacheset.exe.
Say you, Say me, Say the codes together for ever.

aw27

@six_L

NtSetSystemInformation and NtQuerySystemInformation return NTSTATUS which is a DWORD, not a QWORD.
This may be the reason for the results you obtain (but I did not went through the code in detail). Incidentally, I did not get an error message but am sent to the error report routine as if there was an error.




You can find the Mark Russinovich's Cacheset source code in the:
http://web.archive.org/web/20060427210005/http://www.sysinternals.com/  :bgrin:


@JJ

Quote
ClearFileCache MACRO fname
  push ecx
  ClearLastError
  invoke CreateFile, repargA(f$), GENERIC_READ, 0, 0, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN or FILE_FLAG_NO_BUFFERING, 0
  invoke CloseHandle, eax
  pop ecx
ENDM

You intend to flush the cache by opening a file and closing it.
Are you kidding?

The purpose of flushing a cache is mostly related to writing cache data to the disk (not data read from disk and available in the cache to be used by your program). So, we flush data written to the cache while the file is open, we don't open the file to flush it. What is the purpose? Most times we don't do it, because as mentioned earlier, the cache usually does a good job by flushing when it finds convenient. However, there are cases where we may need, or want, to make data appear on hard disk as soon as possible - an example might be the classical producer-consumer problem .



jj2007

Quote from: AW on August 26, 2018, 12:16:51 AMYou intend to flush the cache by opening a file and closing it.
Are you kidding?

No, I am not kidding. My solution of thehay95's problem works fine, as demonstrated in the other thread.

Quote from: thehay95 on August 22, 2018, 06:23:15 PMHow does one invalidate the file cache? I'm trying to solve a performance bug that occurs only when the files are not in the cache.

hutch--

What am I missing in this discussion ? Where I have used it, the API FlushFileBuffers() seems to have worked OK. It is designed to work with a file handle and an open file, I have not yet seen a technique that tells the OS to hurry up when no file is open. The lazy flush is an operating system characteristic which makes sense if the processor(s) are busy but I don't know of an OS command that flushes the file buffers if no file is open.

With CreateFile() I have on occasions used the flag FILE_ATTRIBUTE_TEMPORARY as it generally stays in cache so it can be re-read quickly and deleted quickly. I suggest that the flags to use with CreateFile() are where the action is with the required task, even if it needs to be re-designed.

jj2007

Quote from: hutch-- on August 26, 2018, 02:05:31 AM
What am I missing in this discussion ?

OP in the other thread has a "performance bug" for non-cached files only. So he needs to simulate a situation where the program needs to read the file "fresh from the harddisk". This is what my macro does. FlushFileBuffers is a different animal (and I have no idea what José is trying to do or to prove here 8)).

aw27

@JJ,
Your macro does not do anything you think it does.

You don't know what my code does because you don't know 64-bit Assembly language and your Windows knowledge is at the Charles Petzold 1988 book level.
Six_L understood it perfectly well, it is uncomplicated and self explanatory. There is also the above mentioned Mark Russinovichh code that is along the same lines of  my code.
So, you need to try to learn instead of producing scam code like the ClearFileCache macro.

jj2007

I know I shouldn't feed the troll but...

Quote from: AW on August 26, 2018, 04:25:08 AM
@JJ,
Your macro does not do anything you think it does.

I know what it does, and it does exactly what OP in the other thread wants: ClearFileCache removes one file's content from the file cache. Your (undocumented function) NtSetSystemInformation sets, without any need, system-wide parameters, thus affecting the whole system's performance. Ugly :(

QuoteYou don't know what my code does because you don't know 64-bit Assembly language and your Windows knowledge is at the Charles Petzold 1988 book level.

Why is it that whenever you are short of arguments, you start insulting people? Is your company in such a desperate situation that you don't have your feelings under control? I imagine it's tough to be forced to earn a living with programming, and I feel kind of privileged that I can lean back and happily enjoy coding, but boy, can't you relax a bit from time to time...?  :P

Btw, 64-bit assembly (lines 146 to 149 of your code):
; AW
cmp eax,0
jz @exit
mov rcx, offset CacheInfo
mov edx, 0
nop
; JJ
test eax, eax
jz @exit
lea rcx, CacheInfo
xor edx, edx

Why? Because it's the correct way to code that in 64-bit land, and also 7 bytes shorter:
83 F8 00                         | cmp eax,0                       |
74 1D                            | je <sub_1400010AC>              |
48 B9 44 12 00 40 01 00 00 00    | movabs rcx,140001244            |
BA 00 00 00 00                   | mov edx,0                       |
90                               | nop                             |
85 C0                            | test eax,eax                    |
74 09                            | je <sub_1400010AC>              |
48 8D 0D 9A 01 00 00             | lea rcx,qword ptr ds:[140001244 |
33 D2                            | xor edx,edx                     |

aw27

Ah, shooting blanks now!  :badgrin: The bloatware master speaking about saving 7 bytes.
Fools and fanatics are always so certain of themselves, they always win. Take the prize.

hutch--

Now here is the $64.00 question, why are you both using 32 bit registers when the code is supposed to be 64 bit. We all know that under Win64 that a 32 bit register is zero extended to 64 bit but you are both trying to save a few bytes with non native sized instruction fetches.

aw27

Quote from: hutch-- on August 26, 2018, 07:59:58 PM
Now here is the $64.00 question, why are you both using 32 bit registers when the code is supposed to be 64 bit. We all know that under Win64 that a 32 bit register is zero extended to 64 bit but you are both trying to save a few bytes with non native sized instruction fetches.

I follow the prototypes and compilers follow the same criteria.
Here is how VS 2017 deals with ExitProcess(1)
   mov   ecx, 1
   call   QWORD PTR __imp_ExitProcess

hutch--

 :biggrin:

I am not sure that VC is the criterion to measure assembler code by. Probably does not matter as a high level API is hardly what you would call fast software but I would be very wary of doing it in a speed critical algorithm.

nidud

#13
deleted

daydreamer

Quote from: hutch-- on August 26, 2018, 07:59:58 PM
Now here is the $64.00 question, why are you both using 32 bit registers when the code is supposed to be 64 bit. We all know that under Win64 that a 32 bit register is zero extended to 64 bit but you are both trying to save a few bytes with non native sized instruction fetches.
the 256euro question is ,why dont use avx,avx2 and one or two 64bit register as loop counter?

Nidud, interesting test,but I haven't seen previous tests on fpu(2byte),SSE(3byte),SSE2(4byte) for simple scalar floating Point operations,like mul,div,add,sub would be affected to be executed slower because loop because it becomes too large for code cache

isnt assembly programmers seen as fanatics outside assembler programmers anyway AW?

my none asm creations
https://masm32.com/board/index.php?topic=6937.msg74303#msg74303
I am an Invoker
"An Invoker is a mage who specializes in the manipulation of raw and elemental energies."
Like SIMD coding