News:

Masm32 SDK description, downloads and other helpful links
Message to All Guests
NB: Posting URL's See here: Posted URL Change

Main Menu

mmap & the length of the mapping

Started by mabdelouahab, October 01, 2020, 06:42:13 AM

Previous topic - Next topic

mabdelouahab

I tried the following
.DATA
   _Len   dq  1099511627776   ;010000000000h   = 1 terabyte
.CODE         
   mov mapped    ,rv(_mmap,0, _Len, PROT_READ or PROT_WRITE,MAP_SHARED or MAP_ANON ,-1, 0)
   mov mapped2 ,rv(_mmap,0, 1024, PROT_READ or PROT_WRITE,MAP_SHARED or MAP_ANON ,-1, 0)

mapped2-mapped=007ffbc1363000-007efbc1158000=01000020B000h {1099513769984} = Len+020B000h Greater than 1 TB

The problem is that I only have 500 GB in my disk and 8GB RAM
Where does the system store this remaining space?

hutch--

You will probably find that if you tried to write to that range, it would crash once you exceeded available memory. Its the difference between mapped address space versus available memory.

mabdelouahab

Quote from: hutch-- on October 01, 2020, 06:49:53 AM
You will probably find that if you tried to write to that range, it would crash once you exceeded available memory. Its the difference between mapped address space versus available memory.

mov rax,mapped
mov dword ptr [rax+16],2222
add rax,_Len
mov dword ptr [rax-16],1365


A preliminary try, Writing at the beginning and end of the buffer, Without crashing

hutch--

Try writing to all of it in a linear write.

mabdelouahab

I'm apprehensive, it might replace my files  :biggrin:
I have to make sure of what I do

TouEnMasm


Msdn don't use this function that i don't know from where it come.
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getdiskfreespaceexa
and
Quote
To determine the total number of bytes on a disk or volume, use IOCTL_DISK_GET_LENGTH_INFO
Fa is a musical note to play with CL

mabdelouahab

#6
hi TouEnMasm
Quote from: TouEnMasm on October 01, 2020, 06:13:49 PM
Msdn don't use this function that i don't know from where it come.
Linux: mmap(2) — Linux manual page
windows: Memory Management , MapViewOfFile

mineiro

I received a laptop with 500gb disk to format.
That have linux32 and I will switch to linux64.

Quote from: mabdelouahab on October 01, 2020, 03:06:54 PM
I'm apprehensive, it might replace my files  :biggrin:
I have to make sure of what I do
If you have some example that I can run, feel free to post.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

mabdelouahab

hi mineiro;


; uasm  -elf64 mmaptest.asm
; gcc  -o mmaptest mmaptest.o -fno-pie -no-pie

OPTION LITERALS:ON
_length EQU 010000000000h; 1099511627776
    MAP_ANON    EQU 020h
    MAP_SHARED  EQU 01h
    PROT_READ   EQU 01h
    PROT_WRITE  EQU 02h

printf proto   :ptr, :VARARG
.data
_Len dq  _length
.code
    mmap PROC SYSTEMV _addr:qword ,__length:qword,_prot:dword,_flags:dword,_fd:dword,_offset:qword
       MOV R10,RCX ; SYSCALL uses R10 instead of RCX
       MOV RAX,9
       SYSCALL
       RET
    mmap ENDP
    munmap PROC SYSTEMV _addr:qword,__length:qword
       MOV RAX,11
       SYSCALL
       RET
    munmap ENDP
main proc
local mapped:qword

invoke mmap,0, _Len, PROT_READ or PROT_WRITE,MAP_SHARED or MAP_ANON ,-1, 0
mov mapped  ,rax
.if sqword ptr  rax > 0
invoke printf,"writing ... \n"
mov rax,mapped
;test Write
mov rcx,_length
@@:
mov qword ptr [rax+rcx],rcx
sub rcx,8
jnz @B
;test 3 read
mov rax,mapped
mov rcx,1024
invoke printf,"Read at qword [1024]= {%d}\n",qword ptr [rax+rcx]
mov rax,mapped
mov rcx,1024 * 1024
invoke printf,"Read at qword [1024 * 1024]={%d}\n",qword ptr [rax+rcx]
mov rax,mapped
mov rcx,1024 * 1024 * 256
invoke printf,"Read at qword [1024 * 1024 * 256]={%d}\n",qword ptr [rax+rcx]
invoke munmap,mapped, _Len
.else
invoke printf,"Fail to map memory err={%d}\n",rax
.endif
xor rax,rax
ret
main endp
end


writing  takes about two minutes

mineiro

I'm installing O.S. now, later I will edit this post and tell you what happened.
./mmaptest
Fail to map memory err={-12}

Do you have other thinking? like decreasing memory to be alloc? I'll stay with this computer today and tomorrow, so, if you have more tests, tell and i will try here
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

mabdelouahab

Add this line to the code:
      ;test Write
      mov rcx,_length
      sub rcx,8; <--------------------------
      @@:
         mov qword ptr [rax+rcx],rcx
      sub rcx,8
      jnz @B


mineiro

ok, will post soon.
Realize with 500gb and 100gb give to me same error message.
With 500MB works:
./mmaptest4
writing ...
Read at qword [1024]= {1024}
Read at qword [1024 * 1024]={1048576}
Read at qword [1024 * 1024 * 256]={268435456}

now will try that change.

----edit----
Tried with 1tera, 500gb,100gb and same error, with 500mb works.
Later I will try with sizeof swap space+avaliable RAM.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

mabdelouahab

You have a space limitation of 16 GB, try this command:
echo 1 | sudo tee /proc/sys/vm/overcommit_memory

You can undo by changing the zero by one:
echo 0 | sudo tee /proc/sys/vm/overcommit_memory


mineiro

Will try your last post now.
For a while, these are results:
maptest9=swap+ram
maptest10=ram
maptest11=swap+ram


free
              total       usada       livre    compart.  buff/cache  disponível
Mem.:       3915956      559824     2856780       69340      499352     3063684
Swap:       2097148           0     2097148


./mmaptest9
Fail to map memory err={-12}
./mmaptest10
writing ...
Read at qword [1024]= {1024}
Read at qword [1024 * 1024]={1048576}
Read at qword [1024 * 1024 * 256]={268435456}
./mmaptest11
Fail to map memory err={-12}
sudo ./mmaptest11
Fail to map memory err={-12}


system get slow after mmaptest10.

---------edit-------------
Quote from: mabdelouahab on October 02, 2020, 01:18:21 AM
You have a space limitation of 16 GB, try this command:
echo 1 | sudo tee /proc/sys/vm/overcommit_memory
Ok, after this command I was able to execute "sudo ./mmaptest11"(swap+ram). Spend some time executing ("writing") but after comes a message "death" or "die", not sure, message was in portuguese language. Like process killed I suppose.
Now I will try with 1TB without sudo, process started at 13:21pm. Received segmentation fault (core? image saved). Ended at time 13:22pm.
Now will try with 1TB (sub rcx,8 modification). Was quickly, received messages as been work fine, but execute in less than 10 seconds. No problem to system.
Trying now 500GB (sub modification),12:23pm. Received death message again.
Trying being root 500GB, death again.

Computer was not responsible while doing these tests, so much slow, I was not able to see last typed command in bash shell history.

If you have other ideas, feel free to say.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

mabdelouahab

Quote from: mineiro on October 02, 2020, 02:07:05 AM
Now will try with 1TB (sub rcx,8 modification). Was quickly, received messages as been work fine, but execute in less than 10 seconds. No problem to system.
I want an explanation for this, This is what I am asking for
Where is this space mapped?