mmap & the length of the mapping

[mabdelouahab]

I tried the following
.DATA
   _Len   dq  1099511627776   ;010000000000h   = 1 terabyte
.CODE         
   mov mapped    ,rv(_mmap,0, _Len, PROT_READ or PROT_WRITE,MAP_SHARED or MAP_ANON ,-1, 0)
   mov mapped2 ,rv(_mmap,0, 1024, PROT_READ or PROT_WRITE,MAP_SHARED or MAP_ANON ,-1, 0)

mapped2-mapped=007ffbc1363000-007efbc1158000=01000020B000h {1099513769984} = Len+020B000h Greater than 1 TB

The problem is that I only have 500 GB in my disk and 8GB RAM
Where does the system store this remaining space?

[hutch--]

You will probably find that if you tried to write to that range, it would crash once you exceeded available memory. Its the difference between mapped address space versus available memory.

[mabdelouahab]

You will probably find that if you tried to write to that range, it would crash once you exceeded available memory. Its the difference between mapped address space versus available memory.

Code Select Expand

mov rax,mapped
mov dword ptr [rax+16],2222
add rax,_Len
mov dword ptr [rax-16],1365


A preliminary try, Writing at the beginning and end of the buffer, Without crashing

[hutch--]

Try writing to all of it in a linear write.

[mabdelouahab]

I'm apprehensive, it might replace my files 
I have to make sure of what I do

[TouEnMasm]

Msdn don't use this function that i don't know from where it come.
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getdiskfreespaceexa
and

To determine the total number of bytes on a disk or volume, use IOCTL_DISK_GET_LENGTH_INFO

[mabdelouahab]

hi TouEnMasm

Msdn don't use this function that i don't know from where it come.

Linux: mmap(2) — Linux manual page
windows: Memory Management , MapViewOfFile

[mineiro]

I received a laptop with 500gb disk to format.
That have linux32 and I will switch to linux64.

I'm apprehensive, it might replace my files 
I have to make sure of what I do

If you have some example that I can run, feel free to post.

[mabdelouahab]

hi mineiro;

Code Select Expand



; uasm  -elf64 mmaptest.asm
; gcc  -o mmaptest mmaptest.o -fno-pie -no-pie

OPTION LITERALS:ON
_length EQU 010000000000h; 1099511627776
    MAP_ANON    EQU 020h
    MAP_SHARED  EQU 01h
    PROT_READ   EQU 01h
    PROT_WRITE  EQU 02h

printf proto   :ptr, :VARARG
.data
_Len dq  _length
.code
    mmap PROC SYSTEMV _addr:qword ,__length:qword,_prot:dword,_flags:dword,_fd:dword,_offset:qword
       MOV R10,RCX ; SYSCALL uses R10 instead of RCX
       MOV RAX,9
       SYSCALL
       RET
    mmap ENDP
    munmap PROC SYSTEMV _addr:qword,__length:qword
       MOV RAX,11
       SYSCALL
       RET
    munmap ENDP
main proc
local mapped:qword

invoke mmap,0, _Len, PROT_READ or PROT_WRITE,MAP_SHARED or MAP_ANON ,-1, 0
mov mapped  ,rax
.if sqword ptr  rax > 0
invoke printf,"writing ... \n"
mov rax,mapped
;test Write
mov rcx,_length
@@:
mov qword ptr [rax+rcx],rcx
sub rcx,8
jnz @B
;test 3 read
mov rax,mapped
mov rcx,1024
invoke printf,"Read at qword [1024]= {%d}\n",qword ptr [rax+rcx]
mov rax,mapped
mov rcx,1024 * 1024
invoke printf,"Read at qword [1024 * 1024]={%d}\n",qword ptr [rax+rcx]
mov rax,mapped
mov rcx,1024 * 1024 * 256
invoke printf,"Read at qword [1024 * 1024 * 256]={%d}\n",qword ptr [rax+rcx]
invoke munmap,mapped, _Len
.else
invoke printf,"Fail to map memory err={%d}\n",rax
.endif
xor rax,rax
ret
main endp
end


writing  takes about two minutes

[mineiro]

I'm installing O.S. now, later I will edit this post and tell you what happened.

Code Select Expand

./mmaptest
Fail to map memory err={-12}

Do you have other thinking? like decreasing memory to be alloc? I'll stay with this computer today and tomorrow, so, if you have more tests, tell and i will try here

[mabdelouahab]

Add this line to the code:
      ;test Write
      mov rcx,_length
      sub rcx,8; <--------------------------
      @@:
         mov qword ptr [rax+rcx],rcx
      sub rcx,8
      jnz @B

[mineiro]

ok, will post soon.
Realize with 500gb and 100gb give to me same error message.
With 500MB works:

Code Select Expand

./mmaptest4
writing ...
Read at qword [1024]= {1024}
Read at qword [1024 * 1024]={1048576}
Read at qword [1024 * 1024 * 256]={268435456}

now will try that change.

----edit----
Tried with 1tera, 500gb,100gb and same error, with 500mb works.
Later I will try with sizeof swap space+avaliable RAM.

[mabdelouahab]

You have a space limitation of 16 GB, try this command:
echo 1 | sudo tee /proc/sys/vm/overcommit_memory

You can undo by changing the zero by one:
echo 0 | sudo tee /proc/sys/vm/overcommit_memory

[mineiro]

Will try your last post now.
For a while, these are results:
maptest9=swap+ram
maptest10=ram
maptest11=swap+ram

Code Select Expand



free
              total       usada       livre    compart.  buff/cache  disponível
Mem.:       3915956      559824     2856780       69340      499352     3063684
Swap:       2097148           0     2097148


./mmaptest9
Fail to map memory err={-12}
./mmaptest10
writing ...
Read at qword [1024]= {1024}
Read at qword [1024 * 1024]={1048576}
Read at qword [1024 * 1024 * 256]={268435456}
./mmaptest11
Fail to map memory err={-12}
sudo ./mmaptest11
Fail to map memory err={-12}


system get slow after mmaptest10.

---------edit-------------

You have a space limitation of 16 GB, try this command:
echo 1 | sudo tee /proc/sys/vm/overcommit_memory

Ok, after this command I was able to execute "sudo ./mmaptest11"(swap+ram). Spend some time executing ("writing") but after comes a message "death" or "die", not sure, message was in portuguese language. Like process killed I suppose.
Now I will try with 1TB without sudo, process started at 13:21pm. Received segmentation fault (core? image saved). Ended at time 13:22pm.
Now will try with 1TB (sub rcx,8 modification). Was quickly, received messages as been work fine, but execute in less than 10 seconds. No problem to system.
Trying now 500GB (sub modification),12:23pm. Received death message again.
Trying being root 500GB, death again.

Computer was not responsible while doing these tests, so much slow, I was not able to see last typed command in bash shell history.

If you have other ideas, feel free to say.

[mabdelouahab]

Now will try with 1TB (sub rcx,8 modification). Was quickly, received messages as been work fine, but execute in less than 10 seconds. No problem to system.

I want an explanation for this, This is what I am asking for
Where is this space mapped?