Problem with subroutine to create stdin, stdout, and stderr

[Vortex]

Hi Mark,

That same program counts 80 bytes for my FILE struct as it is defined in my include file.

It looks like that there is something wrong with your FILE stucture. The size of 80 bytes is not correct. Kindly, could you post it here?

[markallyn]

Hello tenkey:

Brilliant!  In this case being "lucky" was actually a disservice I did to myself.  Where I'm lucky is having you ferret out the "luck".  I also note that I was dumb in not remembering that fputs and fgets are not symmetric.

Needless to say, I'll fix the code.

Mark

[markallyn]

Good morning, Vortex,

Here is the FILE struct I have in myincludes64.inc:

FILE   struct
mode   DWORD   ?
fh   DWORD   ?
buf   DWORD   ?
bufend   DWORD   ?
xptr   DWORD   ?
getend   DWORD   ?
putend   DWORD   ?
backptr   DWORD   ?
wbackptr   DWORD   ?
wbackbuf   DWORD 2 dup (?)
getback   DWORD   ?
wgetend   DWORD   ?
wputend   DWORD   ?
wstate   mbstate_t   ?
tmpnam   DWORD   ?
backbuf   BYTE 8 dup (?)
cbuf   BYTE   ?
locknum   DWORD   ?
FILE   ends

LPFILE typedef ptr FILE

It is very different from _iobuf!  The _iobuf resident in the same .inc file has the same members as yours, but whereas you use all QWORDs for your fields my _iobuf varies in byte-size in several instances. 

Thanks again.

Mark

[TimoVJL]

With msvcrt.dll FILE structure size is important.

I use sometimes this with msvcrt.dll

Code Select Expand

typedef struct FAKEFILE {
char *_ptrs[3];
int _ints[5];
} FILE;
so 3 pointers and 5 integers

for x64

Code Select Expand

fakefile STRUCT 8
ptrs PTR 3 DUP(?)
ints DWORD 5 DUP(?)
fakefile ENDS

A test with ml64

Code Select Expand

option casemap:none
exit PROTO C :DWORD
printf PROTO C :PTR,:VARARG
INCLUDELIB msvcrt

fakefile STRUCT ;8
ptrs QWORD 3 DUP(?)
ints DWORD 5 DUP(?)
fill DWORD ?
fakefile ENDS

.data
fmt db "size: %d",10,0
.code
mainCRTStartup PROC C
; invoke printf, ADDR msg
; invoke printf, ADDR fmt, sizeof fakefile
; invoke exit,0
mov rdx, sizeof fakefile
mov rcx, offset fmt
call printf
call exit
mainCRTStartup ENDP
END ;mainCRTStartup
outputs size: 48

[markallyn]

Hello TIMOVJL,

I sort of see what you did inasmuch as the number of each struct member type matches _iobuf.  What I don't get is how the assembler knows to associate your two types (int, ptr) with the same type in the struct.  The _iobuf members are not 3 ptrs in a row followed by 5 ints.  So how does the assembler "know" that the ptr array in your struct needs to be distributed across the "real" _iobuf pointers.  Likewise, the 5 integers.

But, I must say this is very elegant once you know what _iobuf struct looks like.

Thanks.  I'll try it.

Mark

[tenkey]

What I don't get is how the assembler knows to associate your two types (int, ptr) with the same type in the struct.

Hi markallyn,

The assembler doesn't know. You tell the assembler how the data is distributed. And you can lie about it. You could actually use:

Code Select Expand

fakefile STRUCT
filedata BYTE 48 dup(?)
fakefile ENDS


and it would work if the only info you need is the size of the struct.

[TimoVJL]

as pointers are QWORD in x64, programmer sees, what makes difference to 32-bit version, but there are also those align things, you still can't use same struct for 32/64-bit.

[mineiro]

Assembler don't know; so we need instruct assembler.
Generally types are listed in some header file, or include file.
If you like to enter in any project the first question is where are types? Whats their size? ... .
If you try to start translate C, Gtk, QT, windows, ..., you should start by finding types in header files or documentation.
In Linux they are described in ABI paper, I suppose in windows too. Both have "handles" that are "types to type" too. So the rest of documentation is in help files, other header files, ... .

A structure should be aligned to 8 in linux/windows x86-64. When I first meet that without understanding whats happening I adopt a technique to give enough space to not know strucures. So, when I see a structure but don't like to read manuals or translate that to asm I simply do like:

mystruct dq 8*N           ;a 8 bytes multiple

An example of a structure being aligned to 8 and not aligned follow:

mystruct struct
one db ?        ;1 byte     1
two dw ?        ;2 bytes    3
three dd ?      ;4 bytes    7
four dq ?       ;8 bytes    15
mystruct ends

The sizeof mystruct without alignment is 1+2+4+8=15 bytes.
With 8 alignment that sizeof should be multiple of 8, so assembler create some "pad" elements inside that structure.

mystruct struct 8
one db ?        ;1 byte     1
two dw ?        ;2 bytes    3
three dd ?      ;4 bytes    7
                ;next element is 8 bytes, the total sum for a while is 7 bytes.
                ;assembler inserts a "pad" element here to us to be 8 bytes aligned
pad db ?        ;<----|     8
four dq ?       ;8 bytes    16
mystruct ends

-----------------
Mytype equ db
.data
one Mytype ?

[markallyn]

Good morning tenkey, TimoVJL, and Mineiro,

I see your point regarding structure member size.  Also, Mineiro's discussion of padding was very useful.  I think what was bothering me was partly related to padding.  In TimoVJL's simplified _iobuf structure I could see how 3 pointers and 5 ints (3*8 bytes and 5 * 4 bytes) -- would work out to 48 bytes (divisible by 8) and so the assembler would just merrily populate the struct with bytes and the programmer -- as Mineiro says so well -- would know, because of documentation in headers, etc. -- how the bytes in the struct should be handled.  But, what about the case where chars, short ints, were scattered in between 4 and 8 byte members?  Now the programmer would have to insert padding to make the struct work out to something divisible by 8?   Just as Mineiro's first example struct requires padding at certain points.

Aside:  What makes this byte business about structs confusing to a beginner like me is that structs need 8-byte alignment and the stack needs to be 16 byte aligned.  No doubt there is a very good reason why this is so (and it must be processor related I'll bet), but it is confusing.

Many thanks to all of you for a very informative discussion. 

[nidud]

deleted

[nidud]

deleted

[mineiro]

-> But, what about the case where chars, short ints, were scattered in between 4 and 8 byte members?

char equ db
mystruct struct
one char ?      ;1
two ptr ?       ;8
mystruct ends

mystruct struct
one char ?          ;1     ;8-1=7, next member fits in 7 bytes (is dd,dw,db)?
                    ;no, so pad that
pad db 7 dup (?)    ;8
two ptr ?           ;16
mystruct ends

-> Now the programmer would have to insert padding to make the struct work out to something divisible by 8?
If you do that by hands, answer is yes. And this gets more crazy when exist structures inside structures.

When our program starts, stack pointer ends with 8, or, rsp == ???????8h. To do any call to a function, rsp should be rsp=???????0h (16 bytes aligned means that a number ends with 0 in hexadecimal).
The only exception happens with leaf functions. Functions that do not call other functions internally is one example of leaf function. In this case we can call that function with rsp ending with 8 or 0 in hexadecimal. But, to be sure, it's better call that with rsp ending with 0h.

[nidud]

deleted

[mineiro]

Thanks for talking about shadow space sir nidud. You're absolutelly right.

[markallyn]

Good afternoon, Mineiro and Nidud,

First, when I said that the programmer would have to insert the padding in structs not divisible by 8, for clarity I should have said either the programmer OR the assembler would have to do this.  Since asking this question, I subsequently came across an old post by NIDUD in which he showed how to insert padding "by hand" in a struct.  What led me to this was a question I had about programs that could do this--of course, a silly question because that's exactly what assemblers like ml64 do.

NIDUD helpfully pointed out that SIMD on x64 processors (amd, intel) eats chunks of 16 bytes from memory.  I suppose then, by extrapolation, 256 byte vector processors would want 32 byte alignment?