Problem with subroutine to create stdin, stdout, and stderr

[markallyn]

Hello everyone,

Back on August 25, 2019 VORTEX submitted code that would create stdin, stdout, and stderr.  I had been trying for several weeks to figure out how to do this because these streams are very useful.  I finally saw V's post and recognized its brilliance as a solution to my failed efforts.  So, I set out to try to take his solution a bit further and make it a static library function.  In doing so I discovered that Pelles' POCRT does not contain __p_iob but msvcrt.lib has __iob_func which works similarly.

In the following submission I'm using FGETS and FPUTS simply as test functions.  FGETS fails in this code and for the life of me I can't understand why.  I have tried to annotate to help read the darned thing.

;trying to create subroutine to define stdin, stdout, and stderr.  Subroutine myio is based on
;a submission by VORTEX on August 25, 2019 to MASM Forum, Miscellaneous, The Orphanage, as response to JC FULLER post
;August 24.

include   myincludes64.inc    ;Homemade .inc file with typedefs, structs, et. I havve found useful.
includelib   c:\masm32\lib64\msvcrt.lib  ;This MSVCRT contains __iob_func to access stdin, stdout, stderr.

__iob_func PROTO :QWORD
puts       PROTO :QWORD
printf      PROTO :QWORD
fgets      PROTO :QWORD, :DWORD, :QWORD
fputs      PROTO :QWORD, :DWORD, :QWORD
myio      PROTO
.data
msg1   db   "Enter a string:",13,10,0
msg2   db   "The string you entered was:",13,10,0

.data?
stdin   QWORD    ?
stdout   QWORD   ?
stderr   QWORD   ?
buffer   BYTE   256 dup   (?)

.const

.code
main   PROC
enter    0,0               ;this section creates a stack, aligns the stack, and
and   rsp, -10h            ;makes room for spill space
sub   rsp,  20h

lea   rcx,   stdin
lea   rdx,   stdout
lea   r8,      stderr
call   myio            ;this call sets up stdin, stdout, stderr
                     ;Myio successfully returns to here.
mov   r11,   rcx            ;stdin should be in r11
mov   r12,   rdx            ;stdout should be in r12
mov   r13,   r8            ;stderr should be in r13
lea   rcx,   msg1         
call   printf            ;printf successfully prints message1
lea   rcx,    buffer
mov rdx,   80
mov   r8,      r11
call   fgets            ;FGETS fails!  WHY??      
lea rcx,   msg2
call    printf
lea   rcx,   buffer
mov   r8,     r12
call   fputs

add   rsp, 20h            ;this section removes spill space and tears down stack
leave
ret
main   ENDP

myio      PROC           ;This little PROC is based on VORTEX, August 25, 2019.  Thanks.
enter   0,0
and      rsp,   -10h
sub      rsp,   20h

push rcx
mov   rcx,   0            ;this section sets up stdin and stdout
call    __iob_func         ;__iob_func does what VORTEX's crt___p__iob function did in V's version
pop   rcx
mov   qword ptr [rcx],   rax
mov   r9,   SIZEOF (FILE)
add   rax,   r9
mov   qword ptr [rdx],   rax
add   rax,   r9
mov   qword ptr [r8],   rax

add   rsp,   20h
leave
ret                     ;Myio successfully returns.
myio   ENDP
END

Many thanks again to VORTEX for his original contribution.

Mark Allyn

[Vortex]

Hi markallyn,

Did you take care of the structure alignment?

https://www.geeksforgeeks.org/structure-member-alignment-padding-and-data-packing/

[tenkey]

My guess is that r11 gets clobbered by printf. R10 and r11 are not guaranteed to be call-preserved.
https://docs.microsoft.com/en-us/cpp/build/x64-calling-convention?view=msvc-160

[nidud]

deleted

[markallyn]

Hello VORTEX, TENKEY, and NIDUD,

Owing to family responsibilities (my daughter is in deep pregnancy) I only had a chance to look over Vortex's reply.  I read his attachment re structure alignment and found it very informative on a subject I didn't understand well.  However, I need a bit more detail on why Vortex thought there might be a misalignment in my FILE Struct (as this is the only struct I know about in the program).

Tenkey: I will test your idea later this evening and report back.

NIDUD:  You have spent a lot of time on this response and I really appreciate what you have done.  I'm going to learn a lot.  As I think you can tell from what I did I don't fully understand parameter passing using x64 ABI.  That was part of my motivation for trying to do this little project.  I notice that you used __acrt_iob_func!

Regards to each of you,

Mark

[markallyn]

Tenkey,

Of course you're right about r11 not being preserved!  I forgot.  I tried changing r11 to r12 and r12 to r13.  No luck, I'm sorry to report.  Same problem with FGETS.

Thanks for the suggestion!

Mark

[markallyn]

Good Morning, NIDUD,

I have been trying to replicate your code on my machine using visual studio 2017 x64 native tools command.  I cannot link to libcmt.lib and __acrt_iob_func.  I have found this latter function in windows\system32, however.  __acrt_iob_func on my machine is located in libucrt.dll.  But, there is no corresponding .lib or .inc in Windows \system32 or \Windows from which I can get the prototype.

Would you kindly tell me what is going on?

Regards,
Mark

[nidud]

deleted

[markallyn]

Good afternoon, NIDUD,

I looked in ALMOST the same VS 2019 directory that you mention, but there is a change of version in my file.  Instead of 14.28.29333\lib\x64 directory mine is 14.25.28610\lib\x64.  In my version there is no __acrt_iob_func as far as I can tell. 

You answered one question I had as a result of this.  Namely, I can use the __iob_func as a substitute.  But I gotta tell you that I would really like to use the __acrt_iob_func.  Not finding it (so far) is highly unsatisfying.

I agree it could be useful to figure this out--couldn't agree more.  I thought I might be able to use windows\system32 but no .lib to include.  Would have to "roll my own" and I'm lazy. 

Thanks again,
Mark

[Vortex]

Hi markallyn,

You don't need any other specific library to assemble the code below, the Masm64 additions developed by Hutch is doing all the job. The largest element in the _iobuf structure, QWORD has a size of 8 bytes so they need to be aligned to 8 byte boundary. The DWORD members in the same structure will require attention, so we instruct the assembler to do it for us with _iobuf STRUCT 8. You can see that the first DWORD member will "break" the alignment :

Code Select Expand

    _cnt        DWORD   ?

Here is a quick example for the Masm64 SDK :

Code Select Expand

include \masm32\include64\masm64rt.inc

_iobuf STRUCT 8 ; align to 8 byte boundary

    _ptr        QWORD   ?
    _cnt        DWORD   ?
    _base       QWORD   ?
    _flag       DWORD   ?
    _file       DWORD   ?
    _charbuf    DWORD   ?
    _bufsiz     DWORD   ?
    _tmpfname   QWORD   ?
   
_iobuf ENDS

_FILE TYPEDEF _iobuf

EXTERN __iob_func:PROC

.data?

_stdout     dq ?
_stdin      dq ?
_stderr     dq ?

.data

msg db "Please type your name :",13,10,0

.data?

szName db 16 dup(?)

.code

start PROC

    invoke  __iob_func

    mov    _stdin,rax           ; #define stdin  (&__iob_func()[0])
    add    rax,SIZEOF(_FILE)
   
    mov    _stdout,rax          ; #define stdout (&__iob_func()[1])
   
    add    rax,SIZEOF(_FILE)
    mov    _stderr,rax          ; #define stderr (&__iob_func()[2])

    invoke  vc_fputs,ADDR msg,_stdout

    invoke  vc_fgets,ADDR szName,16,_stdin

    invoke  vc_printf,"Nice to meet you, %s",ADDR szName

    invoke  ExitProcess,0

start ENDP

END

[tenkey]

markallyn,
You stored a file pointer with:

Code Select Expand

mov   qword ptr [rcx],   rax


So you need to retrieve the file pointer with:

Code Select Expand

mov   r8,      qword ptr [r11]   ; you will need to change r11


or

Code Select Expand

mov   r8,      stdin   ; not lea !!!


I got it to work by replacing both instances of r11 with r14 and adding qword ptr.

[nidud]

deleted

[markallyn]

Good evening, tenkey, vortex, and nidud,

Tenkey:  I made the change you suggested and the code worked perfectly.  Thanks much for your continued interest.

Nidud and Vortex:  You have both provided code that requires much closer study.  I promise to reply to what you have given me. 

All of you:  I can't tell you how grateful I am for your continued interest in my rather trivial problem.  I have learned a great deal and I am delighted by your generosity.

I wish there was a way to pay you back.

Mark

[markallyn]

Good morning, Vortex,

In your _iobuf struct I make out the size to be 48 bytes including 4 bytes of padding in the first DWORD breakage.  Confirmed with a little program I just wrote to count struct bytes.

That same program counts 80 bytes for my FILE struct as it is defined in my include file.  Leads to a question about my original code.  As follows:

Within my myio proc I set r9 to be sizeof(FILE).  Obviously not the same as your _FILE typedef.  One (ME!) would naively think that this would lead to a failure of my code NO MATTER WHAT ELSE might have been wrong.  I would have expected that adding the incorrect size to rax would have incorrectly computed stdout.

But, after making the change that tenkey suggested, my orginal code in fact runs.

Can you tell me why?

BTW, your previous discussion and link has been very helpful to me in the matter of padding and alignment. 

Thanks again,

Mark

[tenkey]

Hi markallyn,

fputs takes only two arguments.
It fails when you add:

Code Select Expand

mov rdx, r12    ; set argument 2 to address of stdout "variable"


Like the third argument of fgets, the second argument of fputs should be set with:

Code Select Expand

mov rdx, qword ptr [r12]    ; set argument 2 from stdout


Of course, that requires the correct value of sizeof(FILE).
You're just lucky that printf leaves rdx set to the stdout FILE pointer.