The MASM Forum

General => The Campus => Topic started by: MichaelK on June 25, 2013, 03:05:14 AM

Title: Use of TCHAR in UNICODE build
Post by: MichaelK on June 25, 2013, 03:05:14 AM
Hello all. I've been searching for hours for a solution to this problem I'm having. I'm having trouble assembling a program making use of TCHAR. ML throws an error saying "constant value too large" when I try to use TCHAR in a Unicode build or even WORD instead of BYTE. Here is an example source:

.386
.model flat, stdcall
option casemap:none

include windows.inc
include user32.inc
include kernel32.inc
includelib user32.lib
includelib kernel32.lib

.DATA
szTitle TCHAR "ASM Application", 0
szMessage TCHAR "Hello, world.", 0

.CODE
start:
invoke MessageBox, 0, ADDR szMessage, ADDR szTitle, MB_OK OR MB_ICONINFORMATION
invoke ExitProcess, 0
end start

And here is the command and parameters I'm trying to build with:

ml /c /Cp /coff /nologo /W3 /D__UNICODE__ /D__NO_NOISE__ /I \masm32\include /Fo obj/main.o src/main.asm

Is there something I'm missing? That's just what I can't seem to figure out :(
Title: Re: Use of TCHAR in UNICODE build
Post by: hutch-- on June 25, 2013, 03:24:22 AM
Michael,

have a look at the HLHELP.CHM help file in MASM32. Under the heading Unicode support it explains how it works.
Title: Re: Use of TCHAR in UNICODE build
Post by: Zen on June 25, 2013, 03:30:37 AM
MICHAEL K.,
Hi. This is one of those weird problems where the solution is not obvious.
As you know, WINDOWS defines a TCHAR as a single byte character, using the ASCII standard codes,...unless, UNICODE is defined,...then, a TCHAR is defined as a two-byte Unicode character.
Most assembly programmers just use the single-byte character codes  because we all speak English. I think the problem is (and HUTCH mentioned this in a thread monthes ago), that the MASM copmpiler only recognizes the single-byte definition (which doesn't conform to the WINDOWS data type definition).
The solution is simple: find the MASM definition file and alter (temporarily) the value of the TCHAR character, to the double-byte UNICODE type.

Added as an edit: (OOPS,...HUTCH got the answer ahead of me)
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on June 25, 2013, 03:51:54 AM
i had this problem a while back
moreover, i wanted the LENGTHOF operator to return the number of characters, whether it was built for UNICODE or ANSI

qWord (forum member) gave me a nice solution that i have been using without problems, since

;###############################################################################################

;tchr macro by qWord

tchr    MACRO   lbl,args:VARARG
    IFDEF __UNICODE__
        UCSTR lbl,args
    ELSE
        lbl db args
    ENDIF
        ENDM

;###############################################################################################

        .DATA

tchr szNoFile,   'No File Specified',0
tchr szTooBig,   'File Too Large',0
tchr szNotFound, 'File Not Found',0
tchr szAlready,  'File Already Exists',0
tchr szInpFile,  'Input File:',13,10,0
tchr szOutFile,  'Output File:',13,10,0

;***********************************************************************************************
Title: Re: Use of TCHAR in UNICODE build
Post by: MichaelK on June 25, 2013, 06:11:45 PM
Thank you everyone for your replies. Though the docs do not state it clearly, my understanding is that you cannot use TCHAR to statically initialize a variable but only dynamically. Like so:

szBuffer TCHAR "Some text", 0   ; Error
szBuffer TCHAR 50 DUP(0)        ; OK


Also, when I try to use UCSTR like in the help file, ML throws a syntax error. Like this:

UCSTR szBuffer, "Some text", 0


Am I missing something like an include file? I'm using the same include files like the one in my original post.
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on June 25, 2013, 06:14:27 PM
UCSTR szBuffer, "Some text", 0

that assembles ok, here
make sure you are in either the .DATA or .CODE section (.CONST will also work)  :P
Title: Re: Use of TCHAR in UNICODE build
Post by: MichaelK on June 25, 2013, 06:22:56 PM
Unfortunately, not on my system. The whole program is exactly like the original one I posted except the inclusion of the UCSTR line (in the data section), but ML generates the following error message:
error A2008: syntax error : UCSTR
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on June 25, 2013, 06:30:50 PM
ok
you have to include more files   :P

in this case, specifically \masm32\macros\macros.asm

but - to make things simple, get rid of the entire preamble and

include \masm32\include\masm32rt.inc

that file takes care of casemap, processor, model, and a number of includes
Title: Re: Use of TCHAR in UNICODE build
Post by: MichaelK on June 25, 2013, 06:41:59 PM
Thanks. It works perfectly now  :biggrin:
Would you recommend editing masm32rt.inc to include qWord's code snippet so the use of TCHR when in Unicode mode or otherwise becomes transparent?
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on June 25, 2013, 06:45:59 PM
no - i only edit the masm32 files when a bug is found
otherwise, you will not be compatible with other forum members
you post a program that you can assemble, but others cannot - lol

put it in a seperate file
what i quite often do is to create an INC file for the project - that would be a good place
Title: Re: Use of TCHAR in UNICODE build
Post by: MichaelK on June 25, 2013, 07:07:36 PM
Got it. So I've created a separate include file to take care of that. This is how it looks (I modified the original snippet to use qWord's UCCSTR macro to enable support for escape sequences too):

TCHR MACRO lbl, args :VARARG
IFDEF __UNICODE__
UCCSTR lbl, args
ELSE
lbl db args
ENDIF
ENDM


I got a feeling today is gonna be awesome for me. Good start!  :eusa_dance:
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on June 25, 2013, 11:09:51 PM
TCCHR MACRO lbl, args :VARARG
IFDEF __UNICODE__
UCCSTR lbl, args
ELSE
lbl db args
ENDIF
ENDM


spot the difference ?   :biggrin:
Title: Re: Use of TCHAR in UNICODE build
Post by: MichaelK on January 04, 2014, 05:08:15 AM
Pardon me to resurrect this dead thread. But what's the difference between "TCHR" and "TCCHR". AFAIK, "TCHR" isn't defined anywhere.  :icon_confused:
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on January 04, 2014, 05:29:05 AM
TCCHR must be the name of someone's macro

TCHR is the name used by qWord for his macro
i prefer this macro because it returns correct values when used with LENGTHOF and SIZEOF operators
TCHR    MACRO lbl,args:VARARG

    IFNDEF __UNICODE__
        lbl db args
    ELSE
        UCSTR lbl,args
    ENDIF

        ENDM
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on January 04, 2014, 05:31:42 AM
declare the macro early in the source (it generates no code)

then, to use it....
    .DATA

TCHR  szString,"MyString",0


now, it generates the correct ANSI string if the __UNICODE__ symbol is not defined
or the correct UNICODE string if the __UNICODE__ symbol is defined
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on January 04, 2014, 05:35:21 AM
here is a little test program
build it as a console app
try it with/without the __UNICODE__ equate
;###############################################################################################

        __UNICODE__ EQU 1

        .XCREF
        .NoList
        INCLUDE    \Masm32\Include\Masm32rt.inc
        .List

;###############################################################################################

;TCHR macro by qWord

TCHR    MACRO lbl,args:VARARG

    IFNDEF __UNICODE__
        lbl db args
    ELSE
        UCSTR lbl,args
    ENDIF

        ENDM

;###############################################################################################

        .DATA

    TCHR    szTest,"Test",0

;###############################################################################################

        .CODE

;***********************************************************************************************

_main   PROC

    print   offset szTest,13,10
    mov     eax,lengthof szTest
    print   str$(eax),13,10
    mov     eax,sizeof szTest
    print   str$(eax),13,10
    inkey
    exit

_main   ENDP

;###############################################################################################

        END     _main
Title: Re: Use of TCHAR in UNICODE build
Post by: MichaelK on January 04, 2014, 05:39:08 AM
Thanks. But just wondering, why did you post
Quote from: dedndave on June 25, 2013, 11:09:51 PM
spot the difference ?   :biggrin:
earlier in this thread. You spotted an error somewhere?
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on January 04, 2014, 05:46:14 AM
i'm not sure - lol
i am too old to remember what i did yesterday, let alone last week   :biggrin:
Title: Re: Use of TCHAR in UNICODE build
Post by: ragdog on January 04, 2014, 11:45:54 PM
Hi and a happy new year:biggrin:

This is a good question i use for unicode WSTR

WSTR  wszNAME,"Masm32"   

What is now better ;TCHR macro by qWord or WSTR ?

Greets,
Title: Re: Use of TCHAR in UNICODE build
Post by: hutch-- on January 05, 2014, 12:15:46 AM
It is not a case of better or worse, read the reference material and use them for what they do.
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on January 05, 2014, 04:02:21 AM
WSTR is fine if you always want a UNICODE string
well - it doesn't always return correct results if you use SIZEOF/LENGTHOF operators with a string created with WSTR

WSTR szTest,"Test",0

the correct "LENGTHOF szTest" should be 5 - with WSTR, it will be 4
the correct "SIZEOF szTest" should be 10 - with WSTR, it will be 8
WSTR always defines a UNICODE string, whether or not the symbol __UNICODE__ is defined

in the latest version of the masm32 package, Hutch added a number of UNICODE macros written by qWord
of particular interest are qWord's UCSTR and UCCSTR macros
UCCSTR is like UCSTR, except it adds support for many C-style escape sequences (\n, \t, etc)
these macros always create UNICODE strings, but allow more complex definitions
also, the LENGTHOF and SIZEOF operators seem to work well with these macros

Hutch has a STRING macro that defines a UNICODE string if __UNICODE__ is defined and an ANSI string if it isn't
however, the LENGTHOF and SIZEOF operators do not always return proper values

so, qWord wrote the TCHR macro, which is not in the masm32 library
it's a pretty simple macro, because it relies on UCSTR (a complex macro) to do all the UNICODE work

;TCHR macro by qWord

TCHR    MACRO lbl,args:VARARG

    IFNDEF __UNICODE__
        lbl db args
    ELSE
        UCSTR lbl,args
    ENDIF

        ENDM


you can read a previous discussion, here...

http://masm32.com/board/index.php?topic=1305 (http://masm32.com/board/index.php?topic=1305)

and an older topic by qWord, here...

http://masm32.com/board/index.php?topic=680.0 (http://masm32.com/board/index.php?topic=680.0)
Title: Re: Use of TCHAR in UNICODE build
Post by: ragdog on January 05, 2014, 05:36:26 AM
Thanks Dave

in the latest version of the masm32 package,
I use Masm32v10 and for a long time there was problems with the last package with the kernel.inc
for unicode or ascii builds is this all fixed?

What use your?

Greets,
Title: Re: Use of TCHAR in UNICODE build
Post by: dedndave on January 05, 2014, 06:45:37 AM
i have both versions - i can switch back and forth by renaming a few folders   :P

but - the version 11 issues can be overcome by creating a new kernel32.inc and lib

qWord's UCSTR macro is probably in the forum, someplace
if you like, i don't think qWord would mind me posting it
Title: Re: Use of TCHAR in UNICODE build
Post by: qWord on January 05, 2014, 08:02:09 AM
When Hutch request help for the beta release of MASM32v11, my focus was on removing the bug in WSTR (IIRC) and improving the fn/rv macros (-->fnx/rvx). In context of this I proposed the UC* macros, which are indented to have the same behavior as MASM's DB/BYTE directive (can be used in any segment; SIZEOF/LENGTHOF works). At that time I didn't recognize that there is also the need for a macro like TCHR. Remarks that for the C-style version (UCCSTR) there is the issue that "\n" emits {10,13} instead of {13,10}, which should be fixed.
Quote from: dedndave on January 05, 2014, 06:45:37 AMqWord's UCSTR macro is probably in the forum, someplace
if you like, i don't think qWord would mind me posting it
The UC* macros are independent and can be copied from MASM32v11 to v10 without any problems, if needed. Here the fixed C-style version:UCCSTR MACRO lbl,args:VARARG

    IFNDEF uccstr_lbl_cntr
        uccstr_lbl_cntr = 0
    ENDIF
    IFB <&lbl>
        uccstr_lbl TEXTEQU <anonym_WSTR_>,%uccstr_lbl_cntr
        uccstr_lbl_cntr = uccstr_lbl_cntr + 1
    ELSE
        uccstr_lbl TEXTEQU <&lbl>
    ENDIF

    uccstr_size = 0
    uccstr_flg = 0
    uccstr_iarg = 1
    uccstr_esc = 0
    FOR _arg,<args>
        uccstr_flg = 0
        FORC char,<&_arg>
            IF uccstr_flg NE 0
                uccstr_pos INSTR 1,<"'\>,<&char>
                IF uccstr_pos EQ uccstr_flg
                    EXITM
                ELSEIF uccstr_pos EQ 3 AND uccstr_esc EQ 0
                uccstr_esc = 1
                ELSE
                    IF uccstr_esc
                        uccstr_pos INSTR 1,<\0ablrxqtpAn>,<&char>
                       
                        IFE uccstr_pos
                            uccstr_flg=0
                            EXITM
                        ENDIF
                        uccstr_size = uccstr_size + uccstr_pos/12
                        uccstr_esc = 0
                    ENDIF
                    uccstr_size = uccstr_size + 1
                ENDIF
            ELSE
                uccstr_flg INSTR 1,<"'>,<&char>
                IFE uccstr_flg
                    IF (OPATTR _arg) AND 100y
                        uccstr_flg = 3
                    ENDIF
                    EXITM
                ENDIF
            ENDIF
        ENDM
        IF uccstr_flg EQ 0 OR uccstr_esc NE 0
            EXITM
        ELSEIF uccstr_flg EQ 3
            uccstr_size = uccstr_size + 1
        ENDIF       
        uccstr_iarg = uccstr_iarg + 1
    ENDM
    IF uccstr_flg EQ 0 OR uccstr_esc NE 0
        IF uccstr_esc
        %   .err <invalid escape sequence : argument : @CatStr(%uccstr_iarg)>
        ELSE
        %   .err <invalid string specifier : argument : @CatStr(%uccstr_iarg)>
        ENDIF
        EXITM
    ENDIF
   
    align 2
    uccstr_lbl WORD uccstr_size dup (?)
    org $-uccstr_size*2
   
    uccstr_esc = 0
    FOR _arg,<&args>
        uccstr_flg = 0
        FORC char,<&_arg>
            IF uccstr_flg NE 0
                uccstr_pos INSTR 1,<"'\>,<&char>
                IF uccstr_pos EQ uccstr_flg
                    EXITM
                ELSEIF uccstr_pos EQ 3 AND uccstr_esc EQ 0
                    uccstr_esc = 1
                ELSE
                    IFE uccstr_esc
                        uccstr_char CATSTR <dw >,uccstr_quote,<&char>,uccstr_quote
                        uccstr_char
                    ELSE
                        uccstr_pos INSTR 1,<\0ablrxqtpAn>,<&char>
                        IFE uccstr_pos
                            uccstr_flg=0
                            EXITM
                        ENDIF
                        uccstr_char SUBSTR <  5ch00h28h29h3ch3eh21h22h09h25h26h0dh,0ah>,uccstr_pos*3,3+4*(uccstr_pos/12)
                        uccstr_esc = 0
                        dw uccstr_char
                    ENDIF
                ENDIF
            ELSE
                uccstr_flg INSTR 1,<"'>,<&char>
                IFE uccstr_flg
                    IF (OPATTR _arg) AND 100y
                        uccstr_flg = 3
                    ENDIF
                    EXITM
                ENDIF
                uccstr_quote TEXTEQU <&char>
            ENDIF
        ENDM
        IF uccstr_flg EQ 3
            dw _arg
        ENDIF       
    ENDM

endm

Title: Re: Use of TCHAR in UNICODE build
Post by: hutch-- on January 05, 2014, 09:55:15 AM
Arrrrgh,

> however, the LENGTHOF and SIZEOF operators do not always return proper values

The macros were not designed to do that. None the less this is the test result.


IF 0  ; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤
                      Build this template with "CONSOLE ASSEMBLE AND LINK"
ENDIF ; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    __UNICODE__ equ 1

    include \masm32\include\masm32rt.inc

    .code

start:
   
; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

    call main
    inkey
    exit

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

main proc

    WSTR mytext,"1234567890"                ; 10 characters

    print ADDR mytext," original string",13,10

    print ustr$(len(ADDR mytext))," len macro",13,10

    print ustr$(LENGTHOF mytext)," character count",13,10

    print ustr$(SIZEOF mytext)," byte count",13,10

    ret

main endp

; ¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤¤

end start

; results

    1234567890
    10 len macro
    10 character count
    20 byte count
    Press any key to continue ...
Title: Re: Use of TCHAR in UNICODE build
Post by: jj2007 on January 05, 2014, 03:08:38 PM
Folks, we started that debate over five years ago (http://www.masmforum.com/board/index.php?topic=10299.msg75571#msg75571) :badgrin:

include \masm32\MasmBasic\MasmBasic.inc        ; download (http://masm32.com/board/index.php?topic=94.0)
  Init
  wData MyWide$, 32, "Test", 32, 0
  wPrint wStr$("Size of MyWide$: \t%i bytes\n", SIZEOF MyWide$), wStr$("Length of MyWide$: \t%i chars\n", LENGTHOF MyWide$)
  wInkey "And there it is:", wTb$, "[", offset MyWide$, "]"
  Exit
end start

Output:
Size of MyWide$:        14 bytes
Length of MyWide$:      7 chars
And there it is:        [ Test ]


P.S. Just switched on my notebook after a long trip - no sign of life, the screen was dead. Some hours on the radiator helped wake it up, though.

Happy New Year :biggrin:
Title: Re: Use of TCHAR in UNICODE build
Post by: hutch-- on January 06, 2014, 08:56:45 AM
JJ,

Sounds like its really cold over there at the moment. We have the opposite problem although not the panic stations some would have you believe. I build computers with 6 inch fans front and back and monster coolers and on a hot day run a pedestal fan behind me.