Iczelion tutorial 35 Highlighting

[Fumio]

I had worked through this tut some time ago on my old XP machine and all was well. Now on Vista I cannot get the "NewRicheditProc" to work at all. The problem has to be related to the subclassing procedure but I need assistance finding the problem. The error I get is "Access volation when writing to 129990"
Fumio

[MichaelW]

AFAICT the tutorial targets Rich Edit versions 2.0 or 3.0, and judging from this, Vista does not, or at least did not, include these versions.

[jj2007]

AFAICT the tutorial targets Rich Edit versions 2.0 or 3.0, and judging from this, Vista does not, or at least did not, include these versions.

Win7-32 includes
C:\Windows\System32\riched20.dll (20.11.2010)
C:\Program Files\Common Files\Microsoft Shared\OFFICE11 (17.5.2011)
C:\Program Files\Common Files\microsoft shared\OFFICE12\RICHED20.DLL (27.7.2011)
C:\Program Files\Common Files\Microsoft Shared\OFFICE14\RICHED20.DLL (28.12.2010)

For reading RTF files into the control, the Office11 version is about 20% faster than the other versions.

Note that EM_STREAMOUT is broken, at least in some versions. Use EM_GETTEXTEX instead.

[hutch--]

I have been writing richedit based editors for years and I have found that riched 2-3 works on everything from Win2000 to Win7 64 bit but note that I only process text, not RTF.

[Fumio]

Thanks to those who commented on my question. Still looking at the problem and found a makefile for the version that works and it has a cmd to expand the stack:
NAME=IczEdit
$(NAME).exe: $(NAME).obj $(NAME).res
        Link /STACK:2000000,2000000 /SUBSYSTEM:WINDOWS /LIBPATH:c:\masm32\lib $(NAME).obj $(NAME).res
$(NAME).res:$(NAME).rc
   rc $(NAME).rc
$(NAME).obj: $(NAME).asm
        ml /c /coff /Cp $(NAME).asm
And looking at the working exe vs the nonworking exe I see that there is a difference in the main thread stack size with working version being 78000 H bytes larger.
I also note that my non working program fails on an attempt to write to 00129990 which is outside the stack area.
Is there a cmd I can use in RadAsm that will expand the stack?
I have tried ".STACK 2000000; which compiles but still crashes with same error and stack is no bigger.

[dedndave]

nice find   :t

[jj2007]

Just checked RichMasm, and it has standard stack size 100000h/1000h
So I wonder what exactly makes your app crash - it seems not to be the RichEd control.

[Fumio]

Hello dedndave and jj2007: The richedit control itself functions nicely, its only the subclassing of the control that is at issue.
The call in wndproc to invoke SetWindowLong,hwndRichEdit,GWL_WNDPROC, addr NewRichEditProc
      mov OldWndProc,eax
And the call within the newWndproc: invoke CallWindowProc,OldWndProc,hWnd,uMsg,wParam,lParam
to return control to the original WndProc seem to be causing the trouble.
Fumio

[dedndave]

i got it to work ok
i made a "MakeIt.bat" file to build it...

Code: [Select]

\masm32\bin\rc /v IczEdit.rc
if exist rsrc.res del rsrc.res
if exist IczEdit.res ren IczEdit.res rsrc.res
if exist rsrc.obj del rsrc.obj
\masm32\bin\cvtres /machine:ix86 rsrc.res
if exist rsrc.res del rsrc.res

if exist IczEdit.obj del IczEdit.obj
\masm32\bin\ml /c /coff IczEdit.asm

\masm32\bin\Link /STACK:2000000,2000000 /SUBSYSTEM:WINDOWS /OPT:NOREF IczEdit.obj rsrc.obj

if exist rsrc.obj del rsrc.obj
if exist IczEdit.obj del IczEdit.obj
it seems to be sublcassing the control just fine
the one problem i did see was....

Code: [Select]

.elseif uMsg==WM_CLOSE
invoke SetWindowLong,hWnd,GWL_WNDPROC,OldWndProc
.else
invoke CallWindowProc,OldWndProc,hWnd,uMsg,wParam,lParam
ret
.endif
NewRichEditProc endpWM_CLOSE is received - it "un-subclasses" itself - then it goes out to lala land - lol
no RET or anything
probably ok to return 0 and RET, but i did it this way.....

Code: [Select]

.elseif uMsg==WM_CLOSE
invoke SetWindowLong,hWnd,GWL_WNDPROC,OldWndProc
invoke CallWindowProc,OldWndProc,hWnd,uMsg,wParam,lParam
ret
.else
invoke CallWindowProc,OldWndProc,hWnd,uMsg,wParam,lParam
ret
.endif
NewRichEditProc endp
as far as i know, that method of sub-classing works fine under vista
so - i think it's something else (in the WM_PAINT code)

[jj2007]

My RichEd subclass is 500+ lines, and it works just fine with standard stack...
Your problem is where it writes "outside the stack area" at 12990h. That sounds really odd ::)

P.S.: In general, invoke SetWindowLong,hWnd,GWL_WNDPROC,OldWndProc is not necessary, the OS can handle that.

[Fumio]

Hello dedndave: I duplicated your process using the bat file and it works absolutely fine on my vista machine. I had been trying to build the tut35 exe via Radasm 2 and 3 neither worked. It seems as if that "/STACK:2000000,2000000 " option is the stickler. I tried several formats to implement that via the project options in Radasm but was unsuccessful. If you have any tips on how that could be done I sure would like to know. However it's good to know that Vista is not the problem. Thanks for the .bat
Hello jj2007: Yes it is odd the problem show up in the ShowWindow api. I have traced into it and still haven't found exactly where the problem occurs, but it is in the latter section of that routine and even if I find exactly where, I don't know if I would have the expertise to understand what is going wrong. As I mentioned before I compared the starting stack sizes and the exe that works has a much bigger stack than the one that doesn't.
I am able to work with the tut now so that is my main concern but I would like to find out how to make it work via Radasm. Again, I believe it has something to do with the
/STACK:2000000,2000000 option used in the bat file and original make file.
Thanks for your interest and help
Fumio

[Fumio]

Succeeded in increasing stack size but still crashes at with multiple .. 9990

Fumio

[dedndave]

you can modify the EXE header after it has been linked by using editbin.exe

Code: [Select]

editbin /STACK:2000000,2000000 IczEdit.exe
other than that, i am not familiar enough with RadAsm to help, there   :P

[Fumio]

Here is the project file I have been using via Radasm 2.2.1.5 The files are exactly the same as the files used to make a working exe built via a bat file. Note that the asm, rc etc are unchanged versions of Iczelion's Tutorial 35, I have not intentionally changed any of the code.
Fumio

[Fumio]

Hi dedndave: I finally was able to increase stack size by using the link option /STACK: [number] I increased it to the exact size that existed in the working tut35.exe . I attached a pic of Project options tab in  Radasm.

Fumio