Author Topic: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting  (Read 938 times)

Mikl__

  • Member
  • ****
  • Posts: 554
Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« on: August 11, 2017, 03:42:49 PM »
Tutorial 35: RichEdit Control: Syntax Hilighting
Hi, all!
I collected an application from 35-th Iczelion lesson and found a strange and unpleasant mistake. If the comments are not Latin letters (in Russian Cyrillic is used), then everything looks like 00.png
In the attachment, the source text and exe-/rc-files from the site Iczelion https://win32assembly.programminghorizon.com/tut35.html
Tell me how can fix this error?
There is text with russian comments in tut_35.zip

aw27

  • Member
  • ****
  • Posts: 852
  • Let's Make ASM Great Again!
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #1 on: August 11, 2017, 04:41:45 PM »
I don't know what nice and color rich editor you are using but I can see it very well with Notepad ++ and selecting one of the Cyrillic charsets.

StreamInProc proc hFile:QWORD,pBuffer:QWORD, NumBytes:QWORD, pBytesRead:QWORD
; Оo yЛoКВaМХЧ ЙoД ОpeДБopЪeРcЪ enter,40h,0 ОoЩРoЛy rbp=rsp+40h   
   and qword ptr [rbp-20h],0; mov qword ptr [rsp+20h],0
   invoke ReadFile; ,hFile,pBuffer,NumBytes,pBytesRead,0
   xor eax,1
   leave
   retn
StreamInProc endp
;--------------------------------
StreamOutProc proc hFile:QWORD,pBuffer:QWORD, NumBytes:QWORD, pBytesWritten:QWORD
; Оo yЛoКВaМХЧ ЙoД ОpeДБopЪeРcЪ enter,40h,0 ОoЩРoЛy rbp=rsp+40h   
   and qword ptr [rbp-20h],0; mov qword ptr [rsp+20h],0
   invoke WriteFile; ,hFile,pBuffer,NumBytes,pBytesWritten,0
   xor eax,1
   leave
   retn
StreamOutProc endp

jj2007

  • Member
  • *****
  • Posts: 7734
  • Assembler is fun ;-)
    • MasmBasic
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #2 on: August 11, 2017, 06:30:58 PM »
Here is a little converter from Russian (codepage 1251) to UTF8. Just drag your source over the attached exe.

include \masm32\MasmBasic\MasmBasic.inc
  Init
  Let esi=Utf8$(wCL$())  ; Unicode file names are allowed
  Let edi=Left$(esi, Instr_(esi, ".")-1)+"converted.asm"
  Open "O", 1, edi
  Print #1, Utf8Bom
  Recall esi, L$()
  For_ ecx=0 To eax-1
        PrintLine #1, ConvertCp$(L$(ecx), 1251, CP_UTF8)
  Next
  Close
  ShEx edi
EndOfCode

Code: [Select]
; пo yмoлчaнию кoд пpeдвopяeтcя enter,40h,0 пoэтoмy rbp=rsp+40h
; зaпoлняeм пoля cтpyктypы CHARFORMAT, чтoбы ycтaнoвить цвeт тeкcтa
...
;=============================================================
;  Кoгдa пoльзoвaтeль кликaeт нa oднoй из пaлитpы цвeтoв, зaпoлняeм пoля
;  cтpyктypы CHOOSECOLOR и вызывaeм диaлoгoвoe oкнo выбopa цвeтa ChooseColor.
;  Еcли пoльзoвaтeль выбиpaeт цвeт, тo этo знaчeниe colorref вoзвpaщaeтcя в
;  члeнe rgbResult, coхpaняeм этo знaчeниe в пepeмeннoй BackgroundColor.
;  Пocлe этoгo, мы вынyждaeм пepeкpaшивaниe нa пaлитpe цвeтoв, вызывaя
;  InvalidateRect нa дecкpиптop пaлитpы цвeтoв. Пaлитpa цвeтoв пocылaeт
;  WM_CTLCOLORSTATIC cooбщeниe poдитeльcкoмy oкнy
...
;==========================================
;  Кoгдa пoльзoвaтeль кликaeт нa oднoй из пaлитpы цвeтoв, вызывaeтcя диaлoгoвoe
;  oкнo выбopa цвeтa. "Пaлитpa цвeтoв" - cтaтичecкий элeмeнт yпpaвлeния c флaгoм
;  WS_BORDER и SS_NOTIFY. Стaтичecкий элeмeнт yпpaвлeния c флaгoм SS_NOTIFY
;  yвeдoмит poдитeльcкoe oкнo o дeйcтвиях мыши нa нeм, типa BN_CLICKED
;  (STN_CLICKED). Этo - yлoвкa.
;===========================================

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 4924
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #3 on: August 11, 2017, 10:34:08 PM »
I remember doing a syntax colouring version back in about 2000 using Iczelion's technique and it worked well as it was only producing technicolor for the client area and not modifying the entire file. This made it fast and the normal capacity with richedit 2/3 to load very large files (>100 meg) FAST !!!!

I never followed it up because I just don't like technicolor in source code, I find it very distracting and much slower to read.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :biggrin:

HSE

  • Member
  • ****
  • Posts: 552
  • <AMD>< 7-32>
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #4 on: August 11, 2017, 11:56:23 PM »
One or two years ago I tried to use this Iczelion's rutines in a plugin for qEditor. Of course not sucess at all :(
The almost sure thing is that I maked everything wrong  :biggrin:, but look like qEditor set colors after the entry point and a plugins can't to change colors. 

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 4924
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #5 on: August 12, 2017, 12:33:19 AM »
You are right, it will not work in QE as it has its own colour control. If you wanted to try Iczelion's technique I would try it in a basic rich edit control that does not have its own colour control and it should work OK. Its been a long time ago but it was a fast technique and worked well.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :biggrin:

HSE

  • Member
  • ****
  • Posts: 552
  • <AMD>< 7-32>
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #6 on: August 12, 2017, 01:32:25 AM »
Thanks Hutch! It's encouraging to know that the failure is not  totally my fault  :biggrin:

The Iczelion's example it's a complete editor, only very simple. Would be nice Iczelion's colouring in qEditor, just take in count if someday you rework qEditor. For years I used Notepad++. At that time there was no IDE availables that worth the price, and Notepad++ was in the middle between a simple editor and a IDE. For complex projects I'm using RadAsm (and there are others very good IDE for that). For some time now, I prefer qEditor options for single files, and it's the default application for *.asm, *.inc, *.txt, *.ini and some others.  :t   

Later: Notepad++
   
« Last Edit: August 13, 2017, 01:48:33 AM by HSE »

mineiro

  • Member
  • ***
  • Posts: 365
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #7 on: August 12, 2017, 09:40:20 AM »
That tutorial techniquè acts like a transparent frame in front of text, you can reproduce that effect if you change text size by control+mouse whell, the font being used need be sync with text+frame.

I have tried gtk+ and sounds good, easy, well documented, they separated text buffer and text viewer. The thing is based on markup language, supports utf8 and many string conversion routines. I have also an idea about that on decomposed unicode chars, I don't have seen text editors saving unicode chars on decomposed way. I think this can be a solution not only to utf8/16.... encode way but others that can born on future.
Imagine that you're reading a text and you markup all foreginners words on that text, like per example latim words being italics. This way, we can search words on that text not by finding words but styles, and you can remove or list all latim words from that text. Other idea was about accents, but I lost my libid on that moment when I feel that this need be coded. We cannot search for accents only, we actually need search for 'áéíóú' instead of only accents. Well, If I'm a cyrillic writer today I probably adopt utf8 as jj2007 told, or utf16 like aw27. I searched and have not only utf encode, on Asia they are using another encode way.
I'd rather be this ambulant metamorphosis than to have that old opinion about everything

hutch--

  • Administrator
  • Member
  • ******
  • Posts: 4924
  • Mnemonic Driven API Grinder
    • The MASM32 SDK
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #8 on: August 12, 2017, 11:18:54 AM »
The problem is with editors for code is that code has no reason to be written in unicode which generates the problem for folks whose native language cannot be represented using a 256 characters set. Now you can go the way of writing the source in unicode and converting it to ascii to compile or assemble it or you can vertically split the display so that the code is in ascii and the side comments are in unicode. This would be a synchronised pair of edit windows with a splitter bar down the centre.
hutch at movsd dot com
http://www.masm32.com    :biggrin:  :biggrin:

nidud

  • Member
  • *****
  • Posts: 1408
    • https://github.com/nidud/asmc
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #9 on: August 12, 2017, 01:12:38 PM »
The problem is with editors for code is that code has no reason to be written in unicode which generates the problem for folks whose native language cannot be represented using a 256 characters set.

There's only a selected few languages that can't be represented using a 256 characters set. Russian is not one of them so it must be something else.

Quote
Now you can go the way of writing the source in unicode and converting it to ascii to compile or assemble it or you can vertically split the display so that the code is in ascii and the side comments are in unicode. This would be a synchronised pair of edit windows with a splitter bar down the centre.

The source code is written in ASCII so I think it safe to assume it looks and works correctly in qeditor, notepad and other IDE's.

I noticed from using the editor that some highlighted text has a different size than the background but judging from the image attached there seems to be a font problem.

Quote
The rich edit control uses Unicode internally, so this use of character sets differs from the original one used in font specifications. But the CHARFORMAT structure has a well-defined place for the character set.

However, there must be some different handling for the editor than for the highlighted text given the background is correctly displayed (I assume), so changing the font may not fix it.

You may try to manually convert the strings and see if that helps.

Code: [Select]
if 1
invoke lstrlen,edi
lea edx,wbuffer
mov word ptr [edx+eax*2],0
invoke MultiByteToWideChar,866,0,edi,eax,edx,eax
invoke DrawTextW,hdc,addr wbuffer,-1,addr rect,0
else
invoke DrawText,hdc,edi,-1,addr rect,0
endif

nidud

  • Member
  • *****
  • Posts: 1408
    • https://github.com/nidud/asmc
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #10 on: August 12, 2017, 01:46:14 PM »
I noticed from using the editor that some highlighted text has a different size than the background but judging from the image attached there seems to be a font problem.
...
However, there must be some different handling for the editor than for the highlighted text given the background is correctly displayed (I assume), so changing the font may not fix it.

So, the font used by the highlighted text is not the same as the background. The original font must then have been created by using CreateFont() or a similar function returning a handle.

Code: [Select]
NewRichEditProc proc hWnd:DWORD, uMsg:DWORD, wParam:DWORD, lParam:DWORD
LOCAL hdc:DWORD
LOCAL hOldFont:DWORD

Code: [Select]
invoke SelectObject,hdc,hOldRgn
invoke DeleteObject,hRgn
invoke SelectObject,hdc,hOldFont

The SelectObject function selects an object into the specified device context (DC). The new object replaces the previous object of the same type. hOldFont is not initialized so maybe that's the problem?

aw27

  • Member
  • ****
  • Posts: 852
  • Let's Make ASM Great Again!
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #11 on: August 12, 2017, 02:03:53 PM »
Well, If I'm a cyrillic writer today I probably adopt utf8 as jj2007 told, or utf16 like aw27.
Actually, I never proposed to save source code as UTF16 and I am firmly against that.  :icon_eek:
Probably what I said was that the assembler must be able to generate UTF16  at assembly time (I am not talking about adding a 0 to an ASCII character) and it can only do that if the source code is saved as UTF8 (or other Unicode form, like UTF16, of course) when it contains characters not available in the local encoding schema.

This is actually what  compilers like Visual Studio do. When you start writing a program in VS it assumes your local character encoding, be it Latin-1, Greek or Shift JIS.
As soon as you type a character not available in the local encoding schema and save the source, VS will alert you (with a message box) that the source must be saved in UTF8 to preserve the integrity.



aw27

  • Member
  • ****
  • Posts: 852
  • Let's Make ASM Great Again!
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #12 on: August 12, 2017, 02:19:17 PM »
Notepad was in the middle between a simple editor and a IDE.
Notepad is much lower than middle, you can not descend more.
Don't confuse Notepad with Notepad++, which is a true powerful "simple editor". Note the contradiction between  powerful and  simple.

minor28

  • Member
  • **
  • Posts: 109
    • Minor28 DivDev
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #13 on: August 12, 2017, 04:53:31 PM »
A few years ago I wrote a project based on richedit control (msftedit.dll).

Source code at https://minor28.divdev.se/
This and that on my site

jj2007

  • Member
  • *****
  • Posts: 7734
  • Assembler is fun ;-)
    • MasmBasic
Re: Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting
« Reply #14 on: August 12, 2017, 06:25:07 PM »
A few years ago I wrote a project based on richedit control (msftedit.dll).

Looks nice but I encountered a problem: When I saved an asm source as rtf, and tried to reopen it, I had a blank page in front of me. Toolbar, menus etc all ok but no document. Then, after a while, the message below popped up, but at a moment when the editor was no longer an active process. Now, when I keep trying, the editor opens fine but doesn't open any documents.

This is Win7-64. Weird ::)