Iczelion Tutorial 35: RichEdit Control: Syntax Hilighting

[Mikl__]

Tutorial 35: RichEdit Control: Syntax Hilighting
Hi, all!
I collected an application from 35-th Iczelion lesson and found a strange and unpleasant mistake. If the comments are not Latin letters (in Russian Cyrillic is used), then everything looks like 00.png
In the attachment, the source text and exe-/rc-files from the site Iczelion https://win32assembly.programminghorizon.com/tut35.html
Tell me how can fix this error?
There is text with russian comments in tut_35.zip

[aw27]

I don't know what nice and color rich editor you are using but I can see it very well with Notepad ++ and selecting one of the Cyrillic charsets.

StreamInProc proc hFile:QWORD,pBuffer:QWORD, NumBytes:QWORD, pBytesRead:QWORD
; Оo yЛoКВaМХЧ ЙoД ОpeДБopЪeРcЪ enter,40h,0 ОoЩРoЛy rbp=rsp+40h   
   and qword ptr [rbp-20h],0; mov qword ptr [rsp+20h],0
   invoke ReadFile; ,hFile,pBuffer,NumBytes,pBytesRead,0
   xor eax,1
   leave
   retn
StreamInProc endp
;--------------------------------
StreamOutProc proc hFile:QWORD,pBuffer:QWORD, NumBytes:QWORD, pBytesWritten:QWORD
; Оo yЛoКВaМХЧ ЙoД ОpeДБopЪeРcЪ enter,40h,0 ОoЩРoЛy rbp=rsp+40h   
   and qword ptr [rbp-20h],0; mov qword ptr [rsp+20h],0
   invoke WriteFile; ,hFile,pBuffer,NumBytes,pBytesWritten,0
   xor eax,1
   leave
   retn
StreamOutProc endp

[jj2007]

Here is a little converter from Russian (codepage 1251) to UTF8. Just drag your source over the attached exe.

include \masm32\MasmBasic\MasmBasic.inc
  Init
  Let esi=Utf8$(wCL$())  ; Unicode file names are allowed
  Let edi=Left$(esi, Instr_(esi, ".")-1)+"converted.asm"
  Open "O", 1, edi
  Print #1, Utf8Bom
  Recall esi, L$()
  For_ ecx=0 To eax-1
        PrintLine #1, ConvertCp$(L$(ecx), 1251, CP_UTF8)
  Next
  Close
  ShEx edi
EndOfCode

Code Select Expand

; пo yмoлчaнию кoд пpeдвopяeтcя enter,40h,0 пoэтoмy rbp=rsp+40h
; зaпoлняeм пoля cтpyктypы CHARFORMAT, чтoбы ycтaнoвить цвeт тeкcтa
...
;=============================================================
;  Кoгдa пoльзoвaтeль кликaeт нa oднoй из пaлитpы цвeтoв, зaпoлняeм пoля
;  cтpyктypы CHOOSECOLOR и вызывaeм диaлoгoвoe oкнo выбopa цвeтa ChooseColor.
;  Еcли пoльзoвaтeль выбиpaeт цвeт, тo этo знaчeниe colorref вoзвpaщaeтcя в
;  члeнe rgbResult, coхpaняeм этo знaчeниe в пepeмeннoй BackgroundColor.
;  Пocлe этoгo, мы вынyждaeм пepeкpaшивaниe нa пaлитpe цвeтoв, вызывaя
;  InvalidateRect нa дecкpиптop пaлитpы цвeтoв. Пaлитpa цвeтoв пocылaeт
;  WM_CTLCOLORSTATIC cooбщeниe poдитeльcкoмy oкнy
...
;==========================================
;  Кoгдa пoльзoвaтeль кликaeт нa oднoй из пaлитpы цвeтoв, вызывaeтcя диaлoгoвoe
;  oкнo выбopa цвeтa. "Пaлитpa цвeтoв" - cтaтичecкий элeмeнт yпpaвлeния c флaгoм
;  WS_BORDER и SS_NOTIFY. Стaтичecкий элeмeнт yпpaвлeния c флaгoм SS_NOTIFY
;  yвeдoмит poдитeльcкoe oкнo o дeйcтвиях мыши нa нeм, типa BN_CLICKED
;  (STN_CLICKED). Этo - yлoвкa.
;===========================================

[hutch--]

I remember doing a syntax colouring version back in about 2000 using Iczelion's technique and it worked well as it was only producing technicolor for the client area and not modifying the entire file. This made it fast and the normal capacity with richedit 2/3 to load very large files (>100 meg) FAST !!!!

I never followed it up because I just don't like technicolor in source code, I find it very distracting and much slower to read.

[HSE]

One or two years ago I tried to use this Iczelion's rutines in a plugin for qEditor. Of course not sucess at all :(
The almost sure thing is that I maked everything wrong  , but look like qEditor set colors after the entry point and a plugins can't to change colors. 

[hutch--]

You are right, it will not work in QE as it has its own colour control. If you wanted to try Iczelion's technique I would try it in a basic rich edit control that does not have its own colour control and it should work OK. Its been a long time ago but it was a fast technique and worked well.

[HSE]

Thanks Hutch! It's encouraging to know that the failure is not  totally my fault 

The Iczelion's example it's a complete editor, only very simple. Would be nice Iczelion's colouring in qEditor, just take in count if someday you rework qEditor. For years I used Notepad++. At that time there was no IDE availables that worth the price, and Notepad++ was in the middle between a simple editor and a IDE. For complex projects I'm using RadAsm (and there are others very good IDE for that). For some time now, I prefer qEditor options for single files, and it's the default application for *.asm, *.inc, *.txt, *.ini and some others.  :t   

Later: Notepad++
   

[mineiro]

That tutorial techniquè acts like a transparent frame in front of text, you can reproduce that effect if you change text size by control+mouse whell, the font being used need be sync with text+frame.

I have tried gtk+ and sounds good, easy, well documented, they separated text buffer and text viewer. The thing is based on markup language, supports utf8 and many string conversion routines. I have also an idea about that on decomposed unicode chars, I don't have seen text editors saving unicode chars on decomposed way. I think this can be a solution not only to utf8/16.... encode way but others that can born on future.
Imagine that you're reading a text and you markup all foreginners words on that text, like per example latim words being italics. This way, we can search words on that text not by finding words but styles, and you can remove or list all latim words from that text. Other idea was about accents, but I lost my libid on that moment when I feel that this need be coded. We cannot search for accents only, we actually need search for 'áéíóú' instead of only accents. Well, If I'm a cyrillic writer today I probably adopt utf8 as jj2007 told, or utf16 like aw27. I searched and have not only utf encode, on Asia they are using another encode way.

[hutch--]

The problem is with editors for code is that code has no reason to be written in unicode which generates the problem for folks whose native language cannot be represented using a 256 characters set. Now you can go the way of writing the source in unicode and converting it to ascii to compile or assemble it or you can vertically split the display so that the code is in ascii and the side comments are in unicode. This would be a synchronised pair of edit windows with a splitter bar down the centre.

[nidud]

deleted

[nidud]

deleted

[aw27]

Well, If I'm a cyrillic writer today I probably adopt utf8 as jj2007 told, or utf16 like aw27.

Actually, I never proposed to save source code as UTF16 and I am firmly against that.  :icon_eek:
Probably what I said was that the assembler must be able to generate UTF16  at assembly time (I am not talking about adding a 0 to an ASCII character) and it can only do that if the source code is saved as UTF8 (or other Unicode form, like UTF16, of course) when it contains characters not available in the local encoding schema.

This is actually what  compilers like Visual Studio do. When you start writing a program in VS it assumes your local character encoding, be it Latin-1, Greek or Shift JIS.
As soon as you type a character not available in the local encoding schema and save the source, VS will alert you (with a message box) that the source must be saved in UTF8 to preserve the integrity.

[aw27]

Notepad was in the middle between a simple editor and a IDE.

Notepad is much lower than middle, you can not descend more.
Don't confuse Notepad with Notepad++, which is a true powerful "simple editor". Note the contradiction between  powerful and  simple.

[minor28]

A few years ago I wrote a project based on richedit control (msftedit.dll).

Source code at https://minor28.divdev.se/

[jj2007]

A few years ago I wrote a project based on richedit control (msftedit.dll).

Looks nice but I encountered a problem: When I saved an asm source as rtf, and tried to reopen it, I had a blank page in front of me. Toolbar, menus etc all ok but no document. Then, after a while, the message below popped up, but at a moment when the editor was no longer an active process. Now, when I keep trying, the editor opens fine but doesn't open any documents.

This is Win7-64. Weird ::)