Custom draw controls in dialog: can't make it work

[NoCforMe]

(Somewhat long post here: you might want to skip it if your attention span is short ...)

I'm trying to add custom-draw functionality to a dialog. Specifically, I want to make a toolbar in the dialog a different color. Nothing fancy, just want to change the color scheme.

I turned to the custom draw documentation at MSDN. Couple problems here: the docs aren't exactly clear on how this all works, on top of which some of it is just plain wrong (or just plain doesn't work in Win32).  So I'm turning to folks here who hopefully know a lot more about this stuff.

Here's my understanding of how this should work:

I've created a dialog using an in-memory template, which works fine. Using DialogBoxIndirectParam() to open the dialog. First control in the dialog is a toolbar.

What I'm doing in the dialog proc, in pseudocode, is this:

Code Select Expand

case WM_NOTIFY:
   IF lParam->NMHDR.code = NM_CUSTOMDRAW
   AND lParam->NMHDR.idFrom = [my toolbar's control ID]
   AND lParam->NMCUSTOMDRAW.dwDrawStage = CDDS_PREPAINT
         SetWindowLong (hWin, DWL_MSGRESULT, CDRF_NOTIFYITEMDRAW)    ;Request further notifications
         return TRUE

   IF  lParam->NMHDR.code = NM_CUSTOMDRAW
   AND lParam->NMHDR.idFrom = [my toolbar's control ID]
   AND lParam->NMCUSTOMDRAW.dwDrawStage = CDDS_ITEMPREPAINT
          (try various stuff to change toolbar color)
         SetWindowLong (hWin, DWL_MSGRESULT, CDRF_DODEFAULT)    ;Let Windows draw the control
         return TRUE

I say "try various stuff" because that's what I did. Nothing seemed to work.

So again, my understanding of how this works is that the first time the notification comes in with the drawing stage equal to CDDS_PREPAINT, I return CDRF_NOTIFYITEMDRAW to let Windows know that I want to get further notifications of when items are painted or erased. Right? Then I look for those notifications (CDDS_ITEMPREPAINT) to actually do the custom drawing.

So here's what I see is available to me to try to change my toolbar's color:
In the NMCUSTOMDRAW structure, there's a handle to the control's device context. I tried using the DC:

Code Select Expand


SetBkColor (lParam->NMCUSTOMDRAW.hdc, $colorRed)


which didn't work. (Also tried creating a colored brush and selecting it into the DC; no luck.) In the NMTBCUSTOMDRAW structure (which by the way isn't in the MASM package--I had to add it by hand), there are a bunch of color specifiers (of type COLORREF, RGB values). I tried using these, replacing them with another color; no joy.

There are a couple of things about this whole process that are unclear to me:

First, there's the whole thing of sequencing, based on what "draw stage" you're in. Here are the stages:

CDDS_POSTERASE	After the erasing cycle is complete.
CDDS_POSTPAINT	After the painting cycle is complete.
CDDS_PREERASE	Before the erasing cycle begins.
CDDS_PREPAINT	Before the painting cycle begins.
CDDS_ITEMPOSTERASE	After an item has been erased.
CDDS_ITEMPOSTPAINT	After an item has been drawn.
CDDS_ITEMPREERASE	Before an item is erased.
CDDS_ITEMPREPAINT	Before an item is drawn.

My question is, when am I supposed to try to change the color here? Before or after erasing? before or after painting? What's not clear to me is what Windows is doing before or after any of these actions. In other words, which things that I can change (background color, text color, font, etc.) will survive whatever operations the API does after I change it?

The second thing I don't really understand is what I'm supposed to return (from my dialog procedure) after making changes. I understand that I need to do this through SetWindowLong():

Code Select Expand


    INVOKE   SetWindowLong, hWin, DWL_MSGRESULT, [return code]
    MOV        EAX, TRUE
    RET


Thing is, what do I return? Here are the return values I tried, from MSDN:

CDRF_DODEFAULT	The control will draw itself. It will not send additional NM_CUSTOMDRAW notification codes for this paint cycle. This flag cannot be used with any other flag.
CDRF_NEWFONT	Your application specified a new font for the item; the control will use the new font. For more information on changing fonts, see Changing fonts and colors. This occurs when dwDrawStage equals CDDS_ITEMPREPAINT.

(There are other choices, which you can see here. These are the only two that looked like they would do me any good.)

I did notice that when I returned CDRF_NEWFONT, it did change the font (in a bad way), but not the color. I tried this return value because I saw it used in a couple of code examples I found through MSDN, which supposedly worked.

If anyone can shed any light on this, I'd be grateful. It's not a huge deal--it's basically "chrome", not basic functionality, but it's bugging me that I can't make anything work here.

Oh, and I'm running XP, and I'm using "visual styles" (have a manifest). I haven't tried disabling this, but will do. Also am calling InitCommonControlsEx(). Oh, and the toolbar contains only text, no bitmaps.

Thanks in advance!

[fearless]

Some references that might be useful: https://www.codeproject.com/Articles/646482/Custom-Controls-in-Win-API-Control-Customization

Dont think there is a builtin TB_SETBKCOLOR message or a way of directly controlling that background color.

Couple options that i can think of are:

- Might be possible to place a child static control and host a bitmap or image so as to 'paint' the background of the toolbar or rebar control as long as they are transparent for this to work - unsure if it is feasible tbh.

- Use clrBack of rebar info structure and host the toolbars in the rebar - again not sure how feasible this is or if it would work.

- Other option is much like the links at the end of that article - wine using subclassing of rebar and other controls to get round these limitations.

- Create a toolbar control manually and duplicate the features of a toolbar and allow a way of custom painting the background of the toolbar.

[dedndave]

background color for buttons is a hard one to do
it's almost easier to write your own WndProc for a button class of your own

[NoCforMe]

Some references that might be useful: https://www.codeproject.com/Articles/646482/Custom-Controls-in-Win-API-Control-Customization

That is useful! The guy actually explains drawing stages, etc., in a very clear way. (Unfortunately, it still doesn't answer my original questions, but I'm still re-reading and absorbing it.) Thanks!

Another of his articles was immediately useful to me to eliminate flickering in another app (by suppressing WM_ERASEBKGND). Good stuff.

[NoCforMe]

background color for buttons is a hard one to do
it's almost easier to write your own WndProc for a button class of your own

Ackshooly, in this case it'd prolly be easier to lose the toolbar and just put some buttons in the dialog. I guess I could "fake" a toolbar by drawing a thin rectangle behind a row of buttons (or just use the toolbar as a parent for some button child windows--been there, done that, just can't use my dialog editor to place the buttons, they have to be put in "by hand" ...).

[fearless]

Resurrecting this post to add some additional information. Was looking into customdraw toolbar and how to paint the background of it, after some work in trying to figure some stuff out, I have this solution of sorts:

the toolbar requires the TBSTYLE_CUSTOMERASE style set when it is created. This will allow it to receive a CDDS_PREERASE during the custom draw. When handling that event you can override it and draw the background yourself.

Code Select Expand

CustomDrawToolbar PROC USES EBX hWin:DWORD, lParam:DWORD, bDialog:DWORD
    LOCAL hdc:DWORD
    LOCAL hBrush:DWORD
    LOCAL rect:RECT
   
    mov ebx, lParam
    mov eax, (NMTBCUSTOMDRAW PTR [ebx]).nmcd.dwDrawStage
   
    .IF eax == CDDS_PREPAINT
        .IF bDialog == TRUE
            mov eax, CDRF_NOTIFYITEMDRAW
            Invoke SetWindowLong, hWin, DWL_MSGRESULT, eax
            mov eax, TRUE
            ret
        .ELSE
            mov eax, CDRF_NOTIFYITEMDRAW
            ret
        .ENDIF

    .ELSEIF eax == CDDS_PREERASE ; requires style TBSTYLE_CUSTOMERASE
        mov ebx, lParam
        Invoke CopyRect, Addr rect, Addr (NMTBCUSTOMDRAW PTR [ebx]).nmcd.rc
   
        ;----------------------------------------------------------------------
        ; If rectangle is empty then no point doing anything
        ;----------------------------------------------------------------------
        Invoke IsRectEmpty, Addr rect
        .IF eax == TRUE
            .IF bDialog == TRUE
                mov eax, CDRF_DODEFAULT
                Invoke SetWindowLong, hWin, DWL_MSGRESULT, eax
                mov eax, TRUE
                ret
            .ELSE
                mov eax, CDRF_DODEFAULT
                ret
            .ENDIF
        .ENDIF
       
        ;----------------------------------------------------------------------
        ; If hdc is empty then no point doing anything
        ;----------------------------------------------------------------------
        mov ebx, lParam
        mov eax, (NMTBCUSTOMDRAW PTR [ebx]).nmcd.hdc
        .IF eax == 0
            .IF bDialog == TRUE
                mov eax, CDRF_DODEFAULT
                Invoke SetWindowLong, hWin, DWL_MSGRESULT, eax
                mov eax, TRUE
                ret
            .ELSE
                mov eax, CDRF_DODEFAULT
                ret
            .ENDIF
        .ENDIF
        mov hdc, eax
       
        Invoke SaveDC, hdc
        Invoke GetStockObject, DC_BRUSH
        mov hBrush, eax
        Invoke SelectObject, hdc, hBrush
        Invoke SetDCBrushColor, hdc, 37D040h ; = RGB 64,208,55
        Invoke FillRect, hdc, Addr rect, hBrush
        Invoke RestoreDC, hdc, -1
       
        .IF bDialog == TRUE
            mov eax, CDRF_SKIPDEFAULT
            Invoke SetWindowLong, hWin, DWL_MSGRESULT, eax
            mov eax, TRUE
            ret
        .ELSE
            mov eax, CDRF_SKIPDEFAULT
            ret
        .ENDIF
    .ENDIF
   
    .IF bDialog == TRUE
        mov eax, CDRF_DODEFAULT
        Invoke SetWindowLong, hWin, DWL_MSGRESULT, eax
        mov eax, TRUE
        ret
    .ELSE
        mov eax, CDRF_DODEFAULT
        ret
    .ENDIF
   
    ret
CustomDrawToolbar ENDP

You can also get the CDDS_ITEMPREPAINT event, but I couldnt get anything to change in that, except for clrHighlightHotTrack if you return TBCDRF_HILITEHOTTRACK as your result (either for a window or dialog) which changes the background as you hover over a button. So something like this:

Code Select Expand

.ELSEIF eax == CDDS_ITEMPREPAINT
    mov ebx, lParam
    mov (NMTBCUSTOMDRAW PTR [ebx]).clrHighlightHotTrack, RGB(197,218,237)
    mov eax, TBCDRF_HILITEHOTTRACK
    Invoke SetWindowLong, hWin, DWL_MSGRESULT, eax
    mov eax, TRUE
    ret


[fearless]

I should also add that if you use an imagelist and load icons into that, and set the toolbar to use the imagelist then the icons if they have any transparency will draw on top of the toolbar, and thus the buttons will take the background color filled in during the CDDS_PREERASE.