lpCustColors element in CHOOSECOLOR struct

[bluedevil]

Hello =)

You can see CHOOSECOLOR struct in our latest MASM64 SDK Beta2

Code Select Expand


  CHOOSECOLORA STRUCT QWORD
    lStructSize     dd ?
    hwndOwner       dq ?
    hInstance       dq ?
    rgbResult       dd ?
    lpCustColors    dd ?
    Flags           dd ?
    lCustData       dq ?
    lpfnHook        dq ?
    lpTemplateName  dq ?
  CHOOSECOLORA ENDS

  CHOOSECOLORW STRUCT QWORD
    lStructSize     dd ?
    hwndOwner       dq ?
    hInstance       dq ?
    rgbResult       dd ?
    lpCustColors    dd ?
    Flags           dd ?
    lCustData       dq ?
    lpfnHook        dq ?
    lpTemplateName  dq ?
  CHOOSECOLORW ENDS

  IFDEF __UNICODE__
    CHOOSECOLOR equ <CHOOSECOLORW>
  ELSE
    CHOOSECOLOR equ <CHOOSECOLORA>
  ENDIF


Pay attention to the lpCustColors element; it is DWORD. Now let's look in M$ docs :

Code Select Expand


typedef struct tagCHOOSECOLORA {
  DWORD        lStructSize;
  HWND         hwndOwner;
  HWND         hInstance;
  COLORREF     rgbResult;
  COLORREF     *lpCustColors;
  DWORD        Flags;
  LPARAM       lCustData;
  LPCCHOOKPROC lpfnHook;
  LPCSTR       lpTemplateName;
  LPEDITMENU   lpEditInfo;
} CHOOSECOLORA, *LPCHOOSECOLORA;


As you can see above lpCustColors element is a pointer to a dword value COLORREF. Also the name indicates that it is a long pointer.

@hutch-- we should update sdk and change lpCustColors type from dd to dq. Otherwise ChooseColor function does not work!

[hutch--]

Sorry to be slow but I have much to do.

Is this the form you are after ? I don't have a test piece set up to test it.

  CHOOSECOLORA STRUCT QWORD
    lStructSize dd ?
    hwndOwner dq ?
    hInstance dq ?
    rgbResult dd ?
    lpCustColors dq ?           ; pointer to lpCustColors
    Flags dd ?
    lCustData dq ?
    lpfnHook dq ?
    lpTemplateName dq ?
  CHOOSECOLORA ENDS

  CHOOSECOLORW STRUCT QWORD
    lStructSize dd ?
    hwndOwner dq ?
    hInstance dq ?
    rgbResult dd ?
    lpCustColors dq ?           ; pointer to lpCustColors
    Flags dd ?
    lCustData dq ?
    lpfnHook dq ?
    lpTemplateName dq ?
  CHOOSECOLORW ENDS

[bluedevil]

Exactly hutch, and you have shared the corrected ones 

[C3]

Hi,

Yes I can confirm that changing pointer to 64bit's works. Just figured out that and I didn't remember this post at time. After that calling ChooseColor works.


--
Petter

[hutch--]

What happened is that Vasily used my old 32 bit WINDOWS.INC file for his own 64 bit file but this one got past him. When I reused Vasily's include file, I missed it as well as I did not have a test piece to test this structure. Most of the rest has been fed through a multitude of test pieces to get them right and I added a lot of stuff to the include file.

Like any include file, there can always be more, especially as Microsoft keep adding things to their includes.

[C3]

Hello,

This change did not found it way to the newest MASM64 SDK. I were compiling code with the new SDK and found out that it need fixing.

[hutch--]

Sorry Petter,

This new release was aimed at the AV scanner problem, I had to rebuild almost all of the exe tools and then test them through Jotti and VirusTotal to get the false positive count down. If you still have the old file, use it as I have to get in front of all the other problems to do a real update.

[C3]

Ok,

I can still use that "m64sdk.exe" release. I took backup of the old masm64 folder. And I dont remember doing much changes there. And I maybe have to start gathering some API's that are not included already.

[caoenze]

This ChooseColor API troubled me for a day, and I finally found the answer. It turns out it was a data structure issue. Thank you very much!    

[C3]

This ChooseColor API troubled me for a day, and I finally found the answer. It turns out it was a data structure issue. Thank you very much!    

There is a bug in the Hutch's MASM64 SDK, the member lpCustColors needs to be QWORD in both CHOOSECOLORA and CHOOSECOLORW structures. I think I have the latest what Hutch made, but this bug slipped into it.

Ok the first post is already updated.