It could be something like this:
c++:
extern "C" void CalculateDigest(byte* hash_value, const byte* data, size_t length) {
CryptoPP::MD5 hash;
hash.CalculateDigest( hash_value, data, length);
}MASM:
MD5 struct
bytes BYTE 16 dup (?)
MD5 ends
CalculateDigest proto C hash_value: ptr MD5,data:ptr BYTE, _length:DWORD
...
.data?
md5 MD5 <>
.data
foo BYTE "123456789"
.code
...
invoke CalculateDigest,ADDR md5,ADDR foo,SIZEOF foo