Well the obvious is that a 128 bit register and a 64 bit integer register are different animals and the two should not be confused. If an API expects an XMM register, that is what you pass it, if instead it expects a 64 bit memory operand or a 64 bit register, either a REAL8 or a QWORD, then you pass the 64 bit format.
I mentioned the obvious, a QWORD and a REAL8 fit the same hole and you can successfully pass a return value in RAX as long as you load it into a REAL8 memory operand for further processing. We are not dealing with strongly typed compilers here, we are dealing with an assembler.