For some reason in my gdiplus includes for uasm, based on the WinInc includes some of those values where no higher than 2, so SmoothingModeAntiAlias was 2 instead of 4, the InterpolationModeHighQualityBicubic was 4 (or 2 i think) instead of 7 - anyways once i changed them it all appeared more crisp and smoother, so I clearly had some wrong values in them.
I managed to fix the other issues - turns out i was using:
Invoke GdipDrawImage, pGraphicsBuffer, hImage, 0, 0
And not supplying reals, just 0 for the last two parameters. When I changed it over to use I version it solved most of those weird drifting problems
Invoke GdipDrawImageI, pGraphicsBuffer, hImage, 0, 0
The other spinner problem, no 10 in the demo, which was only showing quarter of its image was down to not zeroing registers I think and maybe shifting on eax/ebx instead of rax/rbx before doing some calculations from rectangle values and image width and height, so now it works with:
xor rax, rax
xor rbx, rbx
mov eax, rect.right
shr rax, 1
mov rbx, qwImageWidth
shr rbx, 1
sub rax, rbx
.IF sqword ptr rax < 0
mov rax, 0
.ENDIF
mov pt.x, eax
xor rax, rax
xor rbx, rbx
mov eax, rect.bottom
shr rax, 1
mov rbx, qwImageHeight
shr rbx, 1
sub rax, rbx
.IF sqword ptr rax < 0
mov rax, 0
.ENDIF
mov pt.y, eax
Beforehand it was doing something like
mov eax, rect.right
shr eax, 1
mov rbx, qwImageWidth
shr ebx, 1
sub eax, ebx
.IF sqword ptr rax < 0
mov eax, 0
.ENDIF
mov pt.x, eax
Attached is the updated exe in the zip, which i hope is working now