Adventures in Winsock 2.2

[Raistlin]

Hi all, I thought I'd post in the campus..  :icon_confused:

So I was looking around the Winsock 2.2 specifications and found "Winsock Direct" (WSDP) (circa windows 2003 data center)
a very rare (not much on the net about it) API targeting surprisingly similar technology/terminology
in later research - ex. RDMA, SPI, SMB2.0, SAN, iWarp, RoCE, RNDIS, Infiniband 

Essentially I believe, it boils down to trying to implement zero copy / low latency / asynchronous / kernel bypassing / large block size communications
for peer to peer data transfers. The network card manufacturer could supply a system area network (SAN) service provider interface (SPI)
which could be used to bypass standard winsock TCP/IP network communications. Or encapsulate inside TCP/IP but not incur CPU overhead.
The bypass is hidden from the application layer, therefore no modification to existing applications using winsock is required.

Does anyone know if NIC manufacturers implemented SAN/SPI generically ?
Has anyone played around with Winsock Direct or have experience avoiding it ?

Thanks
Raistlin

[Raistlin]

Ok - So I guess no-one has been playing around Winsock lately. :(

So due diligence dictates that I provide you with some of my research findings in this regard - education is key afterall.
My original motivation stems form trying to gain speed-ups on standard sockets.

The lecture:
Winsock Direct was a stillborn endeavor, however its framework survived in the form of SAN / SPI + Windows Filtering Platform + inter-process communication (IPC),
which still provides the ability for RDMA (remote direct memory access) over specialized network fabrics. Other key terms : zero copy, large segment offload

Specialized Network Fabric = special NICS + special NIC Driver + special Switches + special Cable + only certain Windows Server/Wrkst Kernels
(also noteworthy that the speed-ups have a distance component and administrative component = you need full autonomy (rights) in a localized cluster setting)

Here's some interesting articles on the subject :
https://community.netapp.com/t5/Technology/State-of-RDMA-There-can-only-be-one/ba-p/83325
https://blogs.technet.microsoft.com/wincat/2012/12/05/fast-tcp-loopback-performance-and-low-latency-with-windows-server-2012-tcp-loopback-fast-path/

The only REAL (non-hardware/specialized) method for Winsock speedups, that I could find anyway, is using
hardware threading + non-blocking asynchronous code + dedicated call-back functions + due regard for the network MTU.

If you know any better PLEASE let me know (PM or otherwise) 

[LiaoMi]

Hi Raistlin,

TCP/IP Raw Sockets - In computer networking, a raw socket is an internet socket that allows direct sending and receiving of Internet Protocol packets without any protocol-specific transport layer formatting.
https://msdn.microsoft.com/en-us/library/windows/desktop/ms740548(v=vs.85).aspx

maybe it will be faster, if not enough speed, you can try and you have to write a driver, usually the best solution is multithreading

https://en.wikipedia.org/wiki/Transport_Driver_Interface

[Raistlin]

Thanks Liaomi , unfortunately raw sockets suffer
a fatal flaw: from your link - requires admin rights to create.
perhaps not full administrative...will check.
I am trying to avoid the whole driver = ring zero ++++ thing

[Raistlin]

Hi again,

I found the zero copy Microsoft equivalent right under my nose.  :icon_redface:

WinSock API = TransmitFile supported since NT 3.51 kernel - some restrictions
for earlier Windows versions - re: 2 concurrent max connections - but will do nicely
for my purposes.

I'll keep updating this post if no-one shuts me up, with more socket optimization strategies

Keep the suggestions coming  :t
Raistlin

[jj2007]

MSDN:

The TransmitFile function transmits file data over a connected socket handle

Minimum supported client Windows 8.1

:(

Win7:

Code Select Expand

TransmitFile      8BFF                mov edi, edi
71E5C822          55                  push ebp
71E5C823          8BEC                mov ebp, esp

Could be handy to send files over the router to my second machine... ::)

[Raistlin]

@JJ2007:

Workstation and client versions of Windows optimize the TransmitFile function for minimum memory and resource utilization by limiting the number of concurrent TransmitFile operations allowed on the system to a maximum of two. On Windows Vista, Windows XP, Windows 2000 Professional, and Windows NT Workstation 3.51 and later only two outstanding TransmitFile requests are handled simultaneously; the third request will wait until one of the previous requests is completed.

https://msdn.microsoft.com/en-us/library/windows/desktop/ms740565(v=vs.85).aspx

[jj2007]

So TransmitFile is a dinosaur. MSDN gets more and more ambiguous about "minimum supported client" ;-)

[Raistlin]

@jj2007 - Yes I've noticed that more and more these days - seems if they discontinue support on a Windows version,
               then the next still-supported version becomes the minimum  especially around winsock & IPC APIs.

[Siekmanski]

So TransmitFile is a dinosaur. MSDN gets more and more ambiguous about "minimum supported client" ;-)

It's like in politics, keep the people unaware of the facts, by telling half the truth or complete lies.  :icon13:

[jj2007]

Some time ago, Eugen posted a chat example (attached). I've tested it between my two machines connected via a router but can't get it to work. Probably my complete ignorance in network issues. Can anybody get it to work, and explain how?

[aw27]

Once, more than 10 years ago, I had a software that used a Layered Service Provider to capture traffic. Although the program was  selling, I discontinued it because Microsoft decided to deprecate the LSP technology for Windows Vista. The alternative was and still is the Windows Filtering Platform.
May be I will resurrect all this in 2018.  :idea:

[jj2007]

Interesting comparison: WinINet vs WinHTTP vs Winsock