Author Topic: Adventures in Winsock 2.2  (Read 397 times)

Raistlin

  • Member
  • **
  • Posts: 246
Adventures in Winsock 2.2
« on: October 13, 2017, 05:11:13 PM »
Hi all, I thought I'd post in the campus..  :icon_confused:

So I was looking around the Winsock 2.2 specifications and found "Winsock Direct" (WSDP) (circa windows 2003 data center)
a very rare (not much on the net about it) API targeting surprisingly similar technology/terminology
in later research - ex. RDMA, SPI, SMB2.0, SAN, iWarp, RoCE, RNDIS, Infiniband 

Essentially I believe, it boils down to trying to implement zero copy / low latency / asynchronous / kernel bypassing / large block size communications
for peer to peer data transfers. The network card manufacturer could supply a system area network (SAN) service provider interface (SPI)
which could be used to bypass standard winsock TCP/IP network communications. Or encapsulate inside TCP/IP but not incur CPU overhead.
The bypass is hidden from the application layer, therefore no modification to existing applications using winsock is required.

Does anyone know if NIC manufacturers implemented SAN/SPI generically ?
Has anyone played around with Winsock Direct or have experience avoiding it ?

Thanks
Raistlin
« Last Edit: October 16, 2017, 04:34:21 PM by Raistlin »

Raistlin

  • Member
  • **
  • Posts: 246
Re: Adventures in Winsock 2.2
« Reply #1 on: October 30, 2017, 04:50:45 PM »
Ok - So I guess no-one has been playing around Winsock lately. :(

So due diligence dictates that I provide you with some of my research findings in this regard - education is key afterall.
My original motivation stems form trying to gain speed-ups on standard sockets.

The lecture:
Winsock Direct was a stillborn endeavor, however its framework survived in the form of SAN / SPI + Windows Filtering Platform + inter-process communication (IPC),
which still provides the ability for RDMA (remote direct memory access) over specialized network fabrics. Other key terms : zero copy, large segment offload

Specialized Network Fabric = special NICS + special NIC Driver + special Switches + special Cable + only certain Windows Server/Wrkst Kernels
(also noteworthy that the speed-ups have a distance component and administrative component = you need full autonomy (rights) in a localized cluster setting)

Here's some interesting articles on the subject :
https://community.netapp.com/t5/Technology/State-of-RDMA-There-can-only-be-one/ba-p/83325
https://blogs.technet.microsoft.com/wincat/2012/12/05/fast-tcp-loopback-performance-and-low-latency-with-windows-server-2012-tcp-loopback-fast-path/

The only REAL (non-hardware/specialized) method for Winsock speedups, that I could find anyway, is using
hardware threading + non-blocking asynchronous code + dedicated call-back functions + due regard for the network MTU.

If you know any better PLEASE let me know (PM or otherwise)  :bgrin:

LiaoMi

  • Member
  • **
  • Posts: 155
Re: Adventures in Winsock 2.2
« Reply #2 on: October 30, 2017, 09:25:01 PM »
Hi Raistlin,

TCP/IP Raw Sockets - In computer networking, a raw socket is an internet socket that allows direct sending and receiving of Internet Protocol packets without any protocol-specific transport layer formatting.
https://msdn.microsoft.com/en-us/library/windows/desktop/ms740548(v=vs.85).aspx

maybe it will be faster, if not enough speed, you can try and you have to write a driver, usually the best solution is multithreading

https://en.wikipedia.org/wiki/Transport_Driver_Interface


Raistlin

  • Member
  • **
  • Posts: 246
Re: Adventures in Winsock 2.2
« Reply #3 on: October 31, 2017, 12:54:12 AM »
Thanks Liaomi , unfortunately raw sockets suffer
a fatal flaw: from your link - requires admin rights to create.
perhaps not full administrative...will check.
I am trying to avoid the whole driver = ring zero ++++ thing

Raistlin

  • Member
  • **
  • Posts: 246
Re: Adventures in Winsock 2.2
« Reply #4 on: November 01, 2017, 04:37:40 PM »
Hi again,

I found the zero copy Microsoft equivalent right under my nose.  :icon_redface:

WinSock API = TransmitFile supported since NT 3.51 kernel - some restrictions
for earlier Windows versions - re: 2 concurrent max connections - but will do nicely
for my purposes.

I'll keep updating this post if no-one shuts me up, with more socket optimization strategies

Keep the suggestions coming  :t
Raistlin

jj2007

  • Member
  • *****
  • Posts: 7641
  • Assembler is fun ;-)
    • MasmBasic
Re: Adventures in Winsock 2.2
« Reply #5 on: November 01, 2017, 08:34:20 PM »
MSDN:
Quote
The TransmitFile function transmits file data over a connected socket handle
:biggrin:

Quote
Minimum supported client Windows 8.1
:(

Win7:
Code: [Select]
TransmitFile      8BFF                mov edi, edi
71E5C822          55                  push ebp
71E5C823          8BEC                mov ebp, esp
:biggrin:

Could be handy to send files over the router to my second machine... ::)

Raistlin

  • Member
  • **
  • Posts: 246
Re: Adventures in Winsock 2.2
« Reply #6 on: November 01, 2017, 08:46:48 PM »
@JJ2007:
Quote
Workstation and client versions of Windows optimize the TransmitFile function for minimum memory and resource utilization by limiting the number of concurrent TransmitFile operations allowed on the system to a maximum of two. On Windows Vista, Windows XP, Windows 2000 Professional, and Windows NT Workstation 3.51 and later only two outstanding TransmitFile requests are handled simultaneously; the third request will wait until one of the previous requests is completed.


https://msdn.microsoft.com/en-us/library/windows/desktop/ms740565(v=vs.85).aspx

jj2007

  • Member
  • *****
  • Posts: 7641
  • Assembler is fun ;-)
    • MasmBasic
Re: Adventures in Winsock 2.2
« Reply #7 on: November 01, 2017, 09:01:37 PM »
So TransmitFile is a dinosaur. MSDN gets more and more ambiguous about "minimum supported client" ;-)

Raistlin

  • Member
  • **
  • Posts: 246
Re: Adventures in Winsock 2.2
« Reply #8 on: November 01, 2017, 09:17:18 PM »
@jj2007 - Yes I've noticed that more and more these days - seems if they discontinue support on a Windows version,
               then the next still-supported version becomes the minimum  :badgrin: especially around winsock & IPC APIs.

Siekmanski

  • Member
  • *****
  • Posts: 1111
Re: Adventures in Winsock 2.2
« Reply #9 on: November 02, 2017, 02:05:11 AM »
So TransmitFile is a dinosaur. MSDN gets more and more ambiguous about "minimum supported client" ;-)

It's like in politics, keep the people unaware of the facts, by telling half the truth or complete lies.  :icon13:

jj2007

  • Member
  • *****
  • Posts: 7641
  • Assembler is fun ;-)
    • MasmBasic
Re: Adventures in Winsock 2.2
« Reply #10 on: November 07, 2017, 10:55:50 PM »
Some time ago, Eugen posted a chat example (attached). I've tested it between my two machines connected via a router but can't get it to work. Probably my complete ignorance in network issues. Can anybody get it to work, and explain how?

aw27

  • Member
  • ****
  • Posts: 788
Re: Adventures in Winsock 2.2
« Reply #11 on: November 11, 2017, 01:55:48 AM »
Once, more than 10 years ago, I had a software that used a Layered Service Provider to capture traffic. Although the program was  selling, I discontinued it because Microsoft decided to deprecate the LSP technology for Windows Vista. The alternative was and still is the Windows Filtering Platform.
May be I will resurrect all this in 2018.  :idea: