.net SDK 3.0: Will it use async / await / thread pool?

#1

One of the oddest things about the current .net SDK is the use of discreet threads (lots of them) within the connection pool. The sheer amount of context switching occurring as a result of this design choice must be significant in high load scenarios such as our import platform. I am curious to know if the 3.X branch plans on using a thready pool or the Task mechanism along with async await to reduce the use of discreet threads?

#2

@drGarbinsky1 -

I am not sure I understand the question, there are no threads defined specifically within the connection pool. Any thread can use a connection and these are all based off the threadpool if you use the async methods which exist in 2.x.

You may be referring to connections themselves which have a dedicated read thread; however, any thread can write. This is per connection the default for MinSize is 1 and for MaxSize its 2, so by default its low and in general you should be able to keep the two settings fairly low.

-Jeff

#3

So the dedicated read threads is what I am referring to. The connection count is also multiplied by the number of data service nodes. So in our bulk import environment we use higher connection pool limits (20 - 50) and we have 4 data service nodes. So we see thread counts above 300. So there is also a high degree of context switching. It is not clear why a thread needs to camp on a socket waiting for data. Seems like it would be worth the investigation to move away from the current approach and move to some thread pool implementation.

#4

TBH, your likely using too many connections. More connections doesn’t mean higher performance or more thoroughput, its tied to network speed, disk speed, etc. as well. Adding more connections means more resources for sure - mostly tied to memory. How the client does IO is intended to get the best overall performance while using the least amount of resources. I would tune way down starting with the default settings (MinSize/MaxSize of 1 and 2) and adjust from there.

The default IO supports pipelining multiple requests over a single socket, its an optimization to minimize the number of sockets that need to be used. At any one time multiple threads are writing to a socket; reads are dedicated when a socket is not being used for writes. Other solutions involve sending a request and waiting for a response, this is inefficiant.

Moving reads to the thread pool can definitly be done, the thread would off the thread pool, but still dedicated and used for the lifetime of the socket. There might end up being some thread pool starvation issues that require performance tuning since the thread isn’t returned immediatly to the pool in some situations (large cluster/large MaxSize). This is currently actually done in the SDK 3 branch (master) but we haven’t done any performance measuring.

All that being said, the connection pool and the connections themselves are open to implementation and swappable via configuration. Besides the default MultiplexingConnection, you can revert back to an older blocking Connection, or your interested, you can implement your own IConnection - if you do, feel free to share it by pushing a PR :slight_smile:

-Jeff