Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proof of concept] Async non-blocking IO using task executor preference #2648

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

FranzBusch
Copy link
Member

This a PoC PR mostly for educational reasons. Don't expect any more work on this but I wanted to showcase that it is possible and it might be interesting for others in the community.

Motivation

During the adoption of our new NIOAsyncChannel we came across a few issues such as:

  • The current bootstraps are not providing scoping to the underlying channel which doesn't work nicely with structured concurrency
  • Unnecessary buffering between the NIOAsyncChannel components and the underlying pipeline
  • Things like HTTP upgrades are inherently tied to the NIO pipeline; hence, they must use EventLoopFutures. This leads to API problems higher up the stack where we cannot spell it nicely with async primitives.

The first one can be fixed with new bootstraps but the last one is inherently tied to how our current networking stack works. After thinking about this more, I wanted to try if we could respell NIO with purely asynchronous primitives. This became possible via the newly introduced task executor preference feature that allows us to implement thread hops in normal async code.

Modification

This PR is mostly a showcase of how the very lowest level could work for a non-blocking networking stack that is written using concurrency primitives such as TaskExecutors and AsyncSequence. This PR shows three separate pieces working:

  1. An IOExecutor: TaskExecutor that spawns a pthread. This thread is running a loop to execute UnownedJobs and knows how to handle I/O via a selector (kqueue)
  2. A TCPConenction that uses the IOExecutor to handle non blocking reads and writes. It exposes them via an AsyncSequence and and AsyncWriter
  3. A TCPListener that can accept new inbound TCPConnections.

TODOs

  • Make this work in Linux (epoll)
  • Fix the CoWs inside the executor
  • Support Cancellation (only done for the listener right now)
  • Make sure we close everything correctly
  • Scoped IOExecutor access
  • Handle all events and make sure we handle them correctly

# Motivation

During the adoption of our new `NIOAsyncChannel` we came across a few issues such as:
- The current bootstraps are not providing scoping to the underlying channel which doesn't work nicely with structured concurrency
- Unnecessary buffering between the `NIOAsyncChannel` components and the underlying pipeline
- Things like HTTP upgrades are inherently tied to the NIO pipeline; hence, they must use `EventLoopFutures`. This leads to API problems higher up the stack where we cannot spell it nicely with async primitives.

The first one can be fixed with new bootstraps but the last one is inherently tied to how our current networking stack works. After thinking about this more, I wanted to try if we could respell NIO with purely asynchronous primitives. This became possible via the newly introduced task executor preference feature that allows us to implement thread hops in normal async code.

# Modification

This PR is mostly a showcase of how the very lowest level could work for a non-blocking networking stack that is written using concurrency primitives such as `TaskExecutor`s and `AsyncSequence`. This PR shows three separate pieces working:

1. An `IOExecutor: TaskExecutor` that spawns a `pthread`. This thread is running a loop to execute `UnownedJob`s and knows how to handle I/O via a selector (kqueue)
2. A `TCPConenction` that uses the `IOExecutor` to handle non blocking reads and writes. It exposes them via an `AsyncSequence` and and `AsyncWriter`
3. A `TCPListener` that can accept new inbound `TCPConnection`s.

# TODOs
- Make this work in Linux (epoll)
- Fix the CoWs inside the executor
- Support Cancellation (only done for the listener right now)
- Make sure we close everything correctly
- Scoped IOExecutor access
- Handle all events and make sure we handle them correctly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant