Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move UDP sockets to worker thread #306

Conversation

manekinekko
Copy link
Collaborator

@manekinekko manekinekko commented Dec 8, 2023

  • preview2_udp_bind
    • test_udp_bind_ephemeral_port(&net, IpAddress::IPV4_LOOPBACK);
    • test_udp_bind_ephemeral_port(&net, IpAddress::IPV6_LOOPBACK);
    • test_udp_bind_ephemeral_port(&net, IpAddress::IPV4_UNSPECIFIED);
    • test_udp_bind_ephemeral_port(&net, IpAddress::IPV6_UNSPECIFIED);
    • test_udp_bind_specific_port(&net, IpAddress::IPV4_LOOPBACK);
    • test_udp_bind_specific_port(&net, IpAddress::IPV6_LOOPBACK);
    • test_udp_bind_specific_port(&net, IpAddress::IPV4_UNSPECIFIED);
    • test_udp_bind_specific_port(&net, IpAddress::IPV6_UNSPECIFIED);
    • test_udp_bind_addrinuse(&net, IpAddress::IPV4_LOOPBACK);
    • test_udp_bind_addrinuse(&net, IpAddress::IPV6_LOOPBACK);
    • test_udp_bind_addrinuse(&net, IpAddress::IPV4_UNSPECIFIED);
    • test_udp_bind_addrinuse(&net, IpAddress::IPV6_UNSPECIFIED);
    • test_udp_bind_addrnotavail(&net, RESERVED_IPV4_ADDRESS);
    • test_udp_bind_addrnotavail(&net, RESERVED_IPV6_ADDRESS);
    • test_udp_bind_wrong_family(&net, IpAddressFamily::Ipv4);
    • test_udp_bind_wrong_family(&net, IpAddressFamily::Ipv6);
    • test_udp_bind_dual_stack(&net);
  • preview2_udp_connect
    • test_udp_connect_disconnect_reconnect(&net, IpAddressFamily::Ipv4);
    • test_udp_connect_disconnect_reconnect(&net, IpAddressFamily::Ipv6);
    • test_udp_connect_unspec(&net, IpAddressFamily::Ipv4);
    • test_udp_connect_unspec(&net, IpAddressFamily::Ipv6);
    • test_udp_connect_port_0(&net, IpAddressFamily::Ipv4);
    • test_udp_connect_port_0(&net, IpAddressFamily::Ipv6);
    • test_udp_connect_wrong_family(&net, IpAddressFamily::Ipv4);
    • test_udp_connect_wrong_family(&net, IpAddressFamily::Ipv6);
    • test_udp_connect_dual_stack(&net);
  • preview2_udp_sample_application
    • test_udp_sample_application(ipv4...)
    • test_udp_sample_application(ipv6...)
    • test_udp_dual_stack_conversation()
  • preview2_udp_sockopts
    • test_udp_sockopt_defaults(IpAddressFamily::Ipv4);
    • test_udp_sockopt_defaults(IpAddressFamily::Ipv6);
    • test_udp_sockopt_input_ranges(IpAddressFamily::Ipv4);
    • test_udp_sockopt_input_ranges(IpAddressFamily::Ipv6);
    • test_udp_sockopt_readback(IpAddressFamily::Ipv4);
    • test_udp_sockopt_readback(IpAddressFamily::Ipv6);
  • preview2_udp_states
    • test_udp_unbound_state_invariants(IpAddressFamily::Ipv4);
    • test_udp_unbound_state_invariants(IpAddressFamily::Ipv6);
    • test_udp_bound_state_invariants(&net, IpAddressFamily::Ipv4);
    • test_udp_bound_state_invariants(&net, IpAddressFamily::Ipv6);
    • test_udp_connected_state_invariants(&net, IpAddressFamily::Ipv4);
    • test_udp_connected_state_invariants(&net, IpAddressFamily::Ipv6);

@manekinekko manekinekko changed the title move UDP sockets to worker thread WIP: move UDP sockets to worker thread Dec 8, 2023
Copy link
Collaborator

@guybedford guybedford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thank you!

@manekinekko
Copy link
Collaborator Author

manekinekko commented Dec 12, 2023

40/41 test programs passing. The remaining test progran is failing because of this.

I had to skip some unit tests because we need to stub some of the socket async operations triggered in the worker thread.

@manekinekko manekinekko marked this pull request as ready for review December 12, 2023 15:49
@manekinekko manekinekko self-assigned this Dec 12, 2023
@manekinekko manekinekko force-pushed the chore/wasi-sockets-udp-worker-thread branch from 0cdbb96 to ec40864 Compare December 12, 2023 16:29
Comment on lines +132 to +168
export function getSocketOrThrow(socketId) {
const socket = openedSockets.get(socketId);
if (!socket) throw "invalid-state";
return socket;
}

export function getSocketByPort(port) {
return Array.from(openedSockets.values()).find((socket) => socket.address().port === port);
}

export function getBoundSockets(socketId) {
return Array.from(openedSockets.entries())
.filter(([id, _socket]) => id !== socketId) // exclude source socket
.map(([_id, socket]) => socket.address());
}

export function dequeueReceivedSocketDatagram(socketInfo, maxResults) {
const dgrams = queuedReceivedSocketDatagrams.get(`PORT:${socketInfo.port}`).splice(0, Number(maxResults));
return dgrams;
}
export function enqueueReceivedSocketDatagram(socketInfo, { data, rinfo }) {
const key = `PORT:${socketInfo.port}`;
const chunk = {
data,
rinfo, // sender/remote socket info (source)
socketInfo, // receiver socket info (targeted socket)
};

// create new queue if not exists
if (!queuedReceivedSocketDatagrams.has(key)) {
queuedReceivedSocketDatagrams.set(key, []);
}

// append to queue
const queue = queuedReceivedSocketDatagrams.get(key);
queue.push(chunk);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move all of these into worker-socket-udp.js.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. Will do.

return createFuture(createUdpSocket(addressFamily, reuseAddr));
}

case SOCKET_UDP_BIND: {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For any UDP call that is longer than a couple of lines, like this one, can we abstract it out into a function and instead call that function import from the worker-socket-udp.js file.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Will do.

}
}

case SOCKET_UDP_SEND: {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And for this one.

const exceptionInfo = {};
const value = this.#socket.bufferSize(0, BufferSizeFlags.SO_RCVBUF, exceptionInfo);
// TODO: should we throw if the socket is not bound?
// assert(this[symbolSocketState].isBound === false, "invalid-state");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps raise this on the sockets spec as a question?

test("tcp.connect(): should connect to a valid ipv4 address and port=0", async () => {
// test passing but blocked on socket.connect().
// TODO: figure out how to mock handle.connect()
test.skip("tcp.connect(): should connect to a valid ipv4 address and port=0", async () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For these tests that can't be easily mocked, it would be better to remove them from this unit test suite and instead just rely on the integration test coverage since that is sufficient at this point.

@guybedford guybedford merged commit 5981b3e into bytecodealliance:main Dec 13, 2023
6 checks passed
@guybedford
Copy link
Collaborator

I've gone ahead with the merge for now - let's follow up on the review feedback in follow-up PRs.

@manekinekko
Copy link
Collaborator Author

Great! Thanks @guybedford I will then move my local pending changes to a different PR.

@manekinekko manekinekko changed the title WIP: move UDP sockets to worker thread move UDP sockets to worker thread Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants