Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stability improvements for multiple clients #741

Closed
wants to merge 3 commits into from
Closed

Stability improvements for multiple clients #741

wants to merge 3 commits into from

Conversation

p0rys
Copy link
Contributor

@p0rys p0rys commented May 11, 2022

Public API Changes
None

Description
This changes will improve the stability when multiple clients are connected simultaneously.

Fix 1 (07bd9b8): Changes "has_subscribers" to return the combined count of subscriptions and new subscriptions . This will avoid unregister of subscribers (see line 284-285) if one client disconnects while all others are still waiting for the initial callback (there are still in "new_subscriptions" and not in "subscriptions").

Fix 2 (227bfbe): Removes a possible deadlock if a client is unsubscribed before it receives the initial callback. In that case "self.new_subscriptions.values()" passed as "callbacks" would be empty and trigger a second "self.lock" (line 209) while still in the lock from line 232.

@zflat
Copy link
Contributor

zflat commented May 20, 2022

I believe this fixes the deadlock that I observed in #744
I can reproduce my issue on ros2 commit 8a705de
Then I switch to p0rys:multiple_clients_fixes commit 1c71aee and I do not see the same deadlock issue.

Thanks @p0rys for this fix!

Copy link
Contributor

@zflat zflat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note and not related to this bug fix but the instance member self.lock is not very descriptive about what is being locked. It would be better named as self.subscriptions_lock so it is more clear that this lock is needed every time the self.subscriptions and self.new_subscriptions members are accessed. Or put another way, the locking logic is kinda complex in this class and it would be nice if it was less error prone.

Anyway these changes look good to me.

@TheEppicJR
Copy link

CI for Rolling was updated to Ubuntu 22.04 and should work now

@zflat
Copy link
Contributor

zflat commented Jun 3, 2022

@p0rys What do you think about changing Lock to RLock (https://docs.python.org/3/library/threading.html#threading.RLock) as an alternative way to prevent deadlocks than the changes in 227bfbe

@p0rys
Copy link
Contributor Author

p0rys commented Jun 7, 2022

@zflat Seems reasonable for this problem but i'm far from a python expert. Also i would rather implement #426 then putting more effort into this fix.

@Alqio
Copy link

Alqio commented Jul 25, 2022

This fixes #758

Copy link
Contributor

@achim-k achim-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR avoids the deadlock issue, but it causes some side effects such as existing clients receiving messages twice when a new client subscribes to the topic.

@p0rys do you have time address my comments? Otherwise I can do that quickly myself

Comment on lines +140 to +141
if self.new_subscriber:
self.node_handle.destroy_subscription(self.new_subscriber)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be done inside the with self.lock block as the assignment of self.new_subscriber is also protected by the lock

Comment on lines +240 to +241
# send msg to client
self.callback(msg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would effectively cause existing clients to receive messages twice when a new client connects.
I think it would be better to leave it how it was and instead use an RLock to avoid the potential deadlock.

@achim-k
Copy link
Contributor

achim-k commented Oct 24, 2022

Superseded by #803

@achim-k achim-k closed this Oct 24, 2022
hello-vinitha added a commit to hello-vinitha/rosbridge_suite that referenced this pull request Oct 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants