Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of path functions on cache miss #1443

Merged
merged 8 commits into from
Dec 1, 2024

Conversation

bdraco
Copy link
Member

@bdraco bdraco commented Dec 1, 2024

There were a few places we called the quoter when we know its never going to do anything.

On cache miss for .query_string, this will help a bit. However its not possible to benchmark right now because of #1443 (comment)

We can avoid some of the property accesses internally as well which should speed these up a bit. Since the aiohttp url dispatcher calls path_safe there is a much larger chance we get a cache miss so it makes sense to optimize this one a bit.

Copy link

codspeed-hq bot commented Dec 1, 2024

CodSpeed Performance Report

Merging #1443 will improve performances by 84.24%

Comparing skip_quoting_empty_and_slash (1713ef4) with master (7585a96)

Summary

⚡ 7 improvements
✅ 91 untouched benchmarks

Benchmarks breakdown

Benchmark master skip_quoting_empty_and_slash Change
test_empty_path_safe_uncached 166 µs 90.6 µs +83.31%
test_empty_path_uncached 166.9 µs 90.6 µs +84.24%
test_empty_query_string_uncached 104.6 µs 80.4 µs +30.17%
test_empty_raw_path_qs_uncached 113.6 µs 97.1 µs +17%
test_path_safe_uncached 182.1 µs 161.8 µs +12.58%
test_raw_path_qs_uncached 114.1 µs 93.9 µs +21.55%
test_raw_path_qs_with_query_uncached 145.2 µs 125.1 µs +16.03%

Copy link

codecov bot commented Dec 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.14%. Comparing base (7585a96) to head (1713ef4).
Report is 24 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1443      +/-   ##
==========================================
+ Coverage   96.13%   96.14%   +0.01%     
==========================================
  Files          31       31              
  Lines        5948     5970      +22     
  Branches      362      364       +2     
==========================================
+ Hits         5718     5740      +22     
  Misses        204      204              
  Partials       26       26              
Flag Coverage Δ
CI-GHA 96.14% <100.00%> (+0.01%) ⬆️
MyPy 49.71% <36.66%> (-0.15%) ⬇️
OS-Linux 99.56% <100.00%> (+<0.01%) ⬆️
OS-Windows 99.62% <100.00%> (+<0.01%) ⬆️
OS-macOS 99.31% <100.00%> (+<0.01%) ⬆️
Py-3.10.11 99.29% <100.00%> (+<0.01%) ⬆️
Py-3.10.15 99.52% <100.00%> (+<0.01%) ⬆️
Py-3.11.10 99.52% <100.00%> (+<0.01%) ⬆️
Py-3.11.9 99.29% <100.00%> (+<0.01%) ⬆️
Py-3.12.7 99.52% <100.00%> (+<0.01%) ⬆️
Py-3.13.0 99.52% <100.00%> (+<0.01%) ⬆️
Py-3.9.13 99.25% <100.00%> (+<0.01%) ⬆️
Py-3.9.20 99.48% <100.00%> (+<0.01%) ⬆️
Py-pypy7.3.16 99.54% <100.00%> (+<0.01%) ⬆️
Py-pypy7.3.17 99.56% <100.00%> (+<0.01%) ⬆️
VM-macos-latest 99.31% <100.00%> (+<0.01%) ⬆️
VM-ubuntu-latest 99.56% <100.00%> (+<0.01%) ⬆️
VM-windows-latest 99.62% <100.00%> (+<0.01%) ⬆️
pytest 99.56% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bdraco
Copy link
Member Author

bdraco commented Dec 1, 2024

Its not going to show any difference because the cached_property is already cached and it only does it once

type(url).query_string.wrapped(url) would work to be able to benchmark cache miss wrapped was public in propcache

@bdraco bdraco changed the title Avoid quoting values that never need to be quoted DNM: Avoid quoting values that never need to be quoted Dec 1, 2024
@bdraco
Copy link
Member Author

bdraco commented Dec 1, 2024

Probably should do a new release of propcache with wrapped marked as public but it will likely fail because of pypa/gh-action-pypi-publish#307

@bdraco
Copy link
Member Author

bdraco commented Dec 1, 2024

Its actually unlikely to make much difference since with the caching improvements (unreleased due to the publish issue) its so much more likely to be cached anyways.

bdraco added a commit that referenced this pull request Dec 1, 2024
We did not have the ability to write benchmarks that
benchmarked the first hit because propcache did not
expose the wrapped function until 0.2.1

This will give us a better idea if #1443 is worth it
@bdraco
Copy link
Member Author

bdraco commented Dec 1, 2024

Well I guess it is worth it

@bdraco bdraco changed the title DNM: Avoid quoting values that never need to be quoted Improve performance of path functions on cache miss Dec 1, 2024
@psf-chronographer psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Dec 1, 2024
@bdraco bdraco marked this pull request as ready for review December 1, 2024 19:31
@bdraco bdraco merged commit 558e4b0 into master Dec 1, 2024
46 of 48 checks passed
@bdraco bdraco deleted the skip_quoting_empty_and_slash branch December 1, 2024 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bot:chronographer:provided There is a change note present in this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant