Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qcodes import takes a long time #4543

Open
eendebakpt opened this issue Aug 23, 2022 · 3 comments
Open

qcodes import takes a long time #4543

eendebakpt opened this issue Aug 23, 2022 · 3 comments
Milestone

Comments

@eendebakpt
Copy link
Contributor

Importing qcodes with import qcodes takes a long time (> 2 seconds on the system tested). The import time can be analysed with tools like https://github.com/asottile/importtime-waterfall

Several components could perhaps not be loaded by default, or with a lazy loader (e.g. https://scientific-python.org/specs/spec-0001/, but there are more tools)

An example is the import from opencensus.ext.azure.log_exporter import AzureLogHandler from qcodes.logger.logger. By default qc.config.telemetry.enabled is False, so the import is executed at startup, but not used.

(more examples can be found by analysing the output of importtime-waterfall)

@jenshnielsen
Copy link
Collaborator

@eendebakpt Which version of qcodes have you tested. I spend a fair amount of time improving this before 0.32 for example. For example the imports of opencensus azure are lazy and has been so since 0.32. See 9f90a67

@eendebakpt
Copy link
Contributor Author

@jenshnielsen With qcodes updated to master the import of azure is indeed lazy. There is still room for improvement though.

Running:

python -c "import time; import xarray; import pandas; import scipy; t0=time.perf_counter(); import qcodes; dt=time.perf_counter()-t0; print(f'{1e3*dt:.2f} ms')"

results in a > 1 second import time of qcodes on my system.

Some more candidates to reduce the import time:

  • From the qcodes init:
from qcodes.instrument_drivers.test import test_instrument, test_instruments

Can be removed, or made lazy (for example with a mechanism like sympy/sympy#23832). Tests should not impact runtime import performances.

  • qcodes.data.hdf5_format. This is part of the legacy dataset, but still imported via

https://github.com/QCoDeS/Qcodes/blob/master/qcodes/parameters/array_parameter.py#L10

  • The whole legacy dataset module is imported via

https://github.com/QCoDeS/Qcodes/blob/master/qcodes/dataset/__init__.py#L27

(this triggers also imports of xarray and scipy, maybe they would have been imported anyway)

@astafan8 astafan8 added this to the 0.37.0 milestone Sep 9, 2022
bors bot added a commit that referenced this issue Oct 4, 2022
4616: Reducing qcodes import times r=jenshnielsen a=edumur

Hi all,

Following this discussion #4565 and this issue #4543 I am proposing this pull request to work on reducing qcodes loading time.

I have started with pandas.
I tried to propagate what `@jenshnielsen` did to make pandas import lazy-import.
I am not sure I have succeeded since I still see pandas being imported while doing `importtime-waterfall`.
I would gladly improve the request if someone help me on this.

If this becomes a successful endeavor, I will continue by making other import lazy-import. 

Co-authored-by: Etienne Dumur <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jens H. Nielsen <[email protected]>
@eendebakpt
Copy link
Contributor Author

Closing as the import time was improved in two pr's

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants