Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
nit: stop calling importlib on every api req (#1219)
Each call is like 1-3ms, and we call to get the version as part of every client request's headers. If you're running an eval over a lot of traces (say 10 evaluators on 1k examples), that's slow. Esp if just doing custom code evaluators which should run fast. ### Flame Graphs before and after (ran on 2k smallish examples w 10 evaluators) <img width="1510" alt="Screenshot 2024-11-14 at 5 25 43 PM" src="https://github.com/user-attachments/assets/2366b357-f3de-4f18-b7a8-5b362c992119"> <img width="1507" alt="Screenshot 2024-11-14 at 5 25 53 PM" src="https://github.com/user-attachments/assets/3311e686-373c-4fd7-80e1-d6939a3c4632"> --------- Co-authored-by: William FH <[email protected]>
- Loading branch information