Move host benchmarks to a subdirectory for easier distinction

[Priya2698]

No description provided.

[github-actions]

Review updated until commit 06c353a

Description

• Move host benchmarks to subdirectory for organization

• Update imports to use relative paths

• Add __init__.py for correct import structure

---

Changes walkthrough 📝

	Relevant files
Enhancement	test_adaptive_layernorm_host.py Update import to relative path                                                      benchmarks/python/host/test_adaptive_layernorm_host.py Update import statement to use relative path +1/-1      test_many_pointwise_ops_host.py Update imports to relative paths                                                  benchmarks/python/host/test_many_pointwise_ops_host.py Update import statements to use relative paths +2/-2      test_many_segments_host.py Update import to relative path                                                      benchmarks/python/host/test_many_segments_host.py Update import statement to use relative path +1/-1
Additional files	__init__.py [link]

---

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 No relevant tests

⚡ Recommended focus areas for review Import Path The import path for run_benchmark has been changed from a relative import to an absolute import. Ensure that this change does not break the module's ability to locate run_benchmark when the script is executed. from ..core import run_benchmark Import Path The import path for run_benchmark and PROMOTE_DTYPES has been changed from a relative import to an absolute import. Ensure that this change does not break the module's ability to locate these imports when the script is executed. from ..core import run_benchmark import torch Import Path The import path for run_benchmark has been changed from a relative import to an absolute import. Ensure that this change does not break the module's ability to locate run_benchmark when the script is executed. from ..core import run_benchmark

[naoyam]

LGTM, but let's rename the directory to benchmarks/python/host/.

[xwang233]

I think import benchmarks from global space could be problematic. For example, pytorch used to accidentally add their benchmarks/ folder to the global package and import benchmarks.python.core here would cause import error (and it did). Let's use relative import instead.

Another concern I have is if moving the benchmark script would cause any changes in benchmark JSON output?

[Priya2698]

I think import benchmarks from global space could be problematic. For example, pytorch used to accidentally add their benchmarks/ folder to the global package and import benchmarks.python.core here would cause import error (and it did). Let's use relative import instead.

Oh that's a good point. I will change it.

Another concern I have is if moving the benchmark script would cause any changes in benchmark JSON output?

Can you elaborate? I can check if there are any paths in the JSON output that will be changed. If this is too much overhead for the dashboard, we can only rename benchmarks to have _host in them. I mainly made this change so everyone can easily locate the 3 host benchmarks instead of scrolling through our (growing) list of benchmarks.

[xwang233]

I can check if there are any paths in the JSON output that will be changed

Yes, that's the concern. I guess there aren't paths in the JSON output but let's double check.

Dashboard visualization should be fine. We may need to update the CI script that runs benchmark. Currently it runs test_*.py at depth 1. I'll have to update the script to search benchmark scripts recursively. This won't be a big problem.

[xwang233]

I just verified that there's no change in the path in generated pytest benchmark JSON file. I've also updated the CI script to recursively search test_*.py benchmark scripts.

Once the relative import issue is resolved, we're ready to go.

[Priya2698]

!build

[Priya2698]

I just verified that there's no change in the path in generated pytest benchmark JSON file. I've also updated the CI script to recursively search test_*.py benchmark scripts.

Once the relative import issue is resolved, we're ready to go.

Thanks, Xiao.
The subdir name is host. I also changed test_many_pointwise_ops.py -> test_many_pointwise_ops_host.py for consistency. With the subdirectory we may not need host in the name, though. I