Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TEST FAILURE] Mac OS tests failing on GH Hosted Runners #67135

Open
barneysowood opened this issue Jan 10, 2025 · 5 comments
Open

[TEST FAILURE] Mac OS tests failing on GH Hosted Runners #67135

barneysowood opened this issue Jan 10, 2025 · 5 comments
Labels
Test-Failure Fix the CI failure ⛈

Comments

@barneysowood
Copy link
Contributor

Mac OS tests are failing since work to move CI from AWS to GH Hosted Runners.

Work being done by @dwoz in #67114 - still seeing failing Mac OS tests in test runs related to that PR, eg https://github.com/saltstack/salt/actions/runs/12698967172/job/35399479072

Open this issue to cover tracking the issue with tests on Mac OS

@barneysowood barneysowood added the Test-Failure Fix the CI failure ⛈ label Jan 10, 2025
@barneysowood
Copy link
Contributor Author

Issue can be assigned to me.

@barneysowood
Copy link
Contributor Author

Found pre-migration test run where the tests are passing:

https://github.com/saltstack/salt/actions/runs/11545108391/job/32132765850

In the failing job the test that is failing gives this error:

2025-01-09T22:08:18.9420890Z ERROR tests/pytests/unit/states/test_service.py -
FileNotFoundError: [Errno 2] No such file or directory: 'systemctl'

Looking for that test in the pre-migration tests, I see:

2024-10-28T01:17:42.8086800Z SKIPPED [1] tests/pytests/unit/states/test_service.py:681:
service.running is currently failing on OSX

So looks like that should be bring skipped. If I look at that test in #67114, it is marked to be skipped:

@pytest.mark.skip_on_darwin(reason="service.running is currently failing on OSX")

So pyest.mark.skip_on_darwin comes from pytest-skip-markers. Wondered if it was an issue with pytest-skip-markers on GH runners... but a simple test with a test repo shows @pytest.mark.skip_on_darwin to work.

Looks like there's some more complex reason why @pytest.mark.skip_on_darwin is getting ignored. Setting up some tests to try to debug further.

@barneysowood
Copy link
Contributor Author

Ok, so looks like the issue was actually the _check_systemctl() and check_hostnamectl() functions that are being used in a @pytest.mark.skipif marker in these test files:

tests/integration/modules/test_localemod.py
tests/integration/modules/test_timezone.py
tests/pytests/functional/modules/test_service.py
tests/pytests/functional/modules/test_system.py
tests/pytests/functional/states/test_service.py
tests/pytests/unit/states/test_service.py

the original functions tried to run systemctl to check the state of systemd. On Mac OS (or other non-Linux) systems, that fails as systemctl isn't available. Added a check that the system was linux and had the function return false if not. eg

--- a/tests/pytests/functional/modules/test_service.py
+++ b/tests/pytests/functional/modules/test_service.py
@@ -17,10 +17,14 @@ pytestmark = [

 def _check_systemctl():
     if not hasattr(_check_systemctl, "memo"):
-        proc = subprocess.run(["systemctl"], capture_output=True, check=False)
-        _check_systemctl.memo = (
-            b"Failed to get D-Bus connection: No such file or directory" in proc.stderr
-        )
+        if not salt.utils.platform.is_linux():
+            _check_systemctl.memo = False
+        else:
+            proc = subprocess.run(["systemctl"], capture_output=True, check=False)
+            _check_systemctl.memo = (
+                b"Failed to get D-Bus connection: No such file or directory"
+                in proc.stderr
+            )
     return _check_systemctl.memo

That works and and the Mac OS tests now pass (see this CI job). Not sure it's the best way to handle this - ideally you'd check if it's a systemd with systemd rather than just if it is Linux - @dwoz feel free to change how that works if you think there's a better way.

@dwoz - I can either wait for you to merge #67114 and then rebase and merge #67136 or if it's easier you can just cherry-pick 6b9336d

@dwoz
Copy link
Contributor

dwoz commented Jan 11, 2025

@dwoz - I can either wait for you to merge #67114 and then rebase and merge #67136 or if it's easier you can just cherry-pick 6b9336d

@barneysowood ,

Great find, I've just merged my PR. So you can open a PR for these failures specifically.

@barneysowood
Copy link
Contributor Author

@dwoz - I can either wait for you to merge #67114 and then rebase and merge #67136 or if it's easier you can just cherry-pick 6b9336d

@barneysowood ,

Great find, I've just merged my PR. So you can open a PR for these failures specifically.

Thanks - that's now in a PR ready for review/merging - #67138

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Test-Failure Fix the CI failure ⛈
Projects
None yet
Development

No branches or pull requests

2 participants