ska3-core to use pytest < 8.0.0

[javierggt]

During the release of 2024.1, there was a problem with pytest version 8.0.0 and later. It was decided to downgrade the pytest version for now.

The problem stemmed from passing the -c option to pytest.

The following succeeds:

cd /export/jgonzale/github-workflows/miniconda3/envs/ska3-matlab-2024.2rc3/lib/python3.11/site-packages
pytest Quaternion -c /export/jgonzale/ska_testr/pytest.ini

whereas this fails

cd /export/jgonzale/github-workflows/miniconda3/envs/ska3-matlab-2024.2rc3/lib/python3.11/site-packages
pytest Quaternion -c /proj/sot/ska/jgonzalez/git/ska_testr/pytest.ini

We believe that the behavior has to do with pytest setting the root to the location of pytest.ini. We have not check further why it fails in one case and not the other.

The original error is pasted below:

(ska3-masters) jeanconn-fido> run_testr --include Quaternion
****************************************
*** package Quaternion               ***
****************************************

Copying input tests /proj/sot/ska/jeanproj/git/ska_testr/packages/Quaternion to output dir /proj/sot/ska/jeanproj/git/ska_testr/outputs/logs/Linux_2024-03-01T16-46-36_40e13bc_fido.cfa.harvard.edu/Quaternion
Running python test_unit.py script
============================= test session starts ==============================
platform linux -- Python 3.11.8, pytest-8.0.2, pluggy-1.4.0 -- /fido.real/conda/envs/ska3-masters/bin/python
rootdir: /proj/sot/ska/jeanproj/git/ska_testr
configfile: pytest.ini
plugins: timeout-2.2.0, anyio-4.3.0
collecting ... ERROR: found no collectors for /fido.real/conda/envs/ska3-masters/lib/python3.11/site-packages/Quaternion

collected 0 items / 1 error

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
pluggy/_hooks.py:501: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
pluggy/_manager.py:119: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
_pytest/python.py:211: in pytest_collect_directory
    if pkginit.is_file():
../pathlib.py:1267: in is_file
    return S_ISREG(self.stat().st_mode)
../pathlib.py:1013: in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
E   PermissionError: [Errno 13] Permission denied: '/fido.real/lost+found/__init__.py'
- generated xml file: /proj/sot/ska/jeanproj/git/ska_testr/outputs/logs/Linux_2024-03-01T16-46-36_40e13bc_fido.cfa.harvard.edu/Quaternion/test_unit.py.xml -
=========================== short test summary info ============================
ERROR ../../../../../../../proj/sot/ska/jeanproj/git/ska_testr/::fido.real - ...
=============================== 1 error in 0.26s ===============================
Traceback (most recent call last):
  File "/proj/sot/ska/jeanproj/git/ska_testr/outputs/logs/Linux_2024-03-01T16-46-36_40e13bc_fido.cfa.harvard.edu/Quaternion/test_unit.py", line 2, in <module>
    testr.testr()
  File "/fido.real/conda/envs/ska3-masters/lib/python3.11/site-packages/testr/runner.py", line 51, in testr
    return test(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/fido.real/conda/envs/ska3-masters/lib/python3.11/site-packages/testr/runner.py", line 208, in test
    raise TestError('Failed')
testr.runner.TestError: Failed
Running python post_check_logs.py script
Traceback (most recent call last):
  File "/proj/sot/ska/jeanproj/git/ska_testr/outputs/logs/Linux_2024-03-01T16-46-36_40e13bc_fido.cfa.harvard.edu/Quaternion/post_check_logs.py", line 3, in <module>
    check_files('test_*.log', ['warning', 'error'])
  File "/fido.real/conda/envs/ska3-masters/lib/python3.11/site-packages/testr/packages.py", line 691, in check_files
    raise ValueError('Found matches in check_files:\n{}'.format('\n'.join(matches)))
ValueError: Found matches in check_files:
'error' matched at test_unit.log:5 :: collecting ... ERROR: found no collectors for /fido.real/conda/envs/ska3-masters/lib/python3.11/site-packages/Quaternion
'error' matched at test_unit.log:7 :: collected 0 items / 1 error
'error' matched at test_unit.log:9 :: ==================================== ERRORS ====================================
'error' matched at test_unit.log:10 :: ________________________ ERROR collecting test session _________________________
'error' matched at test_unit.log:21 :: E   PermissionError: [Errno 13] Permission denied: '/fido.real/lost+found/__init__.py'
'error' matched at test_unit.log:24 :: ERROR ../../../../../../../proj/sot/ska/jeanproj/git/ska_testr/::fido.real - ...
'error' matched at test_unit.log:25 :: =============================== 1 error in 0.26s ===============================
'error' matched at test_unit.log:33 :: raise TestError('Failed')
'error' matched at test_unit.log:34 :: testr.runner.TestError: Failed

[jeanconn]

We might want to remove the >= 8 version from the test channel as I don't think we want it even in test environments at this point.

[jeanconn]

pytest-dev/pytest#12311 might help though I don't think it should have been walking the file tree.

[taldcroft]

The behavior of setting the rootdir from the -c option appears to be intended, so it is not going to be fixed.
https://docs.pytest.org/en/stable/reference/customize.html#finding-the-rootdir

Instead, there is a --rootdir option that might work for ska_testr. I did a local test that reproduces the original problem here and providing --rootdir fixed it. I haven't looked at the ska_testr code in a while, but basically --rootdir=$PWD?

[jeanconn]

You lost me a bit at --export? Is the rootdir option new? It looked to me previously like -c was overriding it https://chandramission.slack.com/archives/G01LN40AXPG/p1709319660407249?thread_ts=1709311933.839259&cid=G01LN40AXPG but I'll recheck.

[taldcroft]

Brain fail, that comment is updated.

[taldcroft]

My test consisted of putting pytest.ini in a directory that was otherwise not writable by me. Then with --config-file pointing to that pytest.ini, it gave a warning about not being able to create a cache dir in that non-writeable directory (which I presume it was using as rootdir). But with --rootdir=$PWD it did not produce any such warnings.

[jeanconn]

It looks to me like with pytest 8.3.4 the test collection is no longer failing on /fido.real/lost+found in a similar ska_testr test setup to the one in the issue text from me, so we may not need a change in testr to set rootdir.

[taldcroft]

I think the change in pytest 12311 might be fixing the original error, but only by ignoring permission errors. I still think we don't want it walking the entire filesystem tree, but I might be mistaken about what is actually happening in this case.

The original intent with ska_testr was that the rootdir was the current working directory, so it seems like using that --rootdir option would be a good thing.

[taldcroft]

See sot/testr#52.