Reimplement repoquery in a more generic way #10514
Conversation
dnfwrapper
Outdated
There was a problem hiding this comment.
This is essentially the same as rpmdistro-repoquery but without the repoquery in the last line. We can talk to the maintainers to provide this. While talking about this I was pointed to fedrq as well, which may be nicer.
| DNF_VAR_foremanver=nightly | ||
| DNF_VAR_katellover=nightly | ||
| DNF_VAR_candlepinver=nightly | ||
| DNF_VAR_puppetver=7 |
There was a problem hiding this comment.
It would be nice if these were provided by the package manifest.
This makes it easier to update to version 8.
This approach is based on rpmdist-repoquery where a certain directory structure is used to track repositories. That is then selected, together with a releasever. Our own repositories are made generic by using DNF_VAR_$var, which makes branching easier. It also uses true modularity. For that it hacks together a temporary installroot and uses fakeroot to pretend to be root, because dnf insists you need to be root to enable a module. Due to only using real DNF, we can rely on the modulary metadata and dependency resolution to select the correct modules. Again, reducing duplication. A major motivation for this is the upcoming CentOS Stream 8 EOL where the repositories will be archived. This new repository structure allows easy switching to another Enterprise Linux for repoclosure checks. This needs a different approach in how we configure repoclosure in obal and the wrapper scripts should probably live in another repository as well. Another excercise is the EL7 repoclosure check.
ekohl
force-pushed
the
rpm/develop-improve-repos
branch
from
d1746d1 to
486bb75
Compare
|
Rebased to resolve conflicts. I've also opened #10869 for the first commit. |
| @@ -0,0 +1 @@ | |||
| /usr/share/rpmdistro-repoquery/distros/centos-stream-legacy/centos-stream-legacy.repo | |||
There was a problem hiding this comment.
This file comes from rpmdistro-repoquery, right?
checks https://src.fedoraproject.org/rpms/rpmdistro-repoquery
So EPEL9 will be required on the systems, OK.
There was a problem hiding this comment.
Yes. My idea was that we would push off the actual repo configs to something common, rather than maintaining it ourselves. We can also choose to copy them of course.
| [foreman-$foremanver-staging] | ||
| name=Foreman $foremanver staging EL$releasever | ||
| baseurl=https://stagingyum.theforeman.org/foreman/$foremanver/el$releasever/$basearch/ | ||
| module_hotfixes=1 |
There was a problem hiding this comment.
is this required given you actually enable modules?
There was a problem hiding this comment.
Maybe not, probably a leftover from earlier testing.
| repoclosure centos-stream 9 "" --check foreman-${DNF_VAR_foremanver}-staging --enablerepo puppet7 | ||
| repoclosure centos-stream-legacy 8 foreman-devel:el8 --check foreman-${DNF_VAR_foremanver}-staging --enablerepo puppet7 |
There was a problem hiding this comment.
this hard-codes closure for foreman.
how would I call it for katello?
how for the case where we add a repo on the CLI to perform closure on a copr build?
There was a problem hiding this comment.
This is the area where I ended the day of learning. I imagine the next step would be to something work this into obal so we can use it for our *-packaging repositories. Ideally set the variables from package_manifest.yaml.
Still, I think this script gives a good start to see the general idea.
|
My biggest concern here is that this lives in the wrong place. Today we have three places that do repoclosure: foreman-packaging, pulpcore-packaging, candlepin-packaging. We certainly shouldn't have to have copies of As for |
|
As I wrote in #10514 (comment), I agree. My ideal is that in But then we also have COPR that may replace parts, and my brain was full. |
|
Notes to self: jenkins jobs invokes obal in 2 ways. First of all, there is repoclosure in RPM pipelines: obal(
action: 'repoclosure',
packages: "${repo}-repoclosure-${dist}"
)These correspond to: foreman-packaging/package_manifest.yaml Lines 905 to 1019 in 85637cd And then for individual packages when they've been built: obal(
action: "repoclosure",
packages: package_name,
extraVars: [
'repoclosure_check_repos': [repo['url']],
'repoclosure_target_dist': repo['dist']
]
)These correspond to (among others): foreman-packaging/package_manifest.yaml Lines 1027 to 1050 in 85637cd |
|
Let's first start with deduplicating the configs: #10881 reuses settings in the manifest. |
|
My first impression is we make |
|
I was starting to lean to the same thing. Some further notes for myself below. Essentially one use case calls it with: repoclosure_target_repos:
el8:
- "foreman-{{ foreman_version }}-staging"
repoclosure_lookaside_repos:
el8:
- $OS_REPOS
- puppet-7
repoclosure_target_dist: el8Then the other sets The design I'm starting to lean to (and that's where #10881 tries to advance to) is to always refer to some " This may require some redesign in obal itself, but I think it'll reduce duplication. |
|
Further notes to self: from #10918 (comment) We apply repoclosure 1-by-1. I think we should consider an approach where we group things. |
This approach is based on rpmdist-repoquery where a certain directory structure is used to track repositories. That is then selected, together with a releasever.
Our own repositories are made generic by using DNF_VAR_$var, which makes branching easier.
It also uses true modularity. For that it hacks together a temporary installroot and uses fakeroot to pretend to be root, because dnf insists you need to be root to enable a module. Due to only using real DNF, we can rely on the modulary metadata and dependency resolution to select the correct modules. Again, reducing duplication.
A major motivation for this is the upcoming CentOS Stream 8 EOL where the repositories will be archived. This new repository structure allows easy switching to another Enterprise Linux for repoclosure checks.
This needs a different approach in how we configure repoclosure in obal and the wrapper scripts should probably live in another repository as well. Another excercise is the EL7 repoclosure check.