README: LMDB clarification needed in context of Lightning Stream

[udf2457]

The README for the pdns docker image (https://github.com/PowerDNS/pdns/blob/master/Docker-README.md#lmdb) states the host PID namespace should be used.

Does this still apply where LMDB is used in conjunction with Lightning Stream ? This should proabably be addressed in the README for clarity ?

[hlindqvist]

The README for the pdns docker image (https://github.com/PowerDNS/pdns/blob/master/Docker-README.md#lmdb) states the host PID namespace should be used.

Does this still apply where LMDB is used in conjunction with Lightning Stream ? This should proabably be addressed in the README for clarity ?

As I understand it, that sounds like exactly the kind of setup where that section in the docs becomes relevant.
(If you only run pdns-auth by itself, it would still be the only process using the database files, with our without involving containers.)

For reference, this seems relevant: PowerDNS/lightningstream#79
And separately, but related in terms of being relevant to the same type of setup, also this: PowerDNS/lightningstream#75

Maybe both issues (provided both are still relevant?) could be elaborated on in the docs?

I suppose the point of the somewhat vague instruction in docs becomes relevant specifically when processes from different containers (or container + host) all operate on the same set of LMDB database files? Where said processes may have conflicting PIDs and/or the respective images could have shipped with different libc, etc.

[udf2457]

There's also the lmdb-random-ids config param which is default false and so its also not clear if there is interplay there ?

The description of that param in the docs is:

If some external process is synchronising databases between systems, this will avoid conflicts when objects (domains, keys, etc.) get added

[hlindqvist]

There's also the lmdb-random-ids config param which is default false and so its also not clear if there is interplay there ?

The description of that param in the docs is:

If some external process is synchronising databases between systems, this will avoid conflicts when objects (domains, keys, etc.) get added

To my knowledge that (among other things) must be on for a Lightningstream setup, but that is unrelated to the container specific concerns above.
Reference: https://doc.powerdns.com/lightningstream/pdns-auth-installation/#configuring-powerdns-authoritative-server-48

[udf2457]

Perhaps also of interest is the docker-compose over at the LighningStream repo which describes itself as "Demo with the PowerDNS Authoritative Nameserver" where command: --minimum-pid 50 sync is used, which is also mentioned in your #75 above.

But there's no mention of it on this repo's docker-compose.

That's potentially a more attractive option than host PID namespace, but as always more clarity would be appreciated !

[hlindqvist]

Perhaps also of interest is the docker-compose over at the LighningStream repo which describes itself as "Demo with the PowerDNS Authoritative Nameserver" where command: --minimum-pid 50 sync is used, which is also mentioned in your #75 above.

But there's no mention of it on this repo's docker-compose.

That's potentially a more attractive option than host PID namespace, but as always more clarity would be appreciated !

That setting does seem like a potential workaround (a bit of a hack, but used with care it should avoid the problem),

There are also other aspects of the pid option which may be more appealing than pid: host. From what I can tell docker-compose allows service references for pid.

Ie something along these lines could also be an option for how to handle the pid situation:

services:
  pdns-auth:
    image: pdnsimage

  pdns-lightning:
    image: lightningimage
    pid: "service:pdns-auth"
    depends_on:
      - pdns-auth

[udf2457]

Of course --minimum-pid could bring issues of its own since you could theoretically reach the same number since you are only setting a minimum, not a min/max window.

Looks like you're right on pid settings: https://docs.docker.com/reference/cli/docker/container/run/#pid

I guess the other quesiton here is whether we actually need to start a seperate lightning stream instance at all ? I would have thought pdns-auth brings its own "internal" instance when you enable LMDB/LightningStream in the config ? The docs seem to state that PowerDNS will create the database structure ? https://doc.powerdns.com/authoritative/backends/lmdb.html?highlight=lmdb#lmdb-structure

[hlindqvist]

I guess the other quesiton here is whether we actually need to start a seperate lightning stream instance at all ? I would have thought pdns-auth brings its own "internal" instance when you enable LMDB/LightningStream in the config ? The docs seem to state that PowerDNS will create the database structure ? https://doc.powerdns.com/authoritative/backends/lmdb.html?highlight=lmdb#lmdb-structure

The lmdb-lightning-stream setting in pdns-auth is, to my knowledge, essentially a shortcut to ensure that you have all the settings required for pdns to set up its LMDB database with options that are lightningstream compatible, all in one go.

As I understand, it's specifically because the lightningstream synchronization runs as a separate daemon, potentially in a container with overlapping PIDs or with a different libc, that these LMDB-centric container concerns exist in the first place.

[udf2457]

I have not yet had a chance to use podman systemd quadlets, but if I my understanding is correct, by default podman creates a pod with its own private PID namespace. And so both containers could be started in that.

So, pending any clarification on the orignial question, I am thinking perhaps using podman instead of docker_compose would be the safest / least hacky way forward.