Add NGINX-QUIC

[tvvocold]

Please consider adding support for install nginx-quic: https://quic.nginx.org (not the same as CloudFlare-Quiche)

[MaximeMichaud]

Is this stable yet? I look forward for this NGINX implementation :)

[GwynethLlewelyn]

It's probably as stable as CloudFlare-Quiche... which is not really the most stable implementation in the world (besides demanding a huuuuge compilation chain...).

[GwynethLlewelyn]

... granted, it's really very tough to get it to compile properly!

A few references: https://www.nginx.com/blog/introducing-technology-preview-nginx-support-for-quic-http-3/
And its README: https://hg.nginx.org/nginx-quic/file/quic/README

Now, the issue is that the so-called nginx-quic — which should be merged with the mainline version by the end of 2022 or so — actually compiles well by itself. It works well when adding 'standard modules', i.e. those that are written in plain old C, have been somehow 'endorsed' by the Nginx team, and are well-behaved: they are installed on the default directories such as ../modules/, they fit in nicely with the main nginx compilation toolchain, and so forth. There is a slight catch on nginx-quic, which is that it does its autoconfiguration in a different way than mainline nginx, but... it's manageable.

The problem comes next.

To effectively build nginx-quic, you need either BoringSSL (done by Google) or QuicTLS (a fork of OpenSSL done by Akamai & Microsoft) or potentially some other OpenSSL-compatible library which provides TLS streams over QUIC. The core OpenSSL developers, while acutely aware of the need to implement QUIC support 'sooner or later', have no intention to do so 'soon'. Even the latest and greatest 3.0 version of OpenSSL does not include QUIC. So... there is no current option but to compile something that is OpenSSL-compatible (at the ABI level).

The trouble is that these implementations are often a mix of C/C++ and that requires changing the standard nginx toolchain — which only needs to worry about C, never about C++. Also, each library seems to prefer their own tools; nginx-quic can be used with Ninja, anything with C++ in it will most likely use CMake, some favour Autoconf, others have their own, hand-crafted configuration tool... and so forth. Figuring how all of these different toolchains can be effectively used together is hard; that was really hard work done by @angristan until he managed to get everything tied together.

Unfortunately, I'm at a loss here — it's something beyond my abilities to figure out. I did manage to get nginx-quic working with BoringSSL — but failed to add the PageSpeed module to it. Conversely, I did manage to get nginx-quic to compile with the PageSpeed module, but, of course, it then lacks BoringSSL, and fails to link everything together.

In other words, to get nginx-quic to compile cleanly, you need either:

• Some help from the nginx developers, asking them to test their compilation toolchain against popular third-party modules (in general, they will not bother with those, since that's not their job!)
• Some help from @angristan or @Bastakka, who have been awesome in side-stepping all those issues, and adopt nginx-quic as a valid alternative to nginx patched to use Cloudflare's quiche...

Alas, we need stand-alone, easy-to-manage, dynamic plugins, just like Apache (or PHP!) have, that can be independently compiled and then just included from a simple configuration file, without bothering nginx. Then again, perhaps the reason why nginx is so fast is because everything gets compiled directly into its core... 🤷‍♀️

[muuvmuuv]

Alas, we need stand-alone, easy-to-manage, dynamic plugins, just like Apache (or PHP!) have, that can be independently compiled and then just included from a simple configuration file, without bothering nginx. Then again, perhaps the reason why nginx is so fast is because everything gets compiled directly into its core... 🤷‍♀️

Isn't the answer to this --add-dynamic-module=PATH?

[GwynethLlewelyn]

@muuvmuuv indeed it is. The (current) trouble with that approach, AFAIK, is that not all modules can be compiled that way. I have not enough knowledge about the internals of nginx to know why some can be added as dynamic modules, and others cannot.

[mtx-z]

From Nginx:

Not all modules can be made dynamic. There are some modules, the core modules among them, that are an integral part of NGINX, and these can’t be cut into loadable objects.

There are also some complex modules. For example, Mail is actually a set of several modules, but they’re written in such a way that makes it impossible to separate specific modules. So right now, Mail is a single loadable module that provides eight NGINX modules.

Some third‑party modules like RTMP and Lua are also complex modules.

Nginx also provides help to convert static to dynamic modules.

So it seems that, in addition to the module requiring new specific configuration syntax, some "complex" modules cannot be made dynamic. And as @GwynethLlewelyn stated, some compilation compatibility issues.

@GwynethLlewelyn I thought that you were able to build nginx-quic with PageSpeed module, with the latest OpenSSL+quic (3.0.5).

As you and nginx mentioned:

At the time of writing [July 2021], OpenSSL does not support QUIC, and it might take many months before any operating system distributions ship with a QUIC‑enabled SSL/TLS library. Our QUIC implementation currently supports BoringSSL and the quictls fork of OpenSSL, which must be compiled with NGINX.

We expect the state of QUIC‑enabled SSL/TLS libraries to become clearer during the period when we are merging the nginx-quic development branch into NGINX mainline.

also from summer 2022 comment:

Product Management tells me that we hope to fully support QUIC and HTTP/3 by the end of 2022. Note that this is not a promise :-)

I believe we must wait for nginx-quic to be merged in the Nginx mainline. Meaning they'll consider having "mature enough" SSL+QUIC libraries. I also believe we won't see an official OpenSSL QUIC support very soon. My guess is it'll have to use Google BoringSSL or OpenSSL QUIC fork (Microsoft/Akamai) in the end. We are a bit too early for standardization.

[muuvmuuv]

Ah yes, read that too @GwynethLlewelyn but at least some are compileable...

I would love to see that at the end of the year 'cause we have a major release of all our platforms and I would really like to get around this without any heavy workarounds or performance downgrades. :D

@mtx-z @GwynethLlewelyn do you know of any performance losses with dynamic module loading?

[GwynethLlewelyn]

I'm afraid I have not done any performance tests...

In theory, these days, the losses, if any, shouldn't be perceptible. That depends much more on how the linker does its job properly.

[MaximeMichaud]

Good news finally !

Changes with nginx 1.25.1 13 Jun 2023

*) Feature: the "http2" directive, which enables HTTP/2 on a per-server
   basis; the "http2" parameter of the "listen" directive is now
   deprecated.

*) Change: HTTP/2 server push support has been removed.

*) Change: the deprecated "ssl" directive is not supported anymore.

*) Bugfix: in HTTP/3 when using OpenSSL.

Changes with nginx 1.25.0 23 May 2023

*) Feature: experimental HTTP/3 support.