Grapevine HTTPS/SSL on Linux #26
Replies: 1 comment 3 replies
-
|
I'm afraid you've looked at the wrong version of HttpListener. Those paths are implemented in Mono, not .NET Core. In .NET Core, the code is completely missing. We started our project as .NET Framework application. Later when requests came to support Linux, we have "ported" it to Mono. Beside other problems there were various ones with HttpListener, especially the SSL support. We've filed various bugs, patches and at that time even distributed our own "fixed" version of Mono to customers. Later these were fixed by Mono team and we were able to switch to official version. Then .NET Core came. So we decided to switch again and ported everything to .NET Core. Unfortunately, we were greatly disappointed by lack of support for SSL on non-Windows platforms. We've got "that will never happen" answer from .NET Core team. They said "switch to Kestrel", for which I was not able to find any usable documentation for our case - bind port, not specific paths, access all HTTP headers and authentication etc. The Internet is full of articles how to run ASP.NET on Kestrel but does not seem to be there any article how to use it as servlet engine. Since .NET Core is open source, we've looked into HttpListener code and found out that it looks quite like Mono sources. We have found that certificate handling (that LoadCertificateAndKey method) was completely removed. However, the other code is still there so using reflection it is possible to install the certificate and actually use HttpListener with SSL. But we hit another wall - the same set of bugs we've reported to Mono before. That's why I think .NET Core HttpListener is in fact Mono HttpListener but the version before the fixes we filed to Mono. And with gutted SSL. .NET Core team refused to do any fixes for the code they officially do not support but at least allowed us to file a pull request. So I fixed the problem with application crash when SSL connection cannot be established. There are still other problems like disposed certificate or broken HttpListenerRequest in authentication callback when malformed HTTP request is received (we did not report this issue to .NET Core as they are not willing to do anything about it anyway). Now some code "samples". First, how to install certificate to listener to switch it to SSL. You need to find instance of HttpEndPointListener and set it's _cert field. To do so, you need to call HttpEndPointManager.GetEPListener with correct parameters, best obtained via ListenerPrefix class instance. However, before that, lock collection HttpEndPointManager.s_ipEndPoints on SyncRoot as GetEPListener() is always called when this is locked. string prefix = ...
X509Certificate cert = ...
HttpListener httpListener = ...
httpListener.Prefixes.Add(prefix); // do after adding https prefix
Type lpType = Type.GetType("System.Net.ListenerPrefix, System.Net.HttpListener");
ConstructorInfo lpConstructor = lpType.GetConstructor(new[] { typeof(string) });
Type hepmType = Type.GetType("System.Net.HttpEndPointManager, System.Net.HttpListener");
Type heplType = Type.GetType("System.Net.HttpEndPointListener, System.Net.HttpListener");
FieldInfo hepmIpEndPoints = hepmType.GetField("s_ipEndPoints", BindingFlags.Static | BindingFlags.NonPublic);
MethodInfo getEPListener = hepmType.GetMethod("GetEPListener", BindingFlags.Static | BindingFlags.NonPublic);
FieldInfo heplCert = heplType.GetField("_cert", BindingFlags.NonPublic | BindingFlags.Instance);
// You should check that all of the above is available and do the code below only if so. Because these fields and methods are not available on Windows and it will throw exceptions
object lp = lpConstructor.Invoke(new object[] {prefix});
lock ((hepmIpEndPoints.GetValue(null) as ICollection).SyncRoot) {
object epl = getEPListener.Invoke(null, new[] {lpHost.GetValue(lp), lpPort.GetValue(lp), httpListener, true})
heplCert.SetValue(epl, cert); // and we are done
}Now how to get client certificate from incoming request because the one obtained via request.GetClientCertificate() is disposed. You need to obtain _context property from HttpListenerRequest. From the context, get Connection. From connection, get ConnectedStream and cast it to SslStream. From there just get RemoteCertificate. It looks like it is sometimes X509Certificate2 and sometimes just X509Certificate... X509Certificate2 certificate = request.GetClientCertificate();
if (certificate != null && certificate.Handle == IntPtr.Zero) {
// Is disposed
PropertyInfo hlrContext = typeof(HttpListenerRequest).GetField("_context", BindingFlags.Instance | BindingFlags.NonPublic);
PropertyInfo hlpConnection = typeof(HttpListenerContext).GetProperty("Connection", BindingFlags.Instance | BindingFlags.NonPublic);
PropertyInfo hcConnectedStream = hlpConnection.PropertyType.GetProperty("ConnectedStream", BindingFlags.Instance | BindingFlags.Public);
object context = hlrContext.GetValue(request);
object connection = hlpConnection.GetValue(context);
SslStream stream = (SslStream)hcConnectedStream.GetValue(connection);
certificate = stream.RemoteCertificate as X509Certificate2;
if (certificate == null) {
certificate = new X509Certificate2(stream.RemoteCertificate);
}
}Finally, if you are using authentication callback, be aware that .NET Core calls it even for malformed HTTP requests. It does not call the handling method, just the authentication. Calling various methods on such HttpListenerRequest causes NRE. We just access the property RemoteEndPoint in try-catch and if it throws NRE, just exit the callback because the request is malformed. This shields rest of the code from unexpected NRE. These are problems we've found when trying to use HttpListener in .NET Core on Linux. It works fine on Windows because on Windows it behaves exactly the same as .NET Framework - uses http.sys. I will add one personal note regarding .NET Core. During past three years I've gone through migration of old .NET Framework project to .NET Core and my experience is rather negative. Although it is nice that Microsoft is finally targeting non-Windows platforms, it does it rather wrong. Or may be I don't understand how to use it. I thought I will take old .NET Framework project, do few adjustments, recompile it for .NET Core and it will work anywhere. But that's not true. First, there are many places which are uncompilable and must be rewritten. Some are obvious (no UI support), others less, but that can be done. However, there are worse cases - the code can be compiled, but throws runtime exceptions. We've hit these for example somewhere in security. I don't remember exact classes but the original code was "call some method, then cast to some interface". This was official documentation of .NET Framework (IMHO bad design to force users to cast, but OK). The problem is, that .NET Core returns instance of completely different type on Linux so the cast throws exception! Yes, they have that in documentation, but when porting, who will read all the documentation for already written code? They should have changed it so compilation will fail to inform user that something has changed. Instead, you must have everything covered by tests since otherwise customer will find out, not you. That's wrong. And you need to run these tests on all supported platforms since they can behave quite differently. Another example is HttpListener - works perfectly on Windows, but on Linux, no SSL. And the last one - I read in some Microsoft blog that unlike .NET Framework, .NET Core is able to run in multiple instances on the same machine. Which is great - we no longer need to keep compatibility. What?! So when I want to use something from newer version, I have to rewrite all the code where they decided to break compatibility? And again, they can break it "nicely" by forcing it not to compile, but also "ugly" by changing runtime behavior only. Unfortunately, we've already hit this new "feature" when porting from .NET Core 2.1 to 3.0. So for me, .NET (Core) never again, going back to Java. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your response. There is a lot to process there, and I'm still processing it. |
Beta Was this translation helpful? Give feedback.
-
|
It's slightly off-topic here, but I have a question for Martin:
Do you happen to remember if the Bug #55721: "Memory leak in HttpListener with SSL" ended up being fixed in Mono and/or .NET Core? I'm running into a similar issue, but the Bugzilla attachments don't seem to be accessible anymore and I wasn't able to figure out if it's still unresolved. |
Beta Was this translation helpful? Give feedback.
-
|
Unfortunately, I don't remember. After release of .NET Core, we moved away from Mono. We haven't seen that problem on .NET Core, but may be we just haven't noticed it. It looks like the attachments are working again, I was able to download them (the response says Page Not Found but redirects to download location via Javascript). Anyway, if you have trouble downloading, I can attach them here. |
Beta Was this translation helpful? Give feedback.
-
Grapevine is built as a wrapper for
HttpListener, and as such has the same hard limitations. One of those limitations seems to be using HTTPS/SSL on Linux. Per this issue:I've been asked if Grapevine plans to overcome this issue, so I'd like to have a discussion with the broader community on how important overcoming this is, and solicit ideas on how to approach it in a simple, clean, maintainable and stable way. If that isn't possible, perhaps we can at least provide those looking for answers with a workaround.
I'm coming at this issue from a place of complete ignorance right now. Everything I suggest below is me spit-balling how I might tackle this issue if I were faced with it. Suggestions and/or corrections are welcome.
Approach 1: Use Reflection
The issue seems to be caused by an internal method called
LoadCertificateAndKeyin theHttpListenerclass that looks for certificates to load at a hardcoded path. As this is an internal method, there isn't a straightforward and supported approach to overcoming this limitation.I've heard rumors of some that have used reflection at runtime to overcome this limitation.
via dotnet/runtime#33288 (comment)
As I find code samples, I'll post them here. If you have some, please share (I'm looking at you, @martin-frydl).
Approach 1.2: Symlink Certificate folder
If
HttpListeneris looking to load certificates from a specific path, can we symlink that path to the actual certificates path on a Linux machine? ¯_(ツ)_/¯ I don't know. Can someone with experience answer that?Approach 2: Use a Reverse Proxy
Not my preferred approach, mostly because the whole idea here is to provide something simple and embeddable, and using a reverse proxy is neither simple nor embeddable, I don't think. I'm happy to be wrong on both of those points. But if we allowed the reverse proxy (nginx/ngrok, etc.) to handle the SSL and pass requests to Grapevine using localhost and HTTP, that might get around the issue.
Some Relevant Links:
While there is an effort post GV5 release to have a package that isn't built on top of
HttpListener, I'd like to limit this conversation specifically to the current implementation.Beta Was this translation helpful? Give feedback.
All reactions