Repository navigation
OpenSSL 1.1.1 Support TODO (pls help) #18770
Description
Activity
This probably accounts for some of it:
#16130 (comment)
I filed openssl/openssl#4574 for the immediate trigger, but I think Node's reentrancy is also somewhat questionably supportable.Other bits I suspect come out of them enabling TLS 1.3 by default and the tests being sensitive to there not being a version beyond TLS 1.2.
(But I haven't triaged and am mostly speculating.)
I'm now working on build bindings with 1.1.1-pre1 and will look at test errors later.
Reacted by Benjamin GruenbaumHere is my preliminary work to support OpenSSL-1.1.1 on only Linux and Mac. https://github.com/shigeki/node/tree/openssl111pre1_unix_mac_ok
I can build node binary but several tests are still failed as described above.Windows is not yet supported because OpenSSL-1.1.1 generates makefiles to work only
nmakefor Windows and we have to some integrations togmake. It needs some time to work on Windows.pre2 is out, looks like a ton of pending refactoring and removals got merged in for this one https://github.com/openssl/openssl/commits/master
I haven't tested this yetOpenSSL-1.1.1-pre1 binding without asm support of https://github.com/shigeki/node/tree/WIP_openssl111pre_noasm has almost been done.
Most of CI results of
https://ci.nodejs.org/job/node-test-commit/16544/ are fine except in some platforms. I'm going to work on them with pre2 soon.Most of test failures is due to cipher suite incompatibilities between TLS1.2 and TLS1.3 and it can be resolved by limiting to use TLS1.2 by default. The issue is now discussing in openssl/openssl#5359.
Very nice work @shigeki. I notice there's a failure on testing 1.0.2 support:
../src/node_crypto.cc: In static member function 'static void node::crypto::SecureContext::Init(const v8::FunctionCallbackInfo<v8::Value>&)': 19:03:36 ../src/node_crypto.cc:636:49: error: 'SSL_CTX_set_max_proto_version' was not declared in this scope 19:03:36 SSL_CTX_set_max_proto_version(sc->ctx_, TLS1_2_VERSION);Is it going to be hard to main backward compatibility?
It is the API which is available above 1.1.0 so that
#if OPENSSL_VERSION_NUMBER >= 0x10100000L .. #endif
would be working.So, we are at pre9 now, TLS 1.3 is finalized and ratified as an RFC, and Node 10 LTS is only a month(ish) away. It would be nice to have 1.1.1 in Node 10 before we go LTS but if we have to do it after then so be it. I have a 1.1.1-pre test job for Linux in CI that I'd like to enable so we can start getting ready but we don't quite have compatibility.
@shigeki the commit at the head of
upgrade_openssl111_pre6_tls12_only, shigeki@26388fe, we get most of the way there for OpenSSL 1.1.1 as a shared library. It still failstest-tls-handshake-error,test-tls-set-ciphers-error,test-tls-server-verifyandtest-tls-connect.Any idea whether these are easy to overcome and what the impact on non-Linux is at the moment? CI can only do dynamic compiles on Linux at the moment but it's a start and we could use it to make a basic assertion that 1.1.1 isn't going to be a breaking upgrade. I don't know if other collaborators would accept a PR with support for a -pre library but I wouldn't mind trying soon if we can.
OpenSSL 1.1.1 final is here -> https://www.openssl.org/blog/blog/2018/09/11/release111/
Reacted by Tierney Cyren, Rayan Salhab, senditu, Ali Rahbari, St. Veit and Xingcan LANReacted by Tierney Cyren and sendituHere is my prototype on upgrading latest OpenSSL-1.1.1 release with only TLS1.2 support not TLS1.3.
https://github.com/shigeki/node/tree/WIP_upgrade_openssl111_tls12_only
This has one new feature to have
options.min_version/max_versionto limit the use of TLS versions.
Onetest-tls-server-verifytest was failed due to client auth error during renegotiation that need to be investigated.This is only working on Linux. TODO is to support multi platforms especially on MacOS, Win and s390 that can be checked through CI. I think that it does not have any breaking updates as long as it supports only up to TLS1.2.
Supporting TLS1.3 in Node is an another story. It needs much more works and would have breaking updates because of changes of info callbacks of its TLS state machine and deprecated/new several features for TLS1.3 in OpenSSL-1.1.1.
The failure of
test-tls-server-verifywas identified and an issue was submitted in openssl/openssl#7199.@shigeki it sounds like from openssl/openssl#7199 that we may have some fundamental architecture problems with our TLS layer. It sounds like the TLS 1.3 support problem is related to the
test-tls-server-verifyfailure too, is that right? Does that mean that we're going to have a lot more changes on Node's side to get 1.1.1 in? And if so, do we need to get additional help for that? Is there a risk that we're going to need to have breaking changes just to get 1.1.1 in properly?Point of note: On Ubuntu, we are building OpenSSL 1.1.1 with OPENSSL_TLS_SECURITY_LEVEL set to 0 and all TLS protocol versions enabled. It would be nice for node.js 11.0.0 to use 1.1.1 abi/api, and to explicitly limit max TLS version to 1.2 then. This will thus allow us to ship nodejs, and later when things are fixed for 1.3, enable 1.3 without changing base api/abi requirement of the openssl library.
Reacted by Yahor Siarheyenka@rvagg @shigeki At the request of @mhdawson I started to look at #18770 (comment)
There is one more TODO: rebase on master. When rebasing to master the addition of
min_/max_versionconflicts with #23820 I'm working on this in https://github.com/sam-github/node/tree/WIP_upgrade_openssl111_tls12_only, it builds, but I'm reviewing to make sure its correct.40 remaining items
@kapouer I could use some help, what I found was:
- grpc doesn't support
npm test. I poked around a bit, tried to run find
a way to run the tests, but they always fail because of a missing deps/grpc/etc/roots.pem - @kapouer/uws
- npm install includes no tests (normal), package.json points at upstream
- found https://github.com/kapouer/uWebSockets
- doesn't support
npm test, and I couldn't find any tests
- grpc doesn't support
... but they always fail because of a missing deps/grpc/etc/roots.pem ...
Regarding the missing PEM file, the following may help:
echo "-----BEGIN CERTIFICATE----- MIIDdTCCAl2gAwIBAgILBAAAAAABFUtaw5QwDQYJKoZIhvcNAQEFBQAwVzELMAkG A1UEBhMCQkUxGTAXBgNVBAoTEEdsb2JhbFNpZ24gbnYtc2ExEDAOBgNVBAsTB1Jv b3QgQ0ExGzAZBgNVBAMTEkdsb2JhbFNpZ24gUm9vdCBDQTAeFw05ODA5MDExMjAw MDBaFw0yODAxMjgxMjAwMDBaMFcxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9i YWxTaWduIG52LXNhMRAwDgYDVQQLEwdSb290IENBMRswGQYDVQQDExJHbG9iYWxT aWduIFJvb3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDaDuaZ jc6j40+Kfvvxi4Mla+pIH/EqsLmVEQS98GPR4mdmzxzdzxtIK+6NiY6arymAZavp xy0Sy6scTHAHoT0KMM0VjU/43dSMUBUc71DuxC73/OlS8pF94G3VNTCOXkNz8kHp 1Wrjsok6Vjk4bwY8iGlbKk3Fp1S4bInMm/k8yuX9ifUSPJJ4ltbcdG6TRGHRjcdG snUOhugZitVtbNV4FpWi6cgKOOvyJBNPc1STE4U6G7weNLWLBYy5d4ux2x8gkasJ U26Qzns3dLlwR5EiUWMWea6xrkEmCMgZK9FGqkjWZCrXgzT/LCrBbBlDSgeF59N8 9iFo7+ryUp9/k5DPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8E BTADAQH/MB0GA1UdDgQWBBRge2YaRQ2XyolQL30EzTSo//z9SzANBgkqhkiG9w0B AQUFAAOCAQEA1nPnfE920I2/7LqivjTFKDK1fPxsnCwrvQmeU79rXqoRSLblCKOz yj1hTdNGCbM+w6DjY1Ub8rrvrTnhQ7k4o+YviiY776BQVvnGCv04zcQLcFGUl5gE 38NflNUVyRRBnMRddWQVDf9VMOyGj/8N7yy5Y0b2qvzfvGn9LhJIZJrglfCm7ymP AbEVtQwdpf5pLGkkeB6zpxxxYu7KyJesF12KwvhHhm4qxFYxldBniYUr+WymXUad DKqC5JlR3XC321Y9YeRq4VzW9v493kHMB65jUr9TU/Qr6cf9tveCX4XSQRjbgbME HMUfpIBvFSDJ3gyICh3WZlXi/EjJKSZp4A== -----END CERTIFICATE-----" > globalsign-root-r1.pem wget -q --ca-certificate=globalsign-root-r1.pem https://curl.haxx.se/ca/cacert.pem -O cacert.pem mkdir -p deps/grpc/etc/ cp cacert.pem deps/grpc/etc/roots.pemI can only say "may" because I don't know what is expected in
roots.pem.Helps a bit, still doesn't build. I ran all the gulp targets, twice (in case there was an order dependence). Always fails with:
> grpc@1.16.1 build /home/sam/s/grpc-node/packages/grpc-native-core > node-pre-gyp build make: Entering directory '/home/sam/s/grpc-node/packages/grpc-native-core/build' make: *** No rule to make target 'Release/obj.target/grpc/deps/grpc/src/core/lib/surface/init.o', needed by 'Release/obj.target/libgrpc.a'. Stop.If you can give me directions to test from a clean git clone, I will try again. You can do it, too, might be easier, just build my update_openssl1.1.a-v10.x branch.
There doesn't seem to be any blockers, or activity, so at @mhdawson 's suggestion, I have PRed the update to OpenSSL 1.1.1a: #25381
Also, I have a branch on top of the basic openssl1.1.1a for TLSv1.2 (and below) where I continue to work through the test suites getting them to either pass when run with TLSv1.3, or identify outstanding work. I'm making progress. I think I've identified how to resolve the main issues, but its hard to know when one of the tests will reveal a nasty new problem, so I will wait until I'm through them before feeling too confident.
@shigeki @rvagg TLS1.3 update. I've run through all the test-tls- tests, and found the problems with TLS1.3, as well as read the OpenSSL release notes. I've prototyped solutions to all the problems, except the problems with the info callback. For that, I'm trying to change TLSWrap to use SSL_do_handshake() until the handshake completes. I have had some success with simple connections, but there are still problems, and I'm deep in the weeds of node C++ streams, and reentrant C++/JS.
If either of you have any cred with the OpenSSL team, please, please jump in on this thread: https://mta.openssl.org/pipermail/openssl-project/2019-January/thread.html#1204
It looks like nginx and haprox are also affected by the unnecessary API breakage between openssl 1.1.0 and 1.1.1. I can't even post to the relevant mailing list (Matt Caswell claims its moderated, but AFAICT, my emails are just bounced), maybe you two have access?
If the changes made in that thread occur, then the TLS1.3 handshake problem melts away, and I think I can get TLS1.3 working in relatively short order, just a couple new APIs, some docs, etc.
@nodejs/tsc I've no idea how the OpenSSL core team works, but ☝️ , perhaps an official request from node is in order? I (and others on that thread) believe this to be an API breakage between releases of OpenSSL that are not supposed to have API breakage. There was a judement call made, but with implementation experience available, I think they made it the wrong way.
Reacted by Jonathan Horowitz and Andriy KhamurdaSince you are working in C++ already, maybe Jack Lloyd's Botan would be a better for Node. Botan is a C++ TLS library. It is a cross-platform library and runs on the major platforms, including BSDs, AIX, Linux, OS X and Windows.
Development is active and the code is very clean. Jack also takes a more disciplined approach to the software engineering process (than many other projects found on the web).
One thing it is missing is Andy Polyakov's hand tuned assembly language routines. But Botan uses builtins and intrinsics so performance is not off by much. He uses builtins because of mostly universal compiler support. You can write once, run anywhere.
Something to think about in the future...
@noloader Thanks for weighing in, but switching libraries is off-topic for this thread, as well as not easily done (we need support for more than the major platforms, and we provide the OpenSSL API to native add-ons, so changing libraries is a significant breaking change).
TLS1.3 draft PR: #26209
@sam-github openssl 1.1.1b was released with the api fix. Maybe that means node 10 could upgrade to openssl 1.1.1b without breaking abi ?
node 10 could upgraded to openssl 1.1.1a without breaking ABI or API, and it likely will: #26270
But openssl1.1.1b means I should be able to remove a work-around from Node.js, yes (though the work-around does no harm, it basically just removes TLS1.3 cipher suites, but TLS1.3 isn't supported yet anyhow)(and the workaround goes away in the TLS1.3 PR).
We'll see when #26327 lands if it does so in time to make the next 10.x. It might not, because it would need to be in an 11.x release for a while first, as I understand it. That's OK, once 10.x is using openssl 1.1.1a, then updating to 1.1.1b can be done in a patch release.
/cc @MylesBorins @rvagg
Actually, I can't remove the work-around from Node.js even with openssl1.1.1b, because its possible to link node.js against a system openssl, and it could be openssl1.1.1a. That's OK. Like I said, the work-around disappears in #26209
We're up to openssl-1.1.1b in master, #26327, and TLS1.3 has landed, #26209. Thanks for all the people who helped with this.
I think this issue can be closed now, but if someone feels differently, please reopen.
And that's not to imply that help isn't still wanted for tls/crypto support. More people working on it would be very much appreciated!
@sam-github I‘m not sure, whats working automatically.
getcurves should show up with X25519 and X448. Both curves should be usable inside ECDH class e.g. the Alice and Bob example from the docs.
The complete test changes for X448 in openssl you can see here
rhuijben/openssl@fe93b01I've opened up #26626 for that particular feature request.
@nodejs/crypto
OpenSSL 1.1.1-pre1 was released today. The headline item is TLS 1.3 (worth noting that the spec hasn't quite been finalised yet). This is obviously only a pre-release, not final and not supposed to be entirely bug free.
The OpenSSL team have said previously that 1.1.1 would be API and ABI compatible with 1.1.0. We currently have 1.1.0 support in Node so the theory goes that it shouldn't be too difficult an upgrade path. This is nice because it's possible (but not yet known) that 1.1.1 is the next LTS of OpenSSL, with 1.1.0 going EOL soon. 1.1.0 -> 1.1.1 or just straight to 1.1.1 might have to be our Node 10 strategy (I'm outlining that case here).
So, getting as close to 1.1.1 support as possible even while it's pre-release would be very valuable for us. Maintaining 1.0.2 and 1.1.0 support in the meantime is preferable (perhaps essential thanks to distribution dependencies). There will be a time, after 1.0.2 EOL next year, that we can ditch all the cruft but for now if we can do all 3 then that's what we should do.
Our CI tests 1.0.2 (obviously) and 1.0.2 dynamically linked. It also tests dynamic linking to 1.1.0 in Node 9+ (soon 8 too I think). See https://ci.nodejs.org/job/node-test-commit-linux-containered/ for this happening.
I've also 1.1.1-pre1 to the same containers that are used to run these other dynamic-linked tests and I can turn that on as needed. For now it's too broken to turn on full-time, so this is the call to help fix that!
Node compiles just fine with 1.1.1-pre1 thanks to @davidben's most excellent work in #16130. But it currently fails 55 tests in CI (there may be at least one async-wrap flaky in there).
We need help figuring out whether these are things that we can fix on our end or whether they are upstream problems. If OpenSSL 1.1.1 isn't properly API compatible with 1.1.0 then I'd like us to push back on them to get them to stick to that commitment.
Full output is captured here https://github.com/proxy/gist.github.com/rvagg/cdead09ffa269453d728dcf9bc831d3d (it comes from here but that link is not going to survive).