Repository navigation
Investigate compressing TS Server socket communication #43156
Description
Problem
On desktop, TS Server uses a json based protocol. This protocol up being quite verbose for larger documents, especially for requests such as navTree which return results from the entire file
Just opening checker.ts for example results in individual requests and responses that are a few MB in size.
Proposal
We've seen up to a 10x reduction of on the wire data by enabling websocket per-message compression for VS Code's internal communications. We aren't using web sockets here, but I believe we should be able to fairly easily compress individual messages on the server side and decompress them on the editor side. Most importantly, we could enable this without any changes to the json protocol itself.
For backwards compatibility, this would have to be an opt-in feature
Activity
RyanCavanaugh commentedon Mar 9, 2021
Matt Bierner (@mjbvz) let's discuss more next Wednesday.
mjbvz commentedon Mar 9, 2021
Sounds good. Just a few quick notes from an initial investigation :
For small messages, the savings by compressing the message body using
deflateare minimal. For example going from 76 bytes to 68 bytes. However for large messages, the savings are much better, with a quick example I pulled up going from14529to1884(7x size reduction)From my (limited) understanding, if we used
gzipfor compression we could avoid using thecontent-lengthon wire header entirely since gzip already includes its own header. This would simplify parsing on the client side, although I haven't found good node examples of reading multiple gzip compressed messages from a streamDirk Bäumer (@dbaeumer) Does the LSP use compression? We would like to align with the LSP if possible here
amcasey commentedon Mar 10, 2021
I wonder if there's an intermediate approach where only the
bodyproperty of the response is compressed? That might make it easier for the server to decide per-message whether compression is worthwhile (assuming the client has opted in via user-preference).dbaeumer commentedon Mar 12, 2021
Matt Bierner (@mjbvz) JSONRPC does support compression and even a different encoding (for example MessagePack). But this currently needs to be setup on the client and server side when the connection is created. The LSP spec does currently not have support to upgrade a connection from an uncompressed to a compressed state.
dbaeumer commentedon Mar 12, 2021
To clarify: compression and encoding is pluggable. So you can choose whatever you want as long as client and server do the same :-)
Kingwl commentedon Apr 1, 2021
As far as I know. We may have to apply base64 encode. but that will increase (about) 1/3 size.
I thought we may just add some header (As
Content-Length) likeContent-Encoding?dbaeumer commentedon Apr 1, 2021
Wenlu Wang (@Kingwl) I looked into that approach in LSP / JSONRPC and it makes things complicated with notification. In addition
Content-Encodingheader value in the response would require aAccept-Encodingheader in the request. Otherwise the server doesn't know which content encoding the client understands.For LSP I decided to propose the following route: during initialization handshake the client & server agree on a specific encoding. After the handshake the connection will be upgraded to that encoding or left at the default if no agreement could be reached.
Kingwl commentedon Apr 1, 2021
Seems we are repeating http protocol. :XD.
Another problem is TypeScript does not have external dependency (If i'm correct). So will we add something (for gzip) dependency, or just implement yet another compressor?