So I ran across this problem when trying to consume a RESTful service. I couldn’t figure out why I was seeing nice response bodies in Advanced REST client but in the Node debugger, a bunch of noise. Eventually I saw this in the response header:
'content-encoding': 'gzip'
That’s when it dawned on me what was happening. This response header was telling me that the server was sending back compressed data. So I spent some time chasing down how to gunzip in Node. Interestingly, my requests never asked for compressed http. A brief overview of http compression can be found here.
The example below removes all dependencies on third party libraries and focuses on solution free from distraction. It uses Node’s Http and Zlib objects which are part of Node (v.10.22 at the time of writing this). If you are using Express or some other modules to handle your Http requests, you’ll need to get your data into Zlib in a slightly different way but the approach should be similar.
var zlib = require('zlib');
var http = require(‘http’);
var req = http.request(options, function (res) {
res.on('data', function (data) {
debugger;
// for some reason response from this route is gzipped...
// see the header 'content-encoding': 'gzip'
// so unzip it.
zlib.unzip(data, function(err, gunzipped){
if(err){
// log the error with your logging utility
// respond with an appropriate error.
}
var message = JSON.parse(gunzipped);
// do something here with your gunzipped message body
// send a response
});
});
});
req.write(JSON.stringify({<< Some response body here >>}));
req.end();
req.on('error', function (e) {
debugger;
// log the error
// send an appropriate response.
});
};
The trick lies in the zlib.unzip method. It expects the data from the http.request’s onData event to be passed in. Once gunzipped, we can do what we want with it. In my case I am expecting JSON bodies so I parse it into an object I can use.
In the rest of the example, the req.write, req.end, req.on(‘error’) are standard approaches to using the Http object to make a request to some end point.
It might be a good idea to check the 'content-encoding' header before gunzipping, even when you expect it. The service provider could change the the header underneath you and cause a problem. The service provider shouldn’t make such a change but that discussion is another matter entirely.
