Node.js Axios behind corporate proxies

A short guide to digging tunnels.

Oh, the corporate proxy, how I love thee.

Proxy servers are an integral part of any corporate environment. They’re supposed to improve corporate and institutional security, anonymise and balance internet traffic, speed up and save network bandwidth and control employee internet usage. That is, if they’re set up correctly.

Regretfully, many of the software delivery teams I’ve worked with have suffered from poorly configured and neglected IT infrastructure, including proxies, which instead of yielding those promised benefits have negatively impacted developer productivity.

In this post I’d like to show you how I made Axios, one of my favourite Node.js HTTP clients, play with one such problematic corporate proxy.

The goal

What me and my team wanted to accomplish was to run automated Serenity/JS acceptance tests against an externally deployed system. But, we needed to do that from within a corporate network which sits behind a corporate proxy.

Sounds easy enough, right?

The challenge

The system in question exposed a number of REST endpoints that we needed to interact with, and it did so over HTTPS.

And here lies the problem. Our corporate proxy only supported HTTP traffic, not the HTTPS that we needed.

So how do you connect to a HTTPS endpoint through a HTTP proxy?

The answer is boring. Yes, you actually do a bit of boring. As in: you dig a tunnel. A HTTPS-over-HTTP tunnel to be specific.

In this post I’ll show you how to do that with Axios, TypeScript and Node.js.

Axios and proxies

The axios client is great — it can automatically detect your environment variables, such as thehttps_proxy, and works with any standard proxy setup out of the box:

import axios, { AxiosInstance } from 'axios';
const axiosClient: AxiosInstance = axios.create({
    baseURL: 'https://some.api.com',
});

But what if you don’t have the HTTPS proxy? You combine axios with another npm library — tunnel to establish a HTTPS-over-HTTP tunnel:

import axios, { AxiosInstance } from 'axios';
import * as tunnel from 'tunnel';
const agent = tunnel.httpsOverHttp({
    proxy: {
        host: 'proxy.mycorp.com',
        port: 8000,
    },
});
const axiosClient: AxiosInstance = axios.create({
    baseURL: 'https://some.api.com',
    httpsAgent: agent,
});

That’s not sufficient enough, however, because running the above example results in below error:

Error: write EPROTO 140736451081152:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:797

What this error means is thataxios got slightly confused and instead of connecting to our HTTPS endpoint atsome.api.com on port 443 (HTTPS), it’s trying to get there through port 80 (HTTP).

To fix this issue we need to explicitly tell Axios which port we’re interested in:

import axios, { AxiosInstance } from 'axios';
import * as tunnel from 'tunnel';
const agent = tunnel.httpsOverHttp({
    proxy: {
        host: 'proxy.mycorp.com',
        port: 8000,
    },
});
const axiosClient: AxiosInstance = axios.create({
    baseURL: 'https://some.api.com:443',  // here I specify port 443
    httpsAgent: agent,
});

And we’re done!

Well, almost done.

Remember when I told you that Axios detects your environment variables automatically? If you have them set up they will interfere with our lovely little tunnel. Because of this, we need to disable automatic proxy detection:

import axios, { AxiosInstance } from 'axios';
import * as tunnel from 'tunnel';
const agent = tunnel.httpsOverHttp({
    proxy: {
        host: 'proxy.mycorp.com',
        port: 8000,
    },
});
const axiosClient: AxiosInstance = axios.create({
    baseURL: 'https://some.api.com:443',
    httpsAgent: agent,
    proxy: false,
});

And now we’re really done.

Next steps

So what did we do with that AxiosInstance after all this set up? We used it together with @serenity-js/rest to implement acceptance tests for our REST APIs using the Screenplay Pattern:

const Rob = Actor.named('Rob').whoCan(CallAnApi.using(axiosClient));
Rob.attemptsTo(
  Post.item(product).on('/basket'),
  See.if(Basket.total(), equals(product.price)),
);

I’ll tell you more about @serenity-js/rest in future posts so follow me on Medium and Twitter to stay up to date!

Until next time,
Jan

P.S.
If you like my work and would like to keep posts like this one coming, please consider buying me a coffee ☕️