Craft HTTP requests using nc

To do some low level check on websites, I'd usually use telnet to compose a http requests against the server. The main intention is to talk directly to the server port to make sure the problem we have not caused by some higher level application. For example, to connect to server and issue a GET request:-

telnet k4ml.github.io 80
Trying 199.27.75.133...
Connected to github.map.fastly.net.
Escape character is '^]'.
GET /
Connection closed by foreign host.

There's always a problem with telnet. In the above example, I can only issue a GET request without having a chance to add other HTTP headers such as HOST before the server close the connection. Some websites also time out very quickly when they are not receiving any data after establishing connection. And since the above command is in interactive session, it's not repeatable or scripted. Using nc seem to be much better.

Specify the virtual host:-

echo -en "HEAD / HTTP/1.1\r\nHOST: k4ml.github.io\r\n\r\n" | nc k4ml.github.io 80

You'll get the output as:-

HTTP/1.1 200 OK
Server: GitHub.com
Content-Type: text/html
Last-Modified: Fri, 12 Apr 2013 23:26:51 GMT
Expires: Sun, 26 May 2013 19:40:06 GMT
Cache-Control: max-age=600
Content-Length: 9991
Accept-Ranges: bytes
Date: Sun, 26 May 2013 19:30:07 GMT
Via: 1.1 varnish
Age: 0
Connection: keep-alive
X-Served-By: cache-s34-SJC2
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1369596606.987305880,VS0,VE145
Vary: Accept-Encoding

Without virtualhost:-

echo -en "HEAD / HTTP/1.1\r\n\r\n" | nc k4ml.github.io 80

And the output:-

HTTP/1.1 400 Bad Request
Server: GitHub.com
Content-Type: text/html
Content-Length: 166
Accept-Ranges: bytes
Date: Sun, 26 May 2013 19:32:16 GMT
Via: 1.1 varnish
Age: 0
Connection: keep-alive
X-Served-By: cache-s35-SJC2
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1369596736.277374744,VS0,VE72
Vary: Accept-Encoding

It allow us to fully compose the request and then send it through the opened connection nc created.

Reference:-

http://www.philandstuff.com/2013/05/17/statsd-netcat.html