A few days ago, I discussed with somebody whether a file was cached by Cloudflare or not, and this involved getting the HTTP header, and checking for CF-RAY field to see if data is going through one of Cloudflare data centers. This can be done with curl:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
curl -svo /dev/null http://www.cnx-software.com * Rebuilt URL to: http://www.cnx-software.com/ * Trying 104.28.19.95... * Connected to www.cnx-software.com (104.28.19.95) port 80 (#0) > GET / HTTP/1.1 > Host: www.cnx-software.com > User-Agent: curl/7.47.0 > Accept: */* > < HTTP/1.1 200 OK < Date: Mon, 03 Oct 2016 09:57:17 GMT < Content-Type: text/html; charset=UTF-8 < Transfer-Encoding: chunked < Connection: keep-alive < Set-Cookie: __cfduid=d90ff49c11865e8fda1331c2977559f521475488637; expires=Tue, 03-Oct-17 09:57:17 GMT; path=/; domain=.cnx-software.com; HttpOnly < X-Powered-By: PHP/5.5.9-1ubuntu4.19 < Expires: Wed, 11 Jan 1984 05:00:00 GMT < Cache-Control: no-cache, must-revalidate, max-age=0 < Pragma: no-cache < X-UA-Compatible: IE=edge < Link: <http://www.cnx-software.com/wp-json/>; rel="https://api.w.org/" < Server: cloudflare-nginx < CF-RAY: 2ebf876da273114d-SIN < { [2307 bytes data] * Connection #0 to host www.cnx-software.com left intact |
In the command above, -s stands for silent so that curl does not show the progress meter, -v stands for verbose to show the header, and -o /dev/null is used to discard the packet load.
You can also use -I option (fetch the HTTP-header only) with curl, which – if all you need is the HTTP header – provides a cleaner output:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
curl -I http://www.cnx-software.com HTTP/1.1 200 OK Date: Mon, 03 Oct 2016 10:06:51 GMT Content-Type: text/html; charset=UTF-8 Connection: keep-alive Set-Cookie: __cfduid=d4dda8a9ec8370cf0950d26e5faf37cc21475489211; expires=Tue, 03-Oct-17 10:06:51 GMT; path=/; domain=.cnx-software.com; HttpOnly X-Powered-By: PHP/5.5.9-1ubuntu4.19 Expires: Wed, 11 Jan 1984 05:00:00 GMT Cache-Control: no-cache, must-revalidate, max-age=0 Pragma: no-cache Set-Cookie: bb2_screener_=1475489211+1.1.174.2+1.1.174.2; path=/ X-UA-Compatible: IE=edge Link: <http://www.cnx-software.com/wp-json/>; rel="https://api.w.org/" Server: cloudflare-nginx CF-RAY: 2ebf9574c129081d-SIN |
I also came across httpstat Python script recently via n0where, doing much of the same thing, except it also adds transfer statistics.
It can be installed by downloading httpstat.py, or better using pip:
|
1 |
sudo pip install httpstat |
Let’s try it with this very blog:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
python httpstat.py http://www.cnx-software.com HTTP/1.1 200 OK Date: Mon, 03 Oct 2016 10:08:04 GMT Content-Type: text/html; charset=UTF-8 Transfer-Encoding: chunked Connection: keep-alive Set-Cookie: __cfduid=da79c1b07ec6d72643cd80a5da41c2a8f1475489284; expires=Tue, 03-Oct-17 10:08:04 GMT; path=/; domain=.cnx-software.com; HttpOnly X-Powered-By: PHP/5.5.9-1ubuntu4.19 Expires: Wed, 11 Jan 1984 05:00:00 GMT Cache-Control: no-cache, must-revalidate, max-age=0 Pragma: no-cache X-UA-Compatible: IE=edge Link: <http://www.cnx-software.com/wp-json/>; rel="https://api.w.org/" Server: cloudflare-nginx CF-RAY: 2ebf973db3b7319e-SIN Body stored in: /tmp/tmpbk8pAF DNS Lookup TCP Connection Server Processing Content Transfer [ 4ms | 47ms | 277ms | 192ms ] | | | | namelookup:4ms | | | connect:51ms | | starttransfer:328ms | total:520ms |
The header is the same as with Curl -I, and for good reason, since httpstat relies on curl, and all curl options (except for -w, -D, -o, -s, -S) can be used with the command line. This script can be useful if you are trying to benchmark and lower TTFB (Time To First Byte), or decrease overall download times from your server.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
From the United States (Anaheim, Ca)
me@my-system:~/work/httpstat$ httpstat http://www.cnx-software.com/
HTTP/1.1 200 OK
Date: Mon, 03 Oct 2016 23:36:10 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=db98c5339ef89a4215299d7c6436b149d1475537770; expires=Tue, 03-Oct-17 23:36:10 GMT; path=/; domain=.cnx-software.com; HttpOnly
X-Powered-By: PHP/5.5.9-1ubuntu4.19
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
Pragma: no-cache
X-UA-Compatible: IE=edge
Link: ; rel=”https://api.w.org/”
Server: cloudflare-nginx
CF-RAY: 2ec436fbb25b22b2-LAX
Body stored in: /tmp/tmpArRt0r
DNS Lookup TCP Connection Server Processing Content Transfer
[ 124ms | 18ms | 98ms | 49ms ]
| | | |
namelookup:124ms | | |
connect:142ms | |
starttransfer:240ms |
total:289ms