Is it possible to read only first N bytes from the HTTP server using Linux command?
You can do it natively by the next curl command (no need to donwload whole document). According to culr man page:
RANGES HTTP 1.1 introduced byte-ranges. Using this, a client can request to get only one or more subparts of a specified document.
curl
supports this with the-r
flag.Get the first 100 bytes of a document: curl -r 0-99 http://www.get.this/Get the last 500 bytes of a document: curl -r -500 http://www.get.this/`curl` also supports simple ranges for FTP files as well.Then you can only specify start and stop position.Get the first 100 bytes of a document using FTP: curl -r 0-99 ftp://www.get.this/README
It works for me even with Java web app that deployed to GigaSpaces.
curl <url> | head -c 499
or
curl <url> | dd bs=1 count=499
should do
Also there are simpler utils with perhaps borader availability like
netcat host 80 <<"HERE" | dd count=499 of=output.fragmentGET /urlpath/query?string=more&bloddy=stuffHERE
Or
GET /urlpath/query?string=more&bloddy=stuff
You should also be aware that many HTTP/1.1 servers do not have this feature enabled, so that when you attempt to get a range, you'll instead get the whole document.
You will have to get the whole web anyways, so you can get the web with curl and pipe it to head, for example.
head
c, --bytes=[-]N print the first N bytes of each file; with the leading '-', print all but the last N bytes of each file