Difference Between Varnish `(pipe)` and `(pass)`

I'm not a Varnish expert by any means, but this has been my experience and my understanding on pipe vs. pass.

With pass varnish acts as a regular HTTP proxy; it reads the request and pushes it onto the backend(Apache/Nginx)

Pipe, on the other hand, turns varnish into TCP proxy, in the context of Magento this comes handy when the backend returns a file as a CSV, pdf or downloadable file. For example:

if ( req.url ~ ".*/orderprint/.*"){        set req.http.connection = "close";        return(pipe);}

Without we would constantly see timeouts when serving files or customers trying to download their downloadable products.

Allan basically sums it up pretty well, to be a bit more detailed we can take a look into Varnish and compare both pass() and pipe():

PIPE:

https://github.com/varnish/Varnish-Cache/blob/master/bin/varnishd/cache/cache_req_fsm.c#L552

In cnt_pipe(...) varnish handles the return (pipe); command.bo = VBO_GetBusyObj(wrk, req); generates the new bo (backend-request object) from the current req request object.

http_PrintfHeader(bo->bereq, "X-Varnish: %u", VXID(req->vsl->wid));http_SetHeader(bo->bereq, "Connection: close");VCL_pipe_method(req->vcl, wrk, req, bo, NULL);

at this point Varnish sets the X-Varnish and the Connection: close header on the backend request, and calls VCL_pipe_method (which is taken from the VCL file), so, if you want, you can manipulate the backend request afterwards. The Connection: close does not need to be set explicitly anymore in your VCL.

After some checks Varnish calls VDI_Http1Pipe(req, bo) (within the if-statement).VDI_Http1Pipe calls http1pipe, which calls V1F_SendReq to send the requests and handles all the HTTP stuff. V1P_Process send the data back to the actual client (which is usually the Browser accessing the page).

Back in cnt_pipe the function return return (REQ_FSM_DONE);, so the internal state-machine is done and the request handling is done.

Summed up: Beside explicitly closing the connection and setting the X-Varnish header on the backend request Varnish does nothing else.

PASS:

https://github.com/varnish/Varnish-Cache/blob/master/bin/varnishd/cache/cache_req_fsm.c#L518

Now let's take a look into cnt_pass, which is the function that handles return (pass); from the VCL:Instead of directly calling the VDI_Http1Pipe method Varnish calls VBF_Fetch, which uses Pool_Task to queue the task (the task is to fetch the backend response and deliver that to the client).

When that task is done cnt_pipe cleans up the memory and returns return (REQ_FSM_DONE); to ensure there is no additional work to be done, that includes calling deliver in your VCL file and so on.

CONCLUSION:

pipe does not actually act as a TCP-Pipe as Allan said, but is very similiar. Calling pipe basically tells Varnish to send the request, deliver the result and then stop caring about anything (including logging etc.). Basically some kind of fire-and-forget.

pass also sends out the requests and returns the result to the client, but will ensure that logs are written and Varnish is able to handle it's own stuff, also the deliver function of your VCL is called, which not happens when you use pipe.

My understanding, is a (pipe) reads the data and writes it straight out, and a (pass) reads the data in, looks at it and writes it back to the client. The latter adding Varnish headers and the like, the former adding/subtracting nothing. So a (pipe) is as if Varnish doesn't exist.

Or put another way: I think I'm aiming for a schrodinger cat comparison. (pipe) just gives you the box. (pass) looks in the box first.

A (pipe) will also mean that anything that was added to the request up to the call to (pipe), say for example in beresp, is discarded.

One could argue that you would want the (pipe) to be as early as possible for performance but I doubt you would find any metrics on it to really support that argument.