How to structure a Symfony2 app with ESI?

We have used Varnish on one site for whole-page caching and I've been using Symfony2 for few years, but keep in mind that I haven't used Varnish + Symfony2 + ESI on any production environment.

1. I think the basic idea is OK. If menu is the same in many pages and list of places also the same on many pages, you get common content cached by Varnish or Symfony reverse cache. As Varnish usually holds cache in memory, you get your content faster and don't have to call rendering and DB querying code at each request.

   The hard part is making those ESI requests cached if the user is logged in. As I know, in default Varnish configuration, requests with Cookie in them are never cached. If you tend to pass cookies to ESI requests, those ESI responses will not be shared between users.

   You can try making some rules from URL, but if you use default Symfony twig helpers, generated URLs are /_internal/..., so it might be hard to differ public and private ones.

   Also you can configure to always ignore any cookies if Cache-Control: public is passed. This is done by default in Symfony:

   if ($this->isPrivateRequest($request) && !$response->headers->hasCacheControlDirective('public')) {    $response->setPrivate(true);}

   As you see from the code, if you have public directive, response will never be private.

   I haven't found how Varnish processes this directive - as I understand, it does not cache any requests that have cookie by default. So I think you have to tweak configuration to accomplish this.

2. If the main page is also to be cached, I don't see how you could skip the includes.

   I assume JS is required for your registered users (not search bots), so I would suggest to use Javascript to differ the loading of user data.

   Javascript code can look if the user has cookie session-id etc. and make request to get the data only in this case. It might also be a good idea to set some other cookie, like _loggedin to avoid Javascript code from getting the session id.

   Not logged in users can also have some data in the cookies, like _likedPost:1,2,132. Javascript can get this cookie and make some HTML corrections without even making the additional request.

   As we did with these cookies: we separated JS-only cookies from application cookies. We did this by some pattern, like _\w for JS cookies. Then we tweaked Varnish configuration to split Cookie header and remove these JS-only cookies. Then, if there is no other cookie left, the response is shared with everyone. Application (Symfony) does not get those cookies, as they are stripped.

3. I think it does if it is the same in every page.

4. I think ESI is good as Varnish can hold cache in memory. So it might be that it would not even make any queries to your hard disk for the content. As your controller cache might be also in-memory, I think Varnish would look for the cache more quicker than Symfony framework with all the routing, PHP code, services initialization etc.

5. It depends, but I think that it could be better approach. Keep in mind, that caches live different lives. For example, if your places list is cached for 2 hours, at the end of this time places can have changed - some new items are new on the list and some of them are missing. Your list to the user is still the old one (cached), but you provide user's data about the new list - some of the data is not needed, some of it is missing.

   It might be better approach to get loaded places by javascript, for example searching for some HTML attribute like data-list-item-id and then make ajax request querying data about these items. In this case your user data will be synchronized with current cached list and you can make 1 ajax request for both lists instead of 2.

6. If cache invalidation (PURGE requests) are not used, all HTTP cache sheme is indeed good to scale. You can scale the application to several servers and configure Varnish to call them randomly, by some rule or just to use one of them as a failsafe. If the bandwidth is still too big, you can always modify the cache timeouts and other configuration.