Proper structuring of Lucene.Net usage in an ASP.NET MVC site

The answer to all three of your questions is the same: reuse your readers (and possibly your writers). You can use a singleton pattern to do this (i.e. declare your reader/writer as public static). Lucene's FAQ tells you the same thing: share your readers, because the first query is reaaalllyyyy slow. Lucene handles all the locking for you, so there is really no reason why you shouldn't have a shared reader.

It's probably easiest to just keep your writer around and (using the NRT model) get the readers from that. If it's rare that you are writing to the index, or if you don't have a huge need for speed, then it's probably OK to open your writer each time instead. That is what I do.

Edit: added a code sample:

public static IndexWriter writer = new IndexWriter(myDir);public JsonResult SearchForStuff(string query){    IndexReader reader = writer.GetReader();    IndexSearcher search = new IndexSearcher(reader);    // do the search}

I would probably skip the caching -- Lucene is very, very efficent. Perhaps so efficent that it is faster to search again than cache.

The OnApplication_Start full index feels a bit off to me -- should probably be run in it's own thread so as not to block other expensive startup activities.