Disposing MemoryCache in Finalizer throws AccessViolationException

You can't Dispose managed objects in the finalizer, as they might have already been finalized (or, as you've seen here, portions of the environment may no longer be in the state you're expecting.) This means that if you contain a class which must be Disposed explicitly, your class must also be Disposed explicitly. There's no way to 'cheat' and make the Disposal automatic. Unfortunately, garbage collection is, in cases like this, a leaky abstraction.

I would suggest that objects with finalizers should generally not be exposed to the outside world, and should only hold strong references to things which are actually needed for finalization and are not exposed to anything in the outside world that is not expecting them to be used for that purpose. Public-facing types should not have finalizers themselves, but should instead encapsulate cleanup logic within privately-held instances of finalizable classes whose purpose is to encapsulate such logic.

The only time it really makes sense for a finalizer to attempt to clean up a resource which is owned by another object is when the other object is designed to interface with the finalizer. I can't think of any places where Framework classes have engineered in the proper hooks, but will offer an example how Microsoft could have engineered them to do so.

A File object could offer an event with thread-safe subscribe and unsubscribe methods which would fire (notifying the last subscriber first) when the File object receives either a Dispose call or a finalize request. The event would fire between the time Finalize is called and the time the encapsulated file is actually closed, and could be used by a an external buffering class as a signal that it needs to give the File any information which it has received but not yet passed along.

Note that to make such a thing work properly and safely, it would be necessary that the part of the File object which has the finalizer not be exposed to the public, and that it use a long weak reference to ensure that if it runs while the public-facing object is still alive, it will reregister itself for finalization. Note that if the only reference to a WeakReference object is stored in a finalizable object, its Target property may be invalidated if the finalizable object becomes eligible for finalization even if the actual target of the reference is still alive. Defective design, IMHO, and one which must be worked around somewhat carefully.

It's possible to design objects with finalizers that can cooperate (the easiest way to do it, generally, is to only have one object in the group sport a finalizer), but if things aren't designed to cooperate with finalizers, the best that one can do is generally have a finalizer sound an alarm indicating "This object wasn't supposed to be Disposed but wasn't; because it wasn't, resources are going to leak, and there's nothing to be done about it except fix the code to dispose the object properly in future".