How to lock a file on different application levels?

You've enumerated the possible solutions except the obvious one: remove the dependency on that file

Is there another way for the threads to obtain that data instead of reading it from a file? How about setting up some kind of process who is responsible for coordinating access to that information instead of having all the threads read the file.

A. Sounds like it's time for a database :-).Rather than having a shared file what about storing the data in a database.

B. Alternatively - layering:

1. Lock threads within a process with a standard synchronized lock.
2. Lock inter-process/machine with a file-based lock type thing - e.g. create a directory to hold the lock.

Nest 2 inside 1.

Still has the clean-up problem.

C. Alternatively some kind of write to new file/rename strategy so that reader don't need to lock maybe?

The most simple solution is to create another process (web service or whatever is most simple to you). Only this process reads/writes the file and it listens to read/write requests by the other services.

While it might seem that this is slower than using the network share directly, that's not necessarily the case: Using a network share means to use a client/server which is built into your OS (which does exactly that: send read/write requests to the server which offers the share).

Since your service is optimized for the task (instead of being a general "serve file" service), it might even be faster.