How do I save files to hard disk in a separate thread? How do I save files to hard disk in a separate thread? multithreading multithreading

How do I save files to hard disk in a separate thread?


I think you should avoid starting a new thread for each particular image. Since you have got just a single hard drive and store all files into the single directory, you should use just one disk writer thread. Then I'd recommend using some concurrent queue to transfer jobs from camera thread to disk writer thread. I don't show "code snippet" because this is not a thing you can write in good quality in a few lines of code.

Also you definitely must somewhere put 'new short[20000]' for each image, otherwise it is overwritten by next image before you save it to disk.

Also, I would expect that it is sufficient to write files in the main thread, because Windows uses concurrent techniques (mainly disk cache) automatically when you write data to disk. Are you sure that your hardware is fast enough to write all those data in real time?


When dealing with threads, ordering is no longer in your control. The thread pool can choose to schedule the threads in any order it likes. If you need things to happen sequentially in a specific order, threading does not make much sense anyway.

Regarding the corrupted images, it looks like the short[] image1 instance is being passed around. It is unclear what happens inside ReadImageFromCamera, but since you pass a pre-initialized array into it, chances are that the method will use that array and simply copy data into it (even though the ref keyword indicates that it might create a brand new array instance and assign that instead). Then you pass that array instance to WriteImageToFile on a separate thread.

Meanwhile, in parallell, you get the next image. Now you have a scenario where ReadImageFromCamera might write data into the array at the same time as WriteImageToFile is storing the data on disk. There you have your corrupted image. This can be avoided by passing a new array instance to WriteImageToFile:

ReadImageFromCamera(ref image1)ImageData data;    data.fileName = imageNumber;data.image = (short[])image1.Clone(); // create a new array instance, so that                                      // the next call to ReadImageFromCamera                                       // will not corrupt the dataThreadPool.QueueUserWorkItem(WriteImageToFile, data); 

Still, as has been mentioned by Al Kepp, since you have only one hard drive, launching many threads might not be your best option here. You could look into having one long-running separate thread for storing data on disk, and putting the images into some sort of queue that the storage thread picks up data from and writes to disk. This comes with its own set of problems dealing with concurrency, limiting the size of the queue and what not.


You need to create a distinct buffer for the thread to read data from, otherwise main thread will overwrite it when you dump it to a file. The way you are doing it seems to copy only references (image1 in particular).

So:

ThreadPool.QueueUserWorkItem(WriteImageToFile, data);

instead of data you'll send in a deep copy of data. Since it seems you are already doing it - but in the worker thread - you just need to move the copy before sending it.

HTH