large data download and local processing and storage for monotouch app large data download and local processing and storage for monotouch app sqlite sqlite

large data download and local processing and storage for monotouch app


My suggestion would be to do your work asynchronously - and you're lucky since C# makes that very easy. E.g.

  1. Start a background download;
  2. Process (background) the object as they are downloaded;
  3. Insert (background) objects as they are processed;
  4. If applicable update the UI (from the main thread) for every X object you add;

Since the download is (mostly, see note) network bound then your CPU will be idle for many seconds. That's a waste of time considering your next step (processing) will be CPU bound. Even more since the step afterward will likely be I/O bound (database).

IOW it looks like a good idea to run all three tasks simultaneously while giving feedback of the progress (showing data or a progress bar) to the application user.

Note #1: A gzipped response will download faster. OTOH it will take some extra (CPU) time to uncompress locally. It should be faster but it's worth measuring both options (e.g. using Apple's Instrument tool, which works nicely with Xamarin.iOS).

Note #2: A zip file, as a response, will also need extra time (to uncompress). That's not something you want to do sequentially after the download (but you could uncompress it as it's downloaded).