Load files bigger than 1M from assets folder Load files bigger than 1M from assets folder android android

Load files bigger than 1M from assets folder


Faced the same issue. I've cut up my 4MB file into 1 MB chunks, and on the first run I join the chunks into a data folder on the phone. As an added bonus, the APK is properly compressed. The chunk files are called 1.db, 2.db, etc. The code goes like this:

File Path = Ctxt.getDir("Data", 0);File DBFile = new File(Path, "database.db");if(!DBFile.exists() || DatabaseNeedsUpgrade)  //Need to copy...    CopyDatabase(Ctxt, DBFile);static private void CopyDatabase(Context Ctxt, File DBFile) throws IOException{    AssetManager assets = Ctxt.getAssets();    OutputStream outstream = new FileOutputStream(DBFile);    DBFile.createNewFile();    byte []b = new byte[1024];    int i, r;    String []assetfiles = assets.list("");    Arrays.sort(assetfiles);    for(i=1;i<10;i++) //I have definitely less than 10 files; you might have more    {        String partname = String.format("%d.db", i);        if(Arrays.binarySearch(assetfiles, partname) < 0) //No such file in assets - time to quit the loop            break;        InputStream instream = assets.open(partname);        while((r = instream.read(b)) != -1)            outstream.write(b, 0, r);        instream.close();    }    outstream.close();}


The limitation is on compressed assets. If the asset is uncompressed, the system can memory-map the file data and use the Linux virtual memory paging system to pull in or discard 4K chunks as appropriate. (The "zipalign" tool ensures that uncompressed assets are word-aligned in the file, which means they'll also be aligned in memory when directly mapped.)

If the asset is compressed, the system has to uncompress the entire thing to memory. If you have a 20MB asset, that means 20MB of physical memory is tied up by your application.

Ideally the system would employ some sort of windowed compression, so that only parts need to be present, but that requires some fanciness in the asset API and a compression scheme that works well with random access. Right now APK == Zip with "deflate" compression, so that's not practical.

You can keep your assets uncompressed by giving them a suffix of a file type that doesn't get compressed (e.g. ".png" or ".mp3"). You may also be able to add them manually during the build process with "zip -0" instead of having them bundled up by aapt. This will likely increase the size of your APK.


Like Seva suggested you can split up your file in chunks. I used this to split up my 4MB file

public static void main(String[] args) throws Exception {      String base = "tracks";      String ext = ".dat";      int split = 1024 * 1024;      byte[] buf = new byte[1024];      int chunkNo = 1;      File inFile = new File(base + ext);      FileInputStream fis = new FileInputStream(inFile);      while (true) {        FileOutputStream fos = new FileOutputStream(new File(base + chunkNo + ext));        for (int i = 0; i < split / buf.length; i++) {          int read = fis.read(buf);          fos.write(buf, 0, read);          if (read < buf.length) {            fis.close();            fos.close();            return;          }        }        fos.close();        chunkNo++;      }    }  

If you do not need to combine the files into a single file on the device again, just use this InputStream, which combines them into one on the fly.

import java.io.IOException;  import java.io.InputStream;  import android.content.res.AssetManager;  public class SplitFileInputStream extends InputStream {    private String baseName;    private String ext;    private AssetManager am;    private int numberOfChunks;    private int currentChunk = 1;    private InputStream currentIs = null;    public SplitFileInputStream(String baseName, String ext, int numberOfChunks, AssetManager am) throws IOException {      this.baseName = baseName;      this.am = am;      this.numberOfChunks = numberOfChunks;      this.ext = ext;      currentIs = am.open(baseName + currentChunk + ext, AssetManager.ACCESS_STREAMING);    }    @Override    public int read() throws IOException {      int read = currentIs.read();      if (read == -1 && currentChunk < numberOfChunks) {        currentIs.close();        currentIs = am.open(baseName + ++currentChunk + ext, AssetManager.ACCESS_STREAMING);        return read();      }      return read;    }    @Override    public int available() throws IOException {      return currentIs.available();    }    @Override    public void close() throws IOException {      currentIs.close();    }    @Override    public void mark(int readlimit) {      throw new UnsupportedOperationException();    }    @Override    public boolean markSupported() {      return false;    }    @Override    public int read(byte[] b, int offset, int length) throws IOException {      int read = currentIs.read(b, offset, length);      if (read < length && currentChunk < numberOfChunks) {        currentIs.close();        currentIs = am.open(baseName + ++currentChunk + ext, AssetManager.ACCESS_STREAMING);        read += read(b, offset + read, length - read);      }      return read;    }    @Override    public int read(byte[] b) throws IOException {      return read(b, 0, b.length);    }    @Override    public synchronized void reset() throws IOException {      if (currentChunk == 1) {        currentIs.reset();      } else {        currentIs.close();        currentIs = am.open(baseName + currentChunk + ext, AssetManager.ACCESS_STREAMING);        currentChunk = 1;      }    }    @Override    public long skip(long n) throws IOException {      long skipped = currentIs.skip(n);      if (skipped < n && currentChunk < numberOfChunks) {        currentIs.close();        currentIs = am.open(baseName + ++currentChunk + ext, AssetManager.ACCESS_STREAMING);        skipped += skip(n - skipped);      }      return skipped;    }  }

Usage:
ObjectInputStream ois = new ObjectInputStream(new SplitFileInputStream("mytempfile", ".dat", 4, getAssets()));