Splitting files into chunks with size bigger than 127 Splitting files into chunks with size bigger than 127 hadoop hadoop

Splitting files into chunks with size bigger than 127


The problem is that PART_SIZE is a byte; its maximum value is therefore indeed 127.

The code you have at the moment however is full of problems; for one, incorrect resource handling etc.

Here is a version using java.nio.file:

private static final String FILENAME = "TextFile.txt";private static final int PART_SIZE = xxx; // HEREpublic static void main(final String... args)    throws IOException{    final Path file = Paths.get(FILENAME).toRealPath();    final String filenameBase = file.getFileName().toString();    final byte[] buf = new byte[PART_SIZE];        int partNumber = 0;    Path part;    int bytesRead;    byte[] toWrite;    try (        final InputStream in = Files.newInputStream(file);    ) {        while ((bytesRead = in.read(buf)) != -1) {            part = file.resolveSibling(filenameBase + ".part" + partNumber);            toWrite = bytesRead == PART_SIZE ? buf : Arrays.copyOf(buf, bytesRead);            Files.write(part, toWrite, StandardOpenOption.CREATE_NEW);            partNumber++;        }    }}


List<PDDocument> Pages=new ArrayList<PDDocument>();     Document.load(filePath);    try {        Splitter splitter = new Splitter();        splitter.setSplitAtPage(NoOfPagesDocumentWillContain);    Pages = splitter.split(document);    }catch(Exception e)    {        l        e.getCause().printStackTrace();    }