mercredi 29 avril 2015

How to read a large file of Strings in chunks, each time from different offset, in Java?

I have a large file that consists of several lines, where each line is actually one word of English. The complete file does not fit in memory. As such, I want to process it chunk by chunk. So, I need to implement something like this:

  1. Method1 : Read one chunk, call Method 2.
  2. Method2: Do some processing on the strings in that chunk and go back Step 1.

I have two questions.

1. How can one implement Method1? I know very well how to implement Method2.

I know how to use BufferedReader to read a large file line-by-line. e.g.,

BufferedReader br = new BufferedReader(new FileReader(file)) {
    String line;
    while ((line = br.readLine()) != null) {
       // process the line.

But I would like to read more lines instead of one line, and process all of them in Method2. Then, when I am back to Method1, I want to load several lines once again...

2. My assumption is that it is more efficient (performance wise), to process a chunk of lines rather than process each line. Is this assumption correct?

Aucun commentaire:

Enregistrer un commentaire