Remove duplicate lines from a large text or given document - Techno junkie

Breaking

Be the Techno giant.

Post Top Ad

Responsive Ads Here

Post Top Ad

Responsive Ads Here

Thursday, 27 September 2018

Remove duplicate lines from a large text or given document


import java.io.*;
import java.util.HashSet;
 
public class FileOperation
{
    public static void main(String[] args) throws IOException 
    {
        PrintWriter pw = new PrintWriter("D://output.txt");
         
        BufferedReader br = new BufferedReader(new FileReader("D://input.txt"));
         
        String line = br.readLine();
         
        // set store unique values
        HashSet<String> hs = new HashSet<String>();
         
        // loop for each line of input.txt
        while(line != null)
        {
            // write only if not
            // present in hashset
            if(hs.add(line))
                pw.println(line);
             
            line = br.readLine();
             
        }
         
        pw.flush();
         
        // closing resources
        br.close();
        pw.close();
         
        System.out.println("File operation performed successfully");
    }
}

Output:-



No comments:

Post a Comment

Post Top Ad

Responsive Ads Here