jeudi 5 novembre 2015

How to find similar records of one csv file into another csv file write all records into a result.csv file by python

I would like to search and then copy / merge two CSV files based on the common records of adv_id and user_id in both files based on conv_id, and store records in a new CSV file. That means I will take the records from both CSV files if the adv_id and user_id is same in both files. But when I use nested for loops for to search same adv_id and user_id of fileOne into fileTwo, it takes very long execution times. This is because, fileOne contains 8 Million records, fileTwo contain 50 million records. Below is an example of the files and expected result file. fileOne and fileTwo contain the fillowing columns. Your kind suggestions in highly appreciated.

fileOne: adv_id, conv_id, user_id, dwell, session, pv, elapsed, cvflg.

fileTwo: adv_id, user_id, dwell, session, pv, elapsed, cvflg

Image of fileOne, fileTwo, and Result file

Aucun commentaire:

Enregistrer un commentaire