How to make a CSV upload perform faster? How to make a CSV upload perform faster? codeigniter codeigniter

How to make a CSV upload perform faster?


Why not run the profiler to optimize your code? Codeigniter includes this useful piece for problems like this http://codeigniter.com/user_guide/general/profiling.html

It will give you a breakdown of your SQL queries and what is taking long, and where.

$this->output->enable_profiler(TRUE);


The problem seems to me that you are querying the DB once (or twice ?) per line in your CSV file.
Of course you're going to get horrible performance.
You can do the whole query in one go and have the DB make the CSV for you in no time.

SELECT DISTINCT f1,f2,f3,... FROM tablex WHERE .. INTO OUTFILE 'c:/dir/ca.csv'   FIELDS ESCAPED BY '"' FIELDS TERMINATED BY ';' LINES TERMINATED BY '\n';//note the use of forward slashes even on Windows.

See: http://dev.mysql.com/doc/refman/5.0/en/select-into.html

The speed of the select itself is the limiting factor here.
Make sure you have write permissions on the directory and note that MySQL will never overwrite files.
This command is very fast on MySQL.


$id = $id;

really?

$ckr = $this->Manager_model->check_if_record_exists($id, $id2);

One obvious way to make it go faster would be to have a unique index on eid and id2 and ignore duplicate row errors on the INSERT.

But really, f you want it to go much faster, just tell mysql to parse and load the data.