Node running out of memory in simple loop
You can take advantage of the async nature of Property.create
to allow GC to occur while your code waits for the records to be created. In its current form, your code synchronously queues up all 1M records to create.
You can do this by adding a callback
parameter to your createRandomSeeds
method that you call when the Property.create
call has completed and then replace your synchronous for
loop with one that iterates asynchronously like the whilst
method of the async
library.
But I'd also recommend not creating 1000 docs in a single create
call; 100 is probably a reasonable maximum. So note that change as well in the below example:
var createRandomSeeds = function (index, callback) { var createSeedsTestPropertyData = []; for (var i = 0; i < 100; i++) { createSeedsTestPropertyData.push({ name: i.toString(), location: [ getRandomRange(+90, -90), getRandomRange(+180, -180) ] }); } Property.create(createSeedsTestPropertyData, function(err) { console.log('Created many records, ' + index); callback(err); });}var i = 0;async.whilst( // Keep looping asynchronously while i < 10000 function () { return i < 10000; }, function (callback) { i++; createRandomSeeds(i, callback); }, function (err) { console.log('All done'); });