Node running out of memory in simple loop Node running out of memory in simple loop mongoose mongoose

Node running out of memory in simple loop


You can take advantage of the async nature of Property.create to allow GC to occur while your code waits for the records to be created. In its current form, your code synchronously queues up all 1M records to create.

You can do this by adding a callback parameter to your createRandomSeeds method that you call when the Property.create call has completed and then replace your synchronous for loop with one that iterates asynchronously like the whilst method of the async library.

But I'd also recommend not creating 1000 docs in a single create call; 100 is probably a reasonable maximum. So note that change as well in the below example:

var createRandomSeeds = function (index, callback) {    var createSeedsTestPropertyData = [];    for (var i = 0; i < 100; i++) {        createSeedsTestPropertyData.push({            name: i.toString(),            location: [              getRandomRange(+90, -90), getRandomRange(+180, -180)            ]        });    }    Property.create(createSeedsTestPropertyData, function(err) {        console.log('Created many records, ' + index);        callback(err);    });}var i = 0;async.whilst(    // Keep looping asynchronously while i < 10000    function () { return i < 10000; },    function (callback) {        i++;        createRandomSeeds(i, callback);    },    function (err) {        console.log('All done');    });