How to wait for async actions inside AWS Lambda?
The life of a dev is constantly changing and we now have NodeJS 8 on lambda. For anyone looking at this now check out:
Lambda node 8.10 vs node 6.10 comparison:https://aws.amazon.com/blogs/compute/node-js-8-10-runtime-now-available-in-aws-lambda/
Basics of JS async:https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
Even more aws sdk examples: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/using-promises.html
Details on wtf the .promise() method is in the first link: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Request.html#promise-property
Here is my take at a basic example (try pasting into your own lambda):
exports.handler = async (event) => { function wait(){ return new Promise((resolve, reject) => { setTimeout(() => resolve("hello"), 2000) }); } console.log(await wait()); console.log(await wait()); console.log(await wait()); console.log(await wait()); console.log(await wait()); console.log(await wait()); return 'exiting'};The above yields:
As you can see it waited 12 seconds without killing my function :)
TODO more than one thing per await, use Promise.all([]) syntax like this:
exports.handler = async (event) => { var uploadPromises = []; folder.files.forEach(file => { uploadPromises.push( s3.putObject({ Bucket: "mybucket", Key: file.name, Body: file.data }).promise()); }); await Promise.all(uploadPromises); return 'exiting'}; Orignal Answer Below
I had the exact same issue on my hands.
The problem is the javascript event loop is empty so Lambda thinks it's done.
This is how I solved this problem. I realize this is not ideal, and I wish there was a better way, but I didn't want to a) add libraries, b) coordinate lambda invocations, or c) switch to another language.
At the end of the day it works.
exports.handler = (event, context, callback) => { var response; var callBackCount; /* Ensures the javascript event loop is never empty. This is the key to keeping lambda from exiting early */ setInterval(function(){}, 1000); /* Tell lambda to stop when I issue the callback. This is super important or the lambda funciton will always go until it hits the timeout limit you set. */ context.callbackWaitsForEmptyEventLoop = false; //My way of determining when I'm done with all calls callBackCount = 0; //My info to return response = ""; //Various functions that make rest calls and wait for a response asyncFunction1(); asyncFunction2(); asyncFunction3(); //Same for asyncFunction 2 and 3 function asyncFunction1(){ response += callBackResponseForThisMethod; returnResponse(); } function returnReponse(){ callBackCount++; if(callBackCount == 3){ //Lambda will stop after this as long as context.callbackWaitsForEmptyEventLoop was set to false callback(null, JSON.stringify(response)); } } };http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html
Using async/await
let AWS = require('aws-sdk');let lambda = new AWS.Lambda();let data;exports.handler = async (event) => { try { data = await lambda.getAccountSettings().promise(); } catch (err) { console.log(err); return err; } return data;};
async is not included but that does not mean you cannot add it yourself. Simply add the package locally (npm install async), and include the node_modules folder in your ZIP before uploading your Lambda function.
If you want to handle dev dependencies separately (e.g.: test, aws-sdk to execute your function locally, etc), you can add them under devDependencies in your package.json. Also, if you want to automate the process of developing, testing, deploying and promoting your code, these two repos will turn out to be very handy.
Grunt routine to test, package and deploy your lambdas
Command line tool for running and deploying your lambda functions