AWS Lambda making video thumbnails
The process as outlined by Naveen is correct, but it glosses over a detail that can be pretty painful - including the ffmpeg binary in the zip and accessing it within your lambda function.
I just went through this, it went like this:
- Include the ffmpeg static binary in your zipped lambda function package (I have a gulp task to copy this into the
/dist
every time it builds) When your function is called, move the binary to a(Update Feb 2017: it's reported that this is no longer necessary, re:/tmp/
dir andchmod
it to give yourself access@loretoparisi
and@allen
's answers).- update your PATH to include the ffmpeg executable (I used fluent-ffmpeg which lets you set two env vars to handle that more easily.
Let me know if more detail is necessary, I can update this answer.
The copy and chmod (step 2) is obviously not ideal.... would love to know if anyone's found a better way to handle this, or if this is typical for this architecture style.
(2nd Update, writing it before the first update b/c it's more relevant):
The copy + chmod step is no longer necessary, as @Allen pointed out – I'm executing ffmpeg in Lambda functions directly from /var/task/ with no trouble at this point. Be sure to chmod 755
whatever binaries before uploading them to Lambda (also as @Allen pointed out).
I'm no longer using fluent-ffmpeg to do the work. Rather, I'm updating the PATH to include the process.env['LAMBDA_TASK_ROOT']
and executing simple bash scripts.
At the top of your Lambda function:
process.env['PATH'] = process.env['PATH'] + "/" + process.env['LAMBDA_TASK_ROOT']
For an example that uses ffmpeg: lambda-pngs-to-mp4.
For a slew of useful lambda components: lambduh.
The below update left in for posterity, but no longer necessary:
UPDATE WITH MORE DETAIL:
I downloaded the static ffmpeg binary here. Amazon recommends booting up an EC2 and building a binary for your use on there, because that environment will be the same as the conditions Lambda runs on. Probably a good idea, but more work, and this static download worked for me.
I pulled only the ffmpeg
binary into my project's to-be-archived /dist
folder.
When you upload your zip to lambda, it lives at /var/task/
. For whatever reason, I ran into access issues trying to use the binary at that location, and more issues trying to edit permissions on the file there. A quick work-around is to move the binary to /tmp/
and chmod
permissions on it there.
In Node, you can run shell via a child_process
. What I did looks like this:
require('child_process').exec( 'cp /var/task/ffmpeg /tmp/.; chmod 755 /tmp/ffmpeg;', function (error, stdout, stderr) { if (error) { //handle error } else { console.log("stdout: " + stdout) console.log("stderr: " + stderr) //handle success } })
This much should give you an executable ffmpeg binary in your lambda function – but you still need to make sure it's on your $PATH.
I abandoned fluent-ffmpeg and using node to launch ffmpeg commands in favor of just launching a bash script out of node, so for me, I had to add /tmp/
to my path at the top of the lambda function:
process.env.PATH = process.env.PATH + ':/tmp/'
If you use fluent-ffmpeg, you can set the path to ffmpeg via:
process.env['FFMPEG_PATH'] = '/tmp/ffmpeg';
Somewhat related/shameless self-plug: I'm working on a set of modules to make building Lambda functions out of composable modules easier under the name Lambduh. Might save some time getting these things together. A quick example: handling this scenario with lambduh-execute would be as simple as:
promises.push(execute({ shell: "cp /var/task/ffmpeg /tmp/.; chmod 755 /tmp/ffmpeg",})
Where promises
is an array of promises to be run.
I created a GitHub repo that does exactly this (as well as resizes the video at the same time). Russ Matney's answer was extremely helpful to make the FFmpeg file executable.
I am not sure what custom mode library you would use for the ffmpeg task; nevertheless the steps to accomplish that are the same.
- Create a separate directory for your lambda project
- Run
npm install <package name>
inside that directory ( this would automatically put in place the node_modules and appropriate files ) - Create
index.js
file in the lambda project directory then use therequire(<package-name>)
and perform your main task for video thumbnails creation - Once you are done, you can zip the lambda project folder and upload it I'm AWS management console and configure the index file and handler.
- Rest of configurations follow the same process like IAM Execution Role, Trigger, Memory and Timeout specification etc.