Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference
One word answer: asynchronicity.
Forewords
This topic has been iterated at least a couple of thousands of times, here, in Stack Overflow. Hence, first off I'd like to point out some extremely useful resources:
@Felix Kling's answer to "How do I return the response from an asynchronous call?". See his excellent answer explaining synchronous and asynchronous flows, as well as the "Restructure code" section.
@Benjamin Gruenbaum has also put a lot of effort explaining asynchronicity in the same thread.@Matt Esch's answer to "Get data from fs.readFile" also explains asynchronicity extremely well in a simple manner.
The answer to the question at hand
Let's trace the common behavior first. In all examples, the outerScopeVar is modified inside of a function. That function is clearly not executed immediately, it is being assigned or passed as an argument. That is what we call a callback.
Now the question is, when is that callback called?
It depends on the case. Let's try to trace some common behavior again:
img.onloadmay be called sometime in the future, when (and if) the image has successfully loaded.setTimeoutmay be called sometime in the future, after the delay has expired and the timeout hasn't been canceled byclearTimeout. Note: even when using0as delay, all browsers have a minimum timeout delay cap (specified to be 4ms in the HTML5 spec).- jQuery
$.post's callback may be called sometime in the future, when (and if) the Ajax request has been completed successfully. - Node.js's
fs.readFilemay be called sometime in the future, when the file has been read successfully or thrown an error.
In all cases, we have a callback which may run sometime in the future. This "sometime in the future" is what we refer to as asynchronous flow.
Asynchronous execution is pushed out of the synchronous flow. That is, the asynchronous code will never execute while the synchronous code stack is executing. This is the meaning of JavaScript being single-threaded.
More specifically, when the JS engine is idle -- not executing a stack of (a)synchronous code -- it will poll for events that may have triggered asynchronous callbacks (e.g. expired timeout, received network response) and execute them one after another. This is regarded as Event Loop.
That is, the asynchronous code highlighted in the hand-drawn red shapes may execute only after all the remaining synchronous code in their respective code blocks have executed:
In short, the callback functions are created synchronously but executed asynchronously. You just can't rely on the execution of an asynchronous function until you know it has executed, and how to do that?
It is simple, really. The logic that depends on the asynchronous function execution should be started/called from inside this asynchronous function. For example, moving the alerts and console.logs too inside the callback function would output the expected result, because the result is available at that point.
Implementing your own callback logic
Often you need to do more things with the result from an asynchronous function or do different things with the result depending on where the asynchronous function has been called. Let's tackle a bit more complex example:
var outerScopeVar;helloCatAsync();alert(outerScopeVar);function helloCatAsync() { setTimeout(function() { outerScopeVar = 'Nya'; }, Math.random() * 2000);}Note: I'm using setTimeout with a random delay as a generic asynchronous function, the same example applies to Ajax, readFile, onload and any other asynchronous flow.
This example clearly suffers from the same issue as the other examples, it is not waiting until the asynchronous function executes.
Let's tackle it implementing a callback system of our own. First off, we get rid of that ugly outerScopeVar which is completely useless in this case. Then we add a parameter which accepts a function argument, our callback. When the asynchronous operation finishes, we call this callback passing the result. The implementation (please read the comments in order):
// 1. Call helloCatAsync passing a callback function,// which will be called receiving the result from the async operationhelloCatAsync(function(result) { // 5. Received the result from the async function, // now do whatever you want with it: alert(result);});// 2. The "callback" parameter is a reference to the function which// was passed as argument from the helloCatAsync callfunction helloCatAsync(callback) { // 3. Start async operation: setTimeout(function() { // 4. Finished async operation, // call the callback passing the result as argument callback('Nya'); }, Math.random() * 2000);}Code snippet of the above example:
// 1. Call helloCatAsync passing a callback function,// which will be called receiving the result from the async operationconsole.log("1. function called...")helloCatAsync(function(result) { // 5. Received the result from the async function, // now do whatever you want with it: console.log("5. result is: ", result);});// 2. The "callback" parameter is a reference to the function which// was passed as argument from the helloCatAsync callfunction helloCatAsync(callback) { console.log("2. callback here is the function passed as argument above...") // 3. Start async operation: setTimeout(function() { console.log("3. start async operation...") console.log("4. finished async operation, calling the callback, passing the result...") // 4. Finished async operation, // call the callback passing the result as argument callback('Nya'); }, Math.random() * 2000);}Most often in real use cases, the DOM API and most libraries already provide the callback functionality (the helloCatAsync implementation in this demonstrative example). You only need to pass the callback function and understand that it will execute out of the synchronous flow, and restructure your code to accommodate for that.
You will also notice that due to the asynchronous nature, it is impossible to return a value from an asynchronous flow back to the synchronous flow where the callback was defined, as the asynchronous callbacks are executed long after the synchronous code has already finished executing.
Instead of returning a value from an asynchronous callback, you will have to make use of the callback pattern, or... Promises.
Promises
Although there are ways to keep the callback hell at bay with vanilla JS, promises are growing in popularity and are currently being standardized in ES6 (see Promise - MDN).
Promises (a.k.a. Futures) provide a more linear, and thus pleasant, reading of the asynchronous code, but explaining their entire functionality is out of the scope of this question. Instead, I'll leave these excellent resources for the interested:
More reading material about JavaScript asynchronicity
- The Art of Node - Callbacks explains asynchronous code and callbacks very well with vanilla JS examples and Node.js code as well.
Note: I've marked this answer as Community Wiki, hence anyone with at least 100 reputations can edit and improve it! Please feel free to improve this answer, or submit a completely new answer if you'd like as well.
I want to turn this question into a canonical topic to answer asynchronicity issues which are unrelated to Ajax (there is How to return the response from an AJAX call? for that), hence this topic needs your help to be as good and helpful as possible!
FabrÃcio's answer is spot on; but I wanted to complement his answer with something less technical, which focusses on an analogy to help explain the concept of asynchronicity.
An Analogy...
Yesterday, the work I was doing required some information from a colleague. I rang him up; here's how the conversation went:
Me: Hi Bob, I need to know how we foo'd the bar'd last week. Jim wants a report on it, and you're the only one who knows the details about it.
Bob: Sure thing, but it'll take me around 30 minutes?
Me: That's great Bob. Give me a ring back when you've got the information!
At this point, I hung up the phone. Since I needed information from Bob to complete my report, I left the report and went for a coffee instead, then I caught up on some email. 40 minutes later (Bob is slow), Bob called back and gave me the information I needed. At this point, I resumed my work with my report, as I had all the information I needed.
Imagine if the conversation had gone like this instead;
Me: Hi Bob, I need to know how we foo'd the bar'd last week. Jim want's a report on it, and you're the only one who knows the details about it.
Bob: Sure thing, but it'll take me around 30 minutes?
Me: That's great Bob. I'll wait.
And I sat there and waited. And waited. And waited. For 40 minutes. Doing nothing but waiting. Eventually, Bob gave me the information, we hung up, and I completed my report. But I'd lost 40 minutes of productivity.
This is asynchronous vs. synchronous behavior
This is exactly what is happening in all the examples in our question. Loading an image, loading a file off disk, and requesting a page via AJAX are all slow operations (in the context of modern computing).
Rather than waiting for these slow operations to complete, JavaScript lets you register a callback function which will be executed when the slow operation has completed. In the meantime, however, JavaScript will continue to execute other code. The fact that JavaScript executes other code whilst waiting for the slow operation to complete makes the behaviorasynchronous. Had JavaScript waited around for the operation to complete before executing any other code, this would have been synchronous behavior.
var outerScopeVar; var img = document.createElement('img');// Here we register the callback function.img.onload = function() { // Code within this function will be executed once the image has loaded. outerScopeVar = this.width;};// But, while the image is loading, JavaScript continues executing, and// processes the following lines of JavaScript.img.src = 'lolcat.png';alert(outerScopeVar);In the code above, we're asking JavaScript to load lolcat.png, which is a sloooow operation. The callback function will be executed once this slow operation has done, but in the meantime, JavaScript will keep processing the next lines of code; i.e. alert(outerScopeVar).
This is why we see the alert showing undefined; since the alert() is processed immediately, rather than after the image has been loaded.
In order to fix our code, all we have to do is move the alert(outerScopeVar) code into the callback function. As a consequence of this, we no longer need the outerScopeVar variable declared as a global variable.
var img = document.createElement('img');img.onload = function() { var localScopeVar = this.width; alert(localScopeVar);};img.src = 'lolcat.png';You'll always see a callback is specified as a function, because that's the only* way in JavaScript to define some code, but not execute it until later.
Therefore, in all of our examples, the function() { /* Do something */ } is the callback; to fix all the examples, all we have to do is move the code which needs the response of the operation into there!
* Technically you can use eval() as well, but eval() is evil for this purpose
How do I keep my caller waiting?
You might currently have some code similar to this;
function getWidthOfImage(src) { var outerScopeVar; var img = document.createElement('img'); img.onload = function() { outerScopeVar = this.width; }; img.src = src; return outerScopeVar;}var width = getWidthOfImage('lolcat.png');alert(width);However, we now know that the return outerScopeVar happens immediately; before the onload callback function has updated the variable. This leads to getWidthOfImage() returning undefined, and undefined being alerted.
To fix this, we need to allow the function calling getWidthOfImage() to register a callback, then move the alert'ing of the width to be within that callback;
function getWidthOfImage(src, cb) { var img = document.createElement('img'); img.onload = function() { cb(this.width); }; img.src = src;}getWidthOfImage('lolcat.png', function (width) { alert(width);});... as before, note that we've been able to remove the global variables (in this case width).
Here's a more concise answer for people that are looking for a quick reference as well as some examples using promises and async/await.
Start with the naive approach (that doesn't work) for a function that calls an asynchronous method (in this case setTimeout) and returns a message:
function getMessage() { var outerScopeVar; setTimeout(function() { outerScopeVar = 'Hello asynchronous world!'; }, 0); return outerScopeVar;}console.log(getMessage());undefined gets logged in this case because getMessage returns before the setTimeout callback is called and updates outerScopeVar.
The two main ways to solve it are using callbacks and promises:
Callbacks
The change here is that getMessage accepts a callback parameter that will be called to deliver the results back to the calling code once available.
function getMessage(callback) { setTimeout(function() { callback('Hello asynchronous world!'); }, 0);}getMessage(function(message) { console.log(message);});Promises provide an alternative which is more flexible than callbacks because they can be naturally combined to coordinate multiple async operations. A Promises/A+ standard implementation is natively provided in node.js (0.12+) and many current browsers, but is also implemented in libraries like Bluebird and Q.
function getMessage() { return new Promise(function(resolve, reject) { setTimeout(function() { resolve('Hello asynchronous world!'); }, 0); });}getMessage().then(function(message) { console.log(message); });jQuery Deferreds
jQuery provides functionality that's similar to promises with its Deferreds.
function getMessage() { var deferred = $.Deferred(); setTimeout(function() { deferred.resolve('Hello asynchronous world!'); }, 0); return deferred.promise();}getMessage().done(function(message) { console.log(message); });async/await
If your JavaScript environment includes support for async and await (like Node.js 7.6+), then you can use promises synchronously within async functions:
function getMessage () { return new Promise(function(resolve, reject) { setTimeout(function() { resolve('Hello asynchronous world!'); }, 0); });}async function main() { let message = await getMessage(); console.log(message);}main();