Why are only 32 threads running when calling futures in clojure? Why are only 32 threads running when calling futures in clojure? multithreading multithreading

Why are only 32 threads running when calling futures in clojure?


This

(time (doall (map deref (for [i (range 65)] (future (Thread/sleep 1000)))))); "Elapsed time: 3005.244983 msecs"

should actually take 65 seconds because you are asking to deref the futures in sequence on the main thread. These futures have not even started when the deref comes because for is lazy. The reason you get multiples of 32 is the chunking behavior of range.

Compare to non-chunked version

 (time (doall (map deref (for [i (apply list (range 65))] (future (Thread/sleep 1000)))))); "Elapsed time: 64997.260808 msecs"

As pointed out by Rörd in the comments, adding another doall to remove the laziness of the future creation solves the problem.

(time (doall (map deref (doall (for [i (range 65)] (future (Thread/sleep 1000))))))); "Elapsed time: 999.263631 msecs"

Another way to test completion of all the futures is to wait on a promise that is not delivered until the last future completes.

(defn test-futures [n]  (let [a (atom 0)        p (promise)        f (fn [] (swap! a inc) (when (= @a n) (deliver p n)))]    (doseq [i (range n)] (future (do (Thread/sleep 1000) (f))))    (deref p))) 

And now you can see that completion of 64, 65, or 1000 of these futures occurs each in about 1 second.

(time (test-futures 64)); "Elapsed time: 996.262272 msecs"; 64(time (test-futures 65)); "Elapsed time: 996.554436 msecs"; 65(time (test-futures 1000)); "Elapsed time: 1000.247374 msecs"; 1000