Good ideas.

As I wrote in the article, reading all the files in parallel — in this specific benchmark — leads to EMFILE: too many open files. Promisify in parallel would do the same thing.

I added promisify in serial to see whether it’s any faster. It’s far slower than async. That’s not too surprising: it’s probably implemented in terms of async anyway.

We could try batching reads so we only open 1,000 files at a time. But why? Node’s async strategies add 500ms or 1s of overhead to ~100ms of useful work. If batching somehow makes the useful work take 10x less time, the entire task will still cost 500ms or 1s in overhead.

So the original point stands: async methods are very expensive — often more expensive than synchronous I/O. Developers should always consider whether the overhead is worthwhile.

Journalist, ex software engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store