By your same logic, there’s no point in not minifying the files. It is really fast (if we minify before merge); it doesn’t ruin the debugging process (we’re all using source maps anyway, right?); and if indeed the minified code is the same as the original code, why output unminified code at all?
Here’s another way of looking at it. Webpack/Gulp/Browserify developers and users write extra code to not minify in dev mode. My argument: that extra code creates zero value, in theory.
(I know that some steps, like tree-shaking, should come at the end of the pipeline and might take time. But minification — renaming variables, tweaking control structures and whatnot — should happen at the module level because that’s faster and simpler.)