Webpack from Nothing

WTF is a Webpack

Webpack is becoming the standard in bundling JavaScript, but what does that even mean and how does it work? Let's find out.

I find the configuration of Webpack hard to understand and derive. I find the general concept of Webpack very difficult to grasp, especially when reading the myriad blogs documenting the way to set it up—they are all different!

The source of these problems is that Webpack is a monolithic system that does many unrelated things in an opaque way, while also being designed around extreme flexibility. Unlike a monolithic framework such as Ruby on Rails, nothing in Webpack “just works”—you have to provide configuration for Webpack's most basic features to work.

This is part of JavaScript's culture—each new problem in your development environment is viewed as a chance to invent a solution from first principles and no particular opinion on this is viewed as canonical or idiomatic. This means we'll be spending a lot of time in documentation and a lot of time making decisions that have nothing to do with our users or the problems we're trying to solve for them.

So, let's figure out what even is a Webpack together by starting from nothing. You'll be amazed at how little JSON you need to throw at this problem, Medium blog posts be damned!

What Problem Does Webpack Solve?

Webpack exists to give us a feature of JavaScript that exists in most other languages by default - modularization.

As programmers, we want to put code in different files for the purposes of organization. Said another way, we don't want all our JavaScript in one file. We may also wish to use third-party JavaScript libraries to help us.

Because JavaScript fails to meet many programmer's needs, there has always been a requirement to have some sort of way to deploy code organized in multiple files to a user's web browser.

The simplest way is to use a <script> tag for each file:

<script src="js/main.js"></script>
<script src="js/address.js"></script>
<script src="js/billing.js"></script>

This is hard to maintain, so another way is to concatenate all the files together into one bundle during the build of your application, and deploy just that bundle to the user's browser.

> find js -name \*.js \
          -exec cat {} \; >> bundle.js

<script src="bundle.js"></script>

This is what the Rails Asset Pipeline effectively does.

This is not good enough for many reasons:

If only JavaScript had a module system like every other language. It sorta does, but not in any useful way, because no browser supports the latest version of JavaScript. And no browser ever will (since “the latest version of JavaScript“ is a moving target).

Webpack implements a module system as well as a way to translate JavaScript code that doesn't work in any web browser to JavaScript code that works in most web browsers. It's like a C compiler.

It allows you to write code like this, which is remedial in most other modern languages:

// inside main.js
import address from "address";
import billing from "billing";

billing.some_function();
address.some_other_function();

Webpack allows you to specify that main.js is your main file, and that main.js might contain instructions for locating other files, which Webpack should do, recursively until all needed files have been located. All those files should then be brought together and translated into a single file, suitable for use in a web browser.

Surprisingly, webpack *.js > bundle.js does not do this. Because the JavaScript ecosystem favors monolithic, do-everything tools, Webpack, in fact, does everything (except what it doesn't—we'll get to that). It's super flexible, which means it's hard to use, hard to understand, and hard to learn.

I'm going to try to correct that by starting from a very simple case, and building things up, one step at a time, until we have a reasonable development and production environment, while only adding configuration when there is a problem that needs solving.

Webpack from First Principles

To install Webpack, you'll need Node, so go install it (you might want to use a package manager). You'll need need a JavaScript library downloader, which used to be only NPM, but now you should use Yarn, so go install it.

> node --version
v12.3.0

> yarn --version
1.21.1

Many tutorials and READMEs tell you to install JavaScript packages willy-nilly. We aren't going to do that. We're going to have a file to keep track of all the stuff our project needs. That file is package.json and Yarn can create one for us. This allows us to recreate our system whenever we want without having to re-execute a bunch of commands.

> yarn -y init
yarn init v1.21.1
success Saved package.json
Done in 0.03s.
warning The yes flag has been set. This will automatically answer yes to all questions, which may have security implications.

This will create an empty package.json for us. Now, let's install Webpack! Note that the CLI is a separate module.

> yarn add -D webpack webpack-cli

yarn add v1.21.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info This module is OPTIONAL, you can safely ignore this error
success Saved lockfile.
success Saved 259 new dependencies.
info Direct dependencies
├─ webpack-cli@3.3.10
└─ webpack@4.41.5
info All dependencies
├─ @webassemblyjs/floating-point-hex-parser@1.8.5
├─ @webassemblyjs/helper-code-frame@1.8.5
├─ @webassemblyjs/helper-fsm@1.8.5
├─ @webassemblyjs/helper-wasm-section@1.8.5
├─ @webassemblyjs/wasm-edit@1.8.5
├─ @webassemblyjs/wasm-opt@1.8.5
├─ @xtuc/ieee754@1.2.0
├─ acorn@6.4.0
├─ ajv-errors@1.0.1
├─ ajv-keywords@3.4.1
├─ ajv@6.11.0
├─ ansi-regex@4.1.0
├─ ansi-styles@3.2.1
├─ anymatch@2.0.0
├─ aproba@1.2.0
├─ arr-flatten@1.1.0
├─ asn1.js@4.10.1
├─ assert@1.5.0
├─ assign-symbols@1.0.0
├─ async-each@1.0.3
├─ atob@2.1.2
├─ balanced-match@1.0.0
├─ base@0.11.2
├─ base64-js@1.3.1
├─ big.js@5.2.2
├─ binary-extensions@1.13.1
├─ bluebird@3.7.2
├─ brace-expansion@1.1.11
├─ braces@2.3.2
├─ browserify-aes@1.2.0
├─ browserify-cipher@1.0.1
├─ browserify-des@1.0.2
├─ browserify-sign@4.0.4
├─ browserify-zlib@0.2.0
├─ buffer-xor@1.0.3
├─ buffer@4.9.2
├─ builtin-status-codes@3.0.0
├─ cacache@12.0.3
├─ cache-base@1.0.1
├─ camelcase@5.3.1
├─ chalk@2.4.2
├─ chokidar@2.1.8
├─ chownr@1.1.3
├─ chrome-trace-event@1.0.2
├─ cipher-base@1.0.4
├─ class-utils@0.3.6
├─ cliui@5.0.0
├─ collection-visit@1.0.0
├─ color-convert@1.9.3
├─ color-name@1.1.3
├─ commander@2.20.3
├─ commondir@1.0.1
├─ concat-map@0.0.1
├─ concat-stream@1.6.2
├─ console-browserify@1.2.0
├─ constants-browserify@1.0.0
├─ copy-concurrently@1.0.5
├─ copy-descriptor@0.1.1
├─ core-util-is@1.0.2
├─ create-ecdh@4.0.3
├─ create-hmac@1.1.7
├─ cross-spawn@6.0.5
├─ crypto-browserify@3.12.0
├─ cyclist@1.0.1
├─ debug@2.6.9
├─ decamelize@1.2.0
├─ decode-uri-component@0.2.0
├─ des.js@1.0.1
├─ detect-file@1.0.0
├─ diffie-hellman@5.0.3
├─ domain-browser@1.2.0
├─ duplexify@3.7.1
├─ emoji-regex@7.0.3
├─ emojis-list@2.1.0
├─ enhanced-resolve@4.1.0
├─ errno@0.1.7
├─ escape-string-regexp@1.0.5
├─ eslint-scope@4.0.3
├─ esrecurse@4.2.1
├─ estraverse@4.3.0
├─ events@3.1.0
├─ evp_bytestokey@1.0.3
├─ execa@1.0.0
├─ expand-brackets@2.1.4
├─ expand-tilde@2.0.2
├─ extglob@2.0.4
├─ fast-deep-equal@3.1.1
├─ fast-json-stable-stringify@2.1.0
├─ fill-range@4.0.0
├─ find-cache-dir@2.1.0
├─ findup-sync@3.0.0
├─ flush-write-stream@1.1.1
├─ for-in@1.0.2
├─ from2@2.3.0
├─ fs.realpath@1.0.0
├─ get-caller-file@2.0.5
├─ get-stream@4.1.0
├─ get-value@2.0.6
├─ glob-parent@3.1.0
├─ glob@7.1.6
├─ global-modules@2.0.0
├─ global-prefix@3.0.0
├─ has-value@1.0.0
├─ hash.js@1.1.7
├─ hmac-drbg@1.0.1
├─ https-browserify@1.0.0
├─ ieee754@1.1.13
├─ import-local@2.0.0
├─ infer-owner@1.0.4
├─ inflight@1.0.6
├─ ini@1.3.5
├─ interpret@1.2.0
├─ invert-kv@2.0.0
├─ is-accessor-descriptor@1.0.0
├─ is-binary-path@1.0.1
├─ is-data-descriptor@1.0.0
├─ is-descriptor@1.0.2
├─ is-extglob@2.1.1
├─ is-fullwidth-code-point@2.0.0
├─ is-plain-object@2.0.4
├─ is-stream@1.1.0
├─ is-wsl@1.1.0
├─ isarray@1.0.0
├─ isexe@2.0.0
├─ json-parse-better-errors@1.0.2
├─ json-schema-traverse@0.4.1
├─ json5@1.0.1
├─ kind-of@3.2.2
├─ lcid@2.0.0
├─ loader-runner@2.4.0
├─ loader-utils@1.2.3
├─ locate-path@3.0.0
├─ lru-cache@5.1.1
├─ make-dir@2.1.0
├─ mamacro@0.0.3
├─ map-age-cleaner@0.1.3
├─ map-visit@1.0.0
├─ mem@4.3.0
├─ memory-fs@0.4.1
├─ micromatch@3.1.10
├─ miller-rabin@4.0.1
├─ mimic-fn@2.1.0
├─ minimalistic-crypto-utils@1.0.1
├─ minimatch@3.0.4
├─ minimist@1.2.0
├─ mississippi@3.0.0
├─ mixin-deep@1.3.2
├─ mkdirp@0.5.1
├─ move-concurrently@1.0.1
├─ ms@2.0.0
├─ nanomatch@1.2.13
├─ neo-async@2.6.1
├─ nice-try@1.0.5
├─ node-libs-browser@2.2.1
├─ normalize-path@3.0.0
├─ npm-run-path@2.0.2
├─ object-assign@4.1.1
├─ object-copy@0.1.0
├─ os-browserify@0.3.0
├─ os-locale@3.1.0
├─ p-defer@1.0.0
├─ p-finally@1.0.0
├─ p-is-promise@2.1.0
├─ p-limit@2.2.2
├─ p-locate@3.0.0
├─ p-try@2.2.0
├─ pako@1.0.10
├─ parallel-transform@1.2.0
├─ parse-passwd@1.0.0
├─ pascalcase@0.1.1
├─ path-browserify@0.0.1
├─ path-dirname@1.0.2
├─ path-exists@3.0.0
├─ path-key@2.0.1
├─ pify@4.0.1
├─ posix-character-classes@0.1.1
├─ process-nextick-args@2.0.1
├─ process@0.11.10
├─ promise-inflight@1.0.1
├─ prr@1.0.1
├─ public-encrypt@4.0.3
├─ pumpify@1.5.1
├─ punycode@1.4.1
├─ querystring-es3@0.2.1
├─ querystring@0.2.0
├─ randomfill@1.0.4
├─ readable-stream@2.3.7
├─ readdirp@2.2.1
├─ remove-trailing-separator@1.1.0
├─ repeat-element@1.1.3
├─ require-directory@2.1.1
├─ require-main-filename@2.0.0
├─ resolve-cwd@2.0.0
├─ resolve-dir@1.0.1
├─ resolve-from@3.0.0
├─ resolve-url@0.2.1
├─ ret@0.1.15
├─ rimraf@2.7.1
├─ run-queue@1.0.3
├─ semver@5.7.1
├─ serialize-javascript@2.1.2
├─ set-blocking@2.0.0
├─ set-value@2.0.1
├─ setimmediate@1.0.5
├─ shebang-command@1.2.0
├─ shebang-regex@1.0.0
├─ signal-exit@3.0.2
├─ snapdragon-node@2.1.1
├─ snapdragon-util@3.0.1
├─ source-list-map@2.0.1
├─ source-map-resolve@0.5.3
├─ source-map-support@0.5.16
├─ source-map-url@0.4.0
├─ source-map@0.6.1
├─ split-string@3.1.0
├─ ssri@6.0.1
├─ static-extend@0.1.2
├─ stream-browserify@2.0.2
├─ stream-each@1.2.3
├─ stream-http@2.8.3
├─ string_decoder@1.3.0
├─ strip-ansi@5.2.0
├─ strip-eof@1.0.0
├─ supports-color@6.1.0
├─ terser-webpack-plugin@1.4.3
├─ terser@4.6.3
├─ through2@2.0.5
├─ timers-browserify@2.0.11
├─ to-arraybuffer@1.0.1
├─ to-object-path@0.3.0
├─ to-regex-range@2.1.1
├─ tslib@1.10.0
├─ tty-browserify@0.0.0
├─ typedarray@0.0.6
├─ union-value@1.0.1
├─ unique-filename@1.1.1
├─ unique-slug@2.0.2
├─ unset-value@1.0.0
├─ upath@1.2.0
├─ uri-js@4.2.2
├─ urix@0.1.0
├─ url@0.11.0
├─ use@3.1.1
├─ util-deprecate@1.0.2
├─ util@0.11.1
├─ v8-compile-cache@2.0.3
├─ vm-browserify@1.1.2
├─ watchpack@1.6.0
├─ webpack-cli@3.3.10
├─ webpack-sources@1.4.3
├─ webpack@4.41.5
├─ which-module@2.0.0
├─ which@1.3.1
├─ worker-farm@1.7.0
├─ wrap-ansi@5.1.0
├─ xtend@4.0.2
├─ yallist@3.1.1
├─ yargs-parser@13.1.1
└─ yargs@13.2.4
Done in 6.49s.
warning Error running install script for optional dependency: "/Users/davec/Projects/Personal/wtf-is-a-webpack/work/work/node_modules/fsevents: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments: 
Directory: /Users/davec/Projects/Personal/wtf-is-a-webpack/work/work/node_modules/fsevents
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@3.8.0
gyp info using node@12.3.0 | darwin | x64
gyp info spawn /usr/bin/python
gyp info spawn args [
gyp info spawn args   '/Users/davec/.asdf/installs/nodejs/12.3.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/davec/Projects/Personal/wtf-is-a-webpack/work/work/node_modules/fsevents/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/davec/.asdf/installs/nodejs/12.3.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/davec/.node-gyp/12.3.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/davec/.node-gyp/12.3.0',
gyp info spawn args   '-Dnode_gyp_dir=/Users/davec/.asdf/installs/nodejs/12.3.0/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/davec/.node-gyp/12.3.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/davec/Projects/Personal/wtf-is-a-webpack/work/work/node_modules/fsevents',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  SOLINK_MODULE(target) Release/.node
  CXX(target) Release/obj.target/fse/fsevents.o
In file included from ../fsevents.cc:6:
In file included from ../../nan/nan.h:54:
/Users/davec/.node-gyp/12.3.0/include/node/node.h:107:12: fatal error: 'util-inl.h' file not found
#  include <util-inl.h>
           ^~~~~~~~~~~~
1 error generated.
make: *** [Release/obj.target/fse/fsevents.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/davec/.asdf/installs/nodejs/12.3.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:200:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Darwin 18.7.0
gyp ERR! command \"/Users/davec/.asdf/installs/nodejs/12.3.0/bin/node\" \"/Users/davec/.asdf/installs/nodejs/12.3.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"
gyp ERR! cwd /Users/davec/Projects/Personal/wtf-is-a-webpack/work/work/node_modules/fsevents
gyp ERR! node -v v12.3.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok"

Holy moly is that a lot of stuff! I find it anxiety-inducing to watch the sheer volume of code being downloaded, and often wonder what problem that solves, but nevertheless, it should work. There will be ASCII art. There will be warnings that you have to ignore. But it should work, and you can verify it like so:

> $(yarn bin)/webpack --version
4.41.5

You've now taken your first step into a larger world, which is rife with version incompatibilities, masked or incorrect error messages, and inconsistent behavior, all so you can try to make your life easier while using one of the worst programming languages ever designed! Webpack is one of the least bad things you'll deal with.

With this out of the way, let's see Webpack actually do something.

A Very Simple Project

As we mentioned above, the purpose of Webpack is to take lots of JavaScript modules and produce a bundle that works in a browser, thus allowing you to write organized code.

In Webpack's parlance, the entry is the file it will read to find all other files. The output is the bundle that Webpack is creating and will be used in our browser.

Let's make a directory for our code called js:

> mkdir -p js

Now, we'll make our entry point in js/index.js like so:

console.log("Hello from index.js!");
import address from './address';
import billing from './billing';

address.announce();
billing.announce();

This is referencing two modules, address and billing. We'll create both of those in js/.

This is js/address.js:

console.log("Hello from address.js");
export default {
  announce: function() {
    console.log("Announcing address.js");
  }
}

And this is js/billing.js:

console.log("Hello from billing.js");
export default {
  announce: function() {
    console.log("Announcing billing.js");
  }
}

What we want is to produce a singe file called bundle.js that uses all this code.

We can do this without configuration per se, and just use CLI options:

> $(yarn bin)/webpack  --mode=none --entry ./js/index.js  --output-filename=bundle.js

Hash: 9559073a3b51909d0f1a
Version: webpack 4.41.5
Time: 63ms
Built at: 01/21/2020 5:45:18 PM
    Asset      Size  Chunks             Chunk Names
bundle.js  4.63 KiB       0  [emitted]  null
Entrypoint null = bundle.js
[0] ./js/index.js 144 bytes {0} [built]
[1] ./js/address.js 128 bytes {0} [built]
[2] ./js/billing.js 128 bytes {0} [built]

Now, let's load this in a browser. Create index.html like so:

<!DOCTYPE html>
<html>
  <head>
    <script src="dist/bundle.js"></script>
  </head>
  <h1>Open the Web Inspector</h1>
</html>

Note that we're referencing dist/bundle.js instead of bundle.js. This is because, despite the fact that webpack's CLI clearly documents that bundle file will be written to the current directory, it still writes it to do dist. It's OK, we'll be moving on from this soon enough, so for now, it's Just How it Is™.

Open index.html in a browser, then open the JavaScript console. You should see all our messages:

Hello from address.js
Hello from billing.js
Hello from index.js!
Announcing address.js
Announcing billing.js

Ok then! That was neat!

We don't want to be building our JavaScript bundle from an ever-increasingly-complex command-line invocation. We also don't want the generated code being dumped in our root directory either, so let's set-up a tiny project structure to keep things organized.

Make webpack.config.js look like so:

const path = require('path');

module.exports = {
  entry: './js/index.js',
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'bundle.js'
  },
  mode: "none"
};

This will do what we did before with Webpack, except it will place bundle.js inside dist/ instead of the current directory. This path must be absolute for reasons that are uninteresting and arbitrary. To satisfy this unnecessary requirement, we use the Node module path, which has a function resolve which will take our intention to use dist/ and specify the full path for Webpack.

Also, yes, this file is in the root directory of our project, which is unfortunate, but it'll make things easier for us for the time being, so just get used to it and be glad it's not called WebpackFile.

With this configuration in place, we can run it like so:

> $(yarn bin)/webpack

Hash: 4dcf30d113d1c2237df1
Version: webpack 4.41.5
Time: 103ms
Built at: 01/21/2020 5:45:25 PM
    Asset      Size  Chunks             Chunk Names
bundle.js  4.63 KiB       0  [emitted]  main
Entrypoint main = bundle.js
[0] ./js/index.js 144 bytes {0} [built]
[1] ./js/address.js 128 bytes {0} [built]
[2] ./js/billing.js 128 bytes {0} [built]

> ls dist
bundle.js

Woot!

If we move our index.html into the newly-created dist:

> mv index.html dist

We can open that in a browser, open the JavaScript console, and see the same messages as before.

The configuration file saves us some keystrokes, but even typing our $(yarn bin)/webpack etc etc is a bit cumbersome. We'll use a handy feature of package.json to create an alias for running webpack so we can just type yarn webpack.

We'll add a "scripts" key to package.json:

{
  "scripts": {
    "webpack": "webpack --config webpack.config.js --display-error-details"
  }
}

Yes, you can omit $(yarn bin) inside package.json like this—it knows to find webpack in the right place.

Your entire package.json looks like so:

> cat package.json

{
  "name": "work",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "devDependencies": {
    "webpack": "^4.41.5",
    "webpack-cli": "^3.3.10"
  },
  "scripts": {
    "webpack": "webpack --config webpack.config.js --display-error-details"
  }
}

And now:

> yarn webpack

yarn run v1.21.1
$ webpack --config webpack.config.js --display-error-details
Hash: 4dcf30d113d1c2237df1
Version: webpack 4.41.5
Time: 62ms
Built at: 01/21/2020 5:45:27 PM
    Asset      Size  Chunks             Chunk Names
bundle.js  4.63 KiB       0  [emitted]  main
Entrypoint main = bundle.js
[0] ./js/index.js 144 bytes {0} [built]
[1] ./js/address.js 128 bytes {0} [built]
[2] ./js/billing.js 128 bytes {0} [built]
Done in 0.51s.

This is slightly better than a shell alias, because you can check it into source control.

Now What?

We're not even getting started—this was just an overview of what Webpack even is and why it exists. I found it personally helpful to go through this so I could see a very minimal use-case and learn the basic concepts, namely entry and output.

And just think how much we can accomplish with just eight lines of configuration. We can write JavaScript in a controlled way, putting code in files, and being organized.

So, what's next?

There's a lot we aren't doing, that we need to do on any real project, such as:

These are all possible with Webpack and also happen to be intended use-cases, however it's extremely hard to figure out how to do these things without someone just showing you the magical configuration needed to make them happen.

I don't like that. Boilerplate is just as tedious as “magic” and is just as intention-unrevealing. So, we're going to start with a problem to solve, and figure out together how to solve that with Webpack. There wil be digressions, yak shaving, and a few bumps in the road, but we'll get there.

The two next obvious things we might want to do are using third party libraries and unit testing. Since unit testing requires third party libraries, let's tackle third party libraries first by creating a more realistic application and bringing in some open source code to help write it.