Webpack from Nothing

Going to Production

We can write JavaScript code in modules and bring in third party libraries, and we can write unit tests. That's pretty darn good for a language that doesn't suport literally any of that in any way. Thank Zod for Webpack!

But, we need to go to production. Code that's not live and solving user problems might as well not exist.

Given that our amazing Markdown previewer totally works statically in a browser on localhost, it's just a matter of dumping files up onto some server, right?

Not exactly. It's true that would work, but for any reasonable application we're going to want a bit more. At the very least we want to:

Why?

Minification Saves Time and Money

Our Markdown app is small now, but it could become the world's foremost way of rendering markdown in a browser. We've got VC's knocking at our door, and we need to be web scale.

Uncompressed assets like JavaScript waste bandwidth. The downstream user doesn't care about whitespace or code comments, and we don't need to pay for bandwidth that's ferrying bytes that won't be used. There's literally no reason to send uncompressed JavaScript to a user other than laziness (or, in our case, the lack of wherewithal to configure our awful toolchain to do it as part of our build pipeline).

If you aren't exactly sure what I mean, consider that this JavaScript works the same way as our existing markdownPreviewer.js:

import{m}from "markdown";
var a=function(d,s,p){
return function(e){
var t=d.getElementById(i).value,
q=d.getElementById(p);
q.innerHTML=m.toHTML(t);
e.preventDefault();};}

This takes up less space than our version, but is terrible code. We can use tools to turn our nice code into crappy but small code that works the same way.

This, plus the use of a content-delivery network will make our application performant and fast without a ton of effort. To use a CDN, however, we need to be mindful about how we manage changes.

Unique Names for Each Version of our bundle.js

If we deployed our code to a content delivery network (which we are likely going to want to do at some point), and we used the name bundle.js it will be pretty hard to update it. Some CDNs don't let you update assets (meaning people would use the original version of bundle.js forever, never getting our updates or bugfixes), and others require you to wait for caches to expire, which can be hours or even days. The standard practice to deal with this is that each new version of your code has a unique name.

So, we want to generate a file like 924giuwergoihq3rofdfg-bundle.js, where 924giuwergoihq3rofdfg changes each time our code changes.

This, along with minification, are basic needs for web development, and since Webpack is a monolithic system, we'd expect it to be able to do this for us. Turns out, it can.

Something Automatic!

Accoring to Webpack's documentation, merely using the -p option when invoking it will perform minification! Wow! Let's see if that's true.

First, let's see how big the bundle is:

> yarn webpack
yarn run v1.1.0
$ webpack --config webpack.config.js --display-error-details
Hash: fc323b14ec51881fcc8e
Version: webpack 3.6.0
Time: 178ms
    Asset     Size  Chunks             Chunk Names
bundle.js  77.8 kB       0  [emitted]  main
   [0] ./js/index.js 334 bytes {0} [built]
   [1] ./js/markdownPreviewer.js 381 bytes {0} [built]
   [5] (webpack)/buildin/global.js 509 bytes {0} [built]
    + 6 hidden modules
Done in 0.66s.
> wc -c dist/bundle.js
77846 dist/bundle.js

Now, let's try -p. To make sure Yarn doesn't think we're passing it -p, we use the UNIX special switch that means “stop parsing command-line switches”, which is --:

> yarn webpack -- -p
yarn run v1.1.0
$ webpack --config webpack.config.js --display-error-details "-p"
Hash: fc323b14ec51881fcc8e
Version: webpack 3.6.0
Time: 841ms
    Asset     Size  Chunks             Chunk Names
bundle.js  25.8 kB       0  [emitted]  main
   [0] ./js/index.js 334 bytes {0} [built]
   [1] ./js/markdownPreviewer.js 381 bytes {0} [built]
   [5] (webpack)/buildin/global.js 509 bytes {0} [built]
    + 6 hidden modules
Done in 1.46s.
warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.
> wc -c dist/bundle.js
25770 dist/bundle.js

Not bad! One third the final size. And without any configuration!

So, what did -p actually do?

It configures the UglifyJsPlugin, which uses UglifyJS to minify our code. It also sets the NODE_ENV environment variable to "production", which allows us to configure things only for production if we want. And we will want.

Now, we need to create a hashed bundle to deal with a CDN. Webpack calls this caching.

Creating a Bundle for Long-Term CDN Caching

Creating a hashed filename is called caching, since the filename allows us to put the bundle in a long-term cache. Instead of having some sort of plugin to do this, it turns out the the value of the string we give to the filename key of output isn't just an ordinary string! It's a second configuration format (yay) that allows us to put a hashed value into the filename.

We can set the value to "[chunkhash]-bundle.js", like so:

const path = require('path');

module.exports = {
  entry: './js/index.js',
  output: {
    path: path.resolve(__dirname, 'dist'),
/* start new code */
    filename: '[chunkhash]-bundle.js'
/* end new code */
  }
};

And, it works!

> yarn webpack
yarn run v1.1.0
$ webpack --config webpack.config.js --display-error-details
Hash: ac350f714ace9ab3e6e6
Version: webpack 3.6.0
Time: 197ms
                         Asset     Size  Chunks             Chunk Names
e7080ab82e8b55254202-bundle.js  77.8 kB       0  [emitted]  main
   [0] ./js/index.js 334 bytes {0} [built]
   [1] ./js/markdownPreviewer.js 381 bytes {0} [built]
   [5] (webpack)/buildin/global.js 509 bytes {0} [built]
    + 6 hidden modules
Done in 0.93s.
> ls dist
bundle.js
e7080ab82e8b55254202-bundle.js
index.html

This creates two new problems, however. First, since the filename will now change whenever our code changes, we can't put a static reference to it into our index.html file. The second problem, though, is that we don't want to do this step in development (note that we got a hashed filename without using the -p option). Webpack's documentation warns us:

Don’t use [chunkhash] in development since this will increase compilation time.

If you recall above, we mentioned that the -p option sets the value of NODE_ENV to "production". You should have every reason to believe that you can access that value inside your Webpack configuration to create conditional, production-only configuration like so:

output: {
  path: path.resolve(__dirname, "dist"),
  filename: process.env.NODE_ENV === "production" ? "[chunkhash]-bundle.js" : "bundle.js"
}

Sadly, you would be wrong.

The -p flag doesn't actually set an environment variable. What it really does is to instruct Webpack to replace code that looks like process.env.NODE_ENV in the code Webpack is bundling with the value "production", but not in webpack.config.js. Sigh.

If you want to read a long an annoying tale of why this is, please check out this Webpack issue. Spoiler: there is no solution at the end.

Although it's nice that -p exists to do something for us for a production build, it's clearly insufficient. We've only come to our second production requirement and it won't work.

If you read the docs more, it's clear where this is going - we need one configuration for dev and one for production. This is not uncommon amongst web application development tools, however it would nice if Wepack just supported this directly, since -p is essentially unusable for any real project (if it doesn't work for a 10-line markdown processor, it doesn't work).

The good news is, after we set this up, configuring stuff for dev vs. prod will be much simpler.

The trick is to figure out how to avoid duplicating common configuration. Webpack sort-of supports this via the webpack-merge module, which can smartly merge two Webpack configurations.

So, we'll need to create a common base configuration, and then one for dev-only and another for prod-only, merging the proper one at compile time.

Separating Production and Development Configuration

The way this works is that our main webpack.config.js will simply require an environment-specific webpack configuration. Those environment-specific ones will pull in a common Webpack configuration, then using the webpack-merge module, overwrite or augment anything needed specific to those environments.

Managing Webpack configs for different environments
Click to embiggen

Our general requirements at this point are:

Since we don't have much configuration now, this shouldn't be a problem.

First, install webpack-merge:

> yarn add webpack-merge
yarn add v1.1.0
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 1 new dependency.
└─ webpack-merge@4.1.0
Done in 2.17s.
warning "karma-jasmine@1.1.0" has unmet peer dependency "jasmine-core@*".

Now, to create our four configuration files. I'm willing to tolerate one configuration file at the top-level, but not four. So, we'll be putting the dev, production, and common in a new directory, called webpack:

> mkdir -p webpack

Our top-level webpack.config.js will reference the environment-specific file in webpack:

module.exports = function(env) {
  if (env === undefined) {
    env = "dev"
  }
  return require(`./webpack/${env}.js`)
}

Where does that env come from? It comes from us. We'll need to pass --env=dev or --env=production to webpack to tell it which env we're building for. This is why we've defaulted it to dev so we don't have to type that nonsense every time. The whole "environment for building" vs "runtime environment" is confusing and arbitrary, but this is how it is.

Next, we'll create webpack/dev.js.

This file will bring in a common Webpack config, and modify it for development. This will look very similar to our original webpack.config.js, but our output path is going to be dev instead of dist, so we don't get confused about what files are what.

Also remember that since this file is in webpack/ and we want to put files in dev/ (not webpack/dev), we have to use ../dev.

const path         = require('path');
const Merge        = require('webpack-merge');
const CommonConfig = require('./common.js');

module.exports = Merge(CommonConfig, {
  output: {
    path: path.join(__dirname, '../dev'),
    filename: 'bundle.js'
  }
});

Now, we'll create our production configuration in webpack/production.js:

const path         = require('path');
const Merge        = require('webpack-merge');
const CommonConfig = require('./common.js');

module.exports = Merge(CommonConfig, {
  output: {
    path: path.join(__dirname, '../production'),
    filename: '[chunkhash]-bundle.js'
  }
});

Both files reference common.js, we create that next:

module.exports = {
  entry: './js/index.js'
};

One confusing thing that is confusing must be pointed out. All of our calls to require use a path relative to the file require is called from. Further, when we call path.join(__dirname, "../production"), the ../ is because this call, too, is executed relative to the file it's executed in. But, our entry point is relative to where Webpack is executed from, which is to say, the root directory.

Let that sink in. As an exercise for later, decide for yourself if any of this is an example of a good design decision. Perhaps it's my fault for insisting I tuck away these silly files in webpack/, but I find all of this unnecessarily arbitrary and confusing.

Anyway, we should be able to run webpack as before:

> yarn webpack
yarn run v1.1.0
$ webpack --config webpack.config.js --display-error-details
Hash: fc323b14ec51881fcc8e
Version: webpack 3.6.0
Time: 174ms
    Asset     Size  Chunks             Chunk Names
bundle.js  77.8 kB       0  [emitted]  main
   [0] ./js/index.js 334 bytes {0} [built]
   [1] ./js/markdownPreviewer.js 381 bytes {0} [built]
   [5] (webpack)/buildin/global.js 509 bytes {0} [built]
    + 6 hidden modules
Done in 0.70s.

This places our bundle in dev, so we'll need to move our HTML file there:

> cp dist/index.html dev/index.html

And now, our app should work as before.

Our app still works

Next, we can build for production:

> yarn webpack -- --env=production -p
yarn run v1.1.0
$ webpack --config webpack.config.js --display-error-details "--env=production" "-p"
Hash: ac350f714ace9ab3e6e6
Version: webpack 3.6.0
Time: 606ms
                         Asset     Size  Chunks             Chunk Names
e7080ab82e8b55254202-bundle.js  25.8 kB       0  [emitted]  main
   [0] ./js/index.js 334 bytes {0} [built]
   [1] ./js/markdownPreviewer.js 381 bytes {0} [built]
   [5] (webpack)/buildin/global.js 509 bytes {0} [built]
    + 6 hidden modules
Done in 1.13s.
warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.
> ls production
e7080ab82e8b55254202-bundle.js

Which brings us to our second problem to solve (remember, this was just the first one!), which is how do we get our index.html to reference the file we just built.

Accessing the Hashed Filename in our HTML

So far, our app doesn't need a server to do anything, but we now need something dynamic. Rather than go through that pain, let's hold what we've got, and get the generated filename into our HTML.

In the previous section, we copied our HTML around, and that's not good. We're building a build system here, and it shouldn't include us typing cp!

What we want is to treat our index.html as a rudimentary template, and include a reference to our bundle in there at build time. The HtmlWebpackPlugin was designed to do this!

First, install it:

> yarn add html-webpack-plugin
yarn add v1.1.0
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 29 new dependencies.
├─ boolbase@1.0.0
├─ camel-case@3.0.0
├─ clean-css@4.1.9
├─ commander@2.11.0
├─ css-select@1.2.0
├─ css-what@2.1.0
├─ dom-converter@0.1.4
├─ dom-serializer@0.1.0
├─ domelementtype@1.3.0
├─ domhandler@2.1.0
├─ domutils@1.1.6
├─ entities@1.1.1
├─ he@1.1.1
├─ html-minifier@3.5.5
├─ html-webpack-plugin@2.30.1
├─ htmlparser2@3.3.0
├─ lower-case@1.1.4
├─ ncname@1.0.0
├─ no-case@2.3.2
├─ nth-check@1.0.1
├─ param-case@2.1.1
├─ pretty-error@2.1.1
├─ relateurl@0.2.7
├─ renderkid@2.0.1
├─ toposort@1.0.6
├─ uglify-js@3.1.3
├─ upper-case@1.1.3
├─ utila@0.3.3
└─ xml-char-classes@1.0.0
Done in 3.12s.
warning "karma-jasmine@1.1.0" has unmet peer dependency "jasmine-core@*".

By default, this plugin will produce an index.html file from scratch that brings in our bundle. Since we have particular markup that we need for our app, we need a way to specify a template. HtmlWebpackPlugin allows us to specify one to use and, if it's just straight-up normal HTML, the plugin will insert the <script> tag in the right place.

Let's place that file in a new directory called html. Note that we've omitted the <script> tag we had before.

<!DOCTYPE html>
<html>
  <head>
    <!-- script tag is omitted - Webpack will provide -->
  </head>
  <body>
    <h1>Markdown Preview-o-tron 7000!</h1>
    <form id="editor">
      <textarea id="source" rows="10" cols="80"></textarea>
      <br>
      <input type="submit" value="Preview!">
    </form>
    <hr>
    <section id="preview">
    </section>
  </body>
</html>

Now, we'll bring in the new plugin and configure it. This is common to both production and development, so we'll put this in webpack/common.js:

/* start new code */
const HtmlPlugin     = require('html-webpack-plugin');

module.exports = {
  plugins: [
    new HtmlPlugin({
      template: "./html/index.html"
    })
  ],
/* end new code */
  entry: './js/index.js'
};

Note again the arbitrary nature of relative paths. The template will be accessed from the directory where we run Webpack, so it's just ./html, and not ../html.

Let's clean out the existing dev directory so we can be sure we did what we think we did:

> rm dev/*.*

Now, run Webpack

> yarn webpack
yarn run v1.1.0
$ webpack --config webpack.config.js --display-error-details
Hash: 64819673d4ecd0cde17d
Version: webpack 3.6.0
Time: 369ms
     Asset       Size  Chunks             Chunk Names
 bundle.js    77.8 kB       0  [emitted]  main
index.html  428 bytes          [emitted]  
   [0] ./js/index.js 334 bytes {0} [built]
   [1] ./js/markdownPreviewer.js 381 bytes {0} [built]
   [5] (webpack)/buildin/global.js 509 bytes {0} [built]
    + 6 hidden modules
Child html-webpack-plugin for "index.html":
     1 asset
       [0] ./node_modules/html-webpack-plugin/lib/loader.js!./html/index.html 745 bytes {0} [built]
       [2] (webpack)/buildin/global.js 509 bytes {0} [built]
       [3] (webpack)/buildin/module.js 517 bytes {0} [built]
        + 1 hidden module
Done in 0.99s.

If you look at dev/index.html, you can see Webpack inserted a <script> tag (at the bottom, and messing up our indentation):

> cat dev/index.html
<!DOCTYPE html>
<html>
  <head>
    <!-- script tag is omitted - Webpack will provide -->
  </head>
  <body>
    <h1>Markdown Preview-o-tron 7000!</h1>
    <form id="editor">
      <textarea id="source" rows="10" cols="80"></textarea>
      <br>
      <input type="submit" value="Preview!">
    </form>
    <hr>
    <section id="preview">
    </section>
  <script type="text/javascript" src="bundle.js"></script></body>
</html>

And, if you run this for production, it works as we'd like!

> yarn webpack -- -p --env=production
yarn run v1.1.0
$ webpack --config webpack.config.js --display-error-details "-p" "--env=production"
Hash: 0e1a0cc03a9cee38eb6c
Version: webpack 3.6.0
Time: 711ms
                         Asset       Size  Chunks             Chunk Names
e7080ab82e8b55254202-bundle.js    25.8 kB       0  [emitted]  main
                    index.html  449 bytes          [emitted]  
   [0] ./js/index.js 334 bytes {0} [built]
   [1] ./js/markdownPreviewer.js 381 bytes {0} [built]
   [5] (webpack)/buildin/global.js 509 bytes {0} [built]
    + 6 hidden modules
Child html-webpack-plugin for "index.html":
     1 asset
       [0] ./node_modules/html-webpack-plugin/lib/loader.js!./html/index.html 745 bytes {0} [built]
       [2] (webpack)/buildin/global.js 509 bytes {0} [built]
       [3] (webpack)/buildin/module.js 517 bytes {0} [built]
        + 1 hidden module
Done in 1.30s.
warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.
> cat production/index.html
<!DOCTYPE html>
<html>
  <head>
    <!-- script tag is omitted - Webpack will provide -->
  </head>
  <body>
    <h1>Markdown Preview-o-tron 7000!</h1>
    <form id="editor">
      <textarea id="source" rows="10" cols="80"></textarea>
      <br>
      <input type="submit" value="Preview!">
    </form>
    <hr>
    <section id="preview">
    </section>
  <script type="text/javascript" src="e7080ab82e8b55254202-bundle.js"></script></body>
</html>

Nice!

As a final step, let's modify package.json to handle the production build.

Scripting the Production Build

Doing this the way we've seen will result in duplicating the command-line arguments common to webpack, so let's set that in config section of package.json.

{
  "config": {
    "webpack_args": " --config webpack.config.js --display-error-details"
  }
}

Now, we can reference this configuration var by prefixing it with $npm_package_config_ and save precious keystrokes:

{
  "scripts": {
    "webpack": "webpack $npm_package_config_webpack_args",
    "prod": "webpack  $npm_package_config_webpack_args -p --env=production",
    "karma": "karma start spec/karma.conf.js --single-run --no-color"
  }
}

And with that:

> yarn prod
yarn run v1.1.0
$ webpack  $npm_package_config_webpack_args -p --env=production
Hash: 0e1a0cc03a9cee38eb6c
Version: webpack 3.6.0
Time: 709ms
                         Asset       Size  Chunks             Chunk Names
e7080ab82e8b55254202-bundle.js    25.8 kB       0  [emitted]  main
                    index.html  449 bytes          [emitted]  
   [0] ./js/index.js 334 bytes {0} [built]
   [1] ./js/markdownPreviewer.js 381 bytes {0} [built]
   [5] (webpack)/buildin/global.js 509 bytes {0} [built]
    + 6 hidden modules
Child html-webpack-plugin for "index.html":
     1 asset
       [0] ./node_modules/html-webpack-plugin/lib/loader.js!./html/index.html 745 bytes {0} [built]
       [2] (webpack)/buildin/global.js 509 bytes {0} [built]
       [3] (webpack)/buildin/module.js 517 bytes {0} [built]
        + 1 hidden module
Done in 1.28s.

(Note that we can't use "production" because it has some special meaning that generates an error that says "no command specified" rather than, you know, telling us we can't use the reserved word "production")

The good thing about putting this in "scripts" , other than scripting away tedious stuff to have to type, is that we now have one command that means "make production happen". As our application evolves, we can add more features behind the prod command.

This was a bit of a slog, but we now have a decent project structure, and can do useful and basic things for development.

There's still room for improvement, but before we look at stuff like debugging and general ergonomics, we should look at how to handle CSS, because our Markdown previewer really could use some pizazz, and the only way to do that is CSS.