Luke Evers

Web Application Dependency Management

Posted on 10 October 2014

I've seen web applications structured in a lot of different ways. Everyone has their own way of managing dependencies, and everyone has their own application structure. Here's mine:

app  
  ├── assets
  │   ├── css
  │   │   └── .gitignore
  │   ├── js
  │   │   └── main.js
  │   └── less
  │       └── main.less
  ├── bower.json
  ├── gulpfile.js
  ├── package.json
  └── public
      └── assets
          ├── css
          │   └── .gitignore
          └── js
              └── .gitignore

The folder public is always the only folder that is publicly facing. Everything else is private. Before I get into detail about each element in my basic application, let's talk about the tools we're going to be using.

Gulp

Gulp calls itself "the streaming build system," and is a tool that I use for a few different purposes:

  1. Converting LESS to CSS
  2. Concatenating CSS / JS files into single files
  3. Minifying CSS / JS
  4. Moving minified CSS / JS from private folders to public location

There are other tools out there that does what Gulp does (like Grunt), but I prefer Gulp. In order to use Gulp we need to install it! Assuming you already have NPM and NodeJs installed, installing Gulp is easy:

npm install -g gulp  

Once we have Gulp installed we can create a package.json file so we can load in the Gulp plugins we're going to be using. Here's a minimal package.json file to get you started, but to view a full list of options, checkout this interactive example.

{
  "name": "example",
  "version": "1.0.0",
  "repository": {
    "type": "git",
    "url": "https://github.com/lukevers/example"
  },
  "dependencies": {
    "gulp": "^3.8.7",
    "gulp-concat": "^2.3.4",
    "gulp-less": "^1.3.5",
    "gulp-minify-css": "^0.3.7",
    "gulp-plumber": "^0.6.5",
    "gulp-uglify": "^0.3.1",
    "run-sequence": "^0.3.6"
  }
}

In this file we're telling NPM to download the following modules at specific versions:

  1. gulp at version 3.8.7
  2. gulp-concat at version 2.3.4
  3. gulp-less at version 1.3.5
  4. gulp-minify-css at version 0.3.7
  5. gulp-plumber at version 0.6.5
  6. gulp-uglify at version 0.3.1
  7. run-sequence at 0.3.6

We have to also install gulp locally as well installing it globally. There's a good possibility that by the time you're reading this any number of the versions for the modules that I'm specifying currently could not be the most up-to-date, so if you're going to use these dependencies, get the most recent versions!

Now that we have a package.json file setup, let's install our dependencies:

npm update  

By default, NPM installs modules into a node_modules folder.

The next thing to do before we can run Gulp is to create a gulpfile.js, but we'll do that later. Let's move on to our next tool for now.

Bower

Bower is a good package manager for libraries. If you're writing a very simple web application that does not use any external libraries, then this is not something you're going to need. Most web applications use at least one external library though, and keeping external libraries in repositories is generally1 a bad thing.

Similar to our package.json file for NPM, Bower also has a file you need to create to get dependencies. I'm going to add some libraries to this file that I'll use with our gulpfile later. This is another minimal file, and I recommend checking out the documentation when creating an actual bower.json file.

{
  "name": "example",
  "version": "1.0.0",
  "homepage": "https://github.com/lukevers/example",
  "dependencies": {
    "jquery": "^2.1.1",
    "bootstrap": "^3.2.0",
    "animate.css": "^3.2.0"
  }
}

Since we have a bower.json file setup, just let's install our dependencies. It's fairly similar to NPM:

bower update  

By default, Bower dependencies install into a bower_components folder. You can change that by editing bower.json.

The Gulpfile

Now that we've downloaded everything with Bower and NPM, we're ready to write our gulpfile. The first thing to do (besides from creating gulpfile.js) is to include all of the Gulp plugins we downloaded!

var gulp        = require('gulp');  
var less        = require('gulp-less');  
var concat      = require('gulp-concat');  
var watch       = require('gulp-watch');  
var livereload  = require('gulp-livereload');  
var minifyCSS   = require('gulp-minify-css');  
var uglify      = require('gulp-uglify');  
var plumber     = require('gulp-plumber');  
var runSequence = require('run-sequence');  

If you've used NodeJS before this looks normal, but others who have only ever written client-side JavaScript might not understand the keyword require. The next thing to do is to declare some variables. This makes the rest of the file stay much cleaner.

// Less
var _less  = 'assets/less/';  
var less_  = 'assets/css/';

// CSS
var _css  = 'assets/css/';  
var css_  = 'public/assets/css/';

// JavaScript
var _js   = 'assets/js/';  
var js_   = 'public/assets/js/';

// Bower
var bower = 'bower_components/';  

Assuming you're going by the structure I showed at the top of the article, these paths shouldn't have to change at all. If they are different though, here is the place to change them!

Next we've got a few more variables to continue the process of a cleaner file. I generally @import other LESS files instead of adding them here, but you can also add them here! It's also important to note that with all of these, the order of the files when concatenating and minifying into one file is set here. If you want styles/scripts in a certain order, here is where to do that.

var less_files = [  
    // -- Add LESS files from assets -- //

    'main.less',

    // -- End LESS files from assets -- //
].map(function(str) { return _less + str; });

It's important to explain what is going on here for anyone that is unsure. We're declaring a variable called less_files which is an array of LESS files that will later get compiled into a CSS file. We don't have to specify the full path in the string like "assets/less/main.less" because we're using an anonymous function to add the string "assets/less/" to every element of the array2.

Along with the less_files variable, we have two similar variables for Bower:

// Bower CSS files
var css_bower = [  
    // -- Add CSS files from bower -- //

    'bootstrap/dist/bootstrap.min.css',
    'animate.css/animate.min.css',

    // -- End CSS files from bower -- //
].map(function(str) { return bower + str });

// Bower JS files
var js_bower = [  
    // -- Add JS files from bower -- //

    'jquery/dist/jquery.min.js',
    'bootstrap/dist/js/bootstrap.min.js',

    // -- End JS files from bower -- //
].map(function(str) { return bower + str });

Lastly for our arrays, we have one for all of our CSS files, and one for all of our JavaScript files. Like the previous arrays, we're using an anonymous function to add the paths to the beginning of each element, but we're also doing one other thing with these two arrays. With these two, after we add the paths to each element, we're concatenating the list of bower files with the list of files from our assets folder so we have a full list of all CSS / JavaScript files.

// CSS files
var css_files = css_bower.concat([  
    // -- Add CSS files from assets -- //

    'main.css',

    // -- End CSS files from assets -- //
].map(function(str) { return _css + str }));

// JS files
var js_files = js_bower.concat([  
    // -- Add JS files from assets -- //

    'main.js',

    // -- End JS files from assets -- //
].map(function(str) { return _js + str }));

Now we can start writing tasks. Each task in Gulp handles a different responsibility. We have one task for LESS, CSS and one task for JavaScript. You can create more task for more functionality, but we're just going to create these tasks for now (and a default task).

To run a Gulp task, from the terminal we type the following:

# Default task
gulp

# Particular task
gulp [task]  

Let's start with the default task. The default task runs the other tasks.

gulp.task('default', function() {  
    runSequence('less', ['js', 'css']);
});

And that's all it does! You can have the default task do a lot more than just this, but the default task should not be too complex. I'm using runSequence because the LESS files need to be compiled before they can be concatenated with the CSS files. It runs less first, and then js and css asynchronously.

Now let's write that JavaScript task.

gulp.task('js', function() {  
    return gulp.src(js_files)
        .pipe(concat('main.js'))
        .pipe(plumber({
            errorHandler: function(err) { console.log(err) }
        }))
        .pipe(uglify())
        .pipe(gulp.dest(js_));
});

When we run our JavaScript task, the following happens:

  1. Gulp takes the array of javascript files and concatenates them to a file called main.js
  2. We setup plumber to catch all errors and print them to our console
  3. Gulp minifies our JavaScript file
  4. Gulp places our minified JavaScript file in the location we specify

Our LESS and CSS tasks are very similar to our JavaScript task. When we minify our CSS we're currently keeping absolutely no comments. There are configurable options for most gulp plugins, and you can read about each one individually on their respected pages.

// LESS
gulp.task('less', function() {  
    return gulp.src(less_files)
        .pipe(plumber({
            errorHandler: function(err) { console.log(err) }
        }))
        .pipe(less())
        .pipe(concat('main.css'))
        .pipe(gulp.dest(less_));
});

// CSS
gulp.task('css', function() {  
    return gulp.src(css_files)
        .pipe(concat('main.css'))
        .pipe(minifyCSS({keepSpecialComments: 0}))
        .pipe(gulp.dest(css_));
});

By now you should have a complete gulpfile.js and a basic structure for a web application. There are a lot of Gulp plugins out there, and these are just some of the more basic ones. I always be sure to include gulp-watch and gulp-livereload for developing and testing at a faster pace, but they aren't necessarily needed when it comes to a basic management system.

Footnotes

  1. Keeping an external library in a repository, in my opinion, is okay to do so if you absolutely can't find a reliable place to get the library from with a package manager.
  2. For more information about anonymous functions, see this article (http://transitiontech.ca/lambda) on lambdas in JavaScript.
comments powered by Disqus