<Blog />
Managing environment workflows with gulp.js
October 03, 2017
Over the years, I’ve used a few different combinations of frameworks, libraries and tools, some of which come with environment workflows out-of-the-box and others that do not. With the rising popularity of Node.js, the need to roll one’s own environment workflows is also on the rise, as is the number of resources with which to do so. I recently give a presentation about this topic at one of the software development Meetups I attend; below is an overview of the presentation, the lion’s share of which was a live demo.
Background
I like writing code; I don’t like performing repetitive tasks like manipulating files, migrating folders, running scripts, et cetera. Accordingly, I make optimizing develop, build and deploy workflows across development, test and production environments a priority in every project. Inevitably, this is a tradeoff between magic and boilerplate.
When I was getting started with development, I used Ruby On Rails, which is preconfigured with pretty good, albeit magic-laden, develop and build workflows for development and production environments that are powered by its Asset Pipeline. Additionally, Heroku offers a nice Rails deploy workflow for test and production environments. If you’ve worked with Rails like this before, then you know that it’s great… until it isn’t.
While I still use Rails in some existing projects (including PeakBucket), I’ve come to prefer Node.js when starting new ones. Node leaves configuration almost entirely up to the developer, but with freedom comes responsibility—and often boilerplate. Some tools, like Create React App, provide preconfigured (i.e. boilerplate-free) develop and build workflows for development and production environments with the option to “eject” and customize the configuration at any time. For more complex applications, however, you often need more control; for me, this is where gulp.js comes into play.
With over 3 million downloads a month, gulp needs little introduction. While it’s not the only tool of its kind, nor is it as widely used as webpack (with over 8 million downloads a month), its flexibility, strong ecosystem and interoperability with other tools (including webpack) make it my choice for managing environment workflows in more complex applications.
Example
At work, I maintain a Node project that’s an Express application comprised of multiple single-page React applications. Configured like this, Express handles database connections, view/web service routes and user authentication server-side, while React handles user interactions client-side. Here are the workflows I’ve configured for this application:
Develop
Optimized for developer experience, the develop workflow is tuned to minimize wait time and manual tasks in order to maximize code quality and output in the development environment. On startup, it copies static files, transpiles SASS files and compiles JS(X) files into the public directory. It uses nodemon for a runtime, which automatically restarts the server when watched files change, and uses individual database connections per web service request to avoid acquiring additional connections with each restart (the application uses database pools in other environments). Because webpack bundles the view scripts, it also takes advantage of Hot Module Replacement (including React Hot Loader) in addition to Webpack Bundle Analyzer.
Build
One of 12 items on The Joel Test, making a build in one step, for use in either the test or production environment, is what this workflow is all about. Prior to making a build, it runs a unit test suite, which covers all aspects of the application (including database connections, web service calls, business logic and user authentication/interaction). In addition to copying/transpiling/compiling minified public assets, it also copies all of the other files required for running the application (models, views, controllers, routes, et cetera) into a build directory.
Deploy
One additional piece of complexity with this application is that it’s deployed to a Windows Server hosted on a VPN. My goal in creating this workflow was to replicate the ease of $ git push heroku master, which I’ve enjoyed with other applications. It’s split into two commands: one run locally and another run on the server, which leaves the flexibility to deploy to multiple environments server-side.
Locally, the deploy workflow runs the build workflow, zips the build directory and coppies the zipped build directory to the server.
On the server, the build can be deployed to either the test or production environment:
In the test environment, the deploy workflow on the server unzips the zipped build directory, migrates build files and updates node modules. The test environment runtime is started manually as a process attached to the console (i.e. $ npm start).
In the production environment, the deploy workflow on the server unzips the zipped build directory, performs a database backup, stops the runtime daemon process, migrates build files, updates node modules and starts the runtime daemon process. The production environment runtime is a native Windows service courtesy of the node-windows module.
Feedback
Have questions, comments or suggestions? Reach out to me on Twitter (@colinrbrooks).
— Colin
