I am a fan of simple solutions. In this post, I explain how we used Jekyll and GitHub Pages to solve our need for a new developer docs system.

Like most startups, our first developer docs were just another controller in a monolithic application. That method served us well for a few years but had obvious problems:

  • Trivial changes to the docs required a full production deployment of the monolith.
  • Only developers with access to the monolith codebase could make changes to docs.
  • Simple HTML was served via a server-side scripting language, instead of static content.

We wanted to solve those problems. After a quick brainstorming session, we decided on using GitHub Pages as a way to cheaply and quickly deploy documentation changes. Using Jekyll with GitHub Pages for developer docs has a bunch of advantages for us.

Developer docs are decoupled

Not having developer docs bundled with our monolithic application makes development and deployment a breeze. Deployment is much safer because you only access developer docs, instead of sensitive code.

Anyone within WePay can make changes

I can’t stress enough how impactful this has been. One of our engineers, Cameron Alley, put together a training guide to contributing using GitHub Desktop. Now we get contributions and suggestions as pull requests from our support and sales engineering teams!

Developer docs are static instead of dynamic pages

Static HTML pages are less prone to downtime. If they do go down, it is easy to host them anywhere with a simple copy-paste job. The downside is a lack of dynamic forms or live demos. However, that’s getting easier to do with Javascript.

CDN, scaling, and deployment managed by Github

GitHub Pages is served by a global content delivery network. Many of our customers have global development teams and they really appreciate the faster load times. We don’t need any special tooling for deploying the docs since that’s handled by GitHub as well.

Free

We already pay for public GitHub.com and GitHub Enterprise. There isn’t an extra cost for developer docs.

Repeatable

Whenever we need a static website, we can leverage the same setup. This blog also uses Jekyll and GitHub Pages.

Architecture

Staging

A key requirement was to have a staging site for internal validation before deploying to production. GitHub Pages makes it possible with two repos with different CNAME files. However, having two repos is not ideal – it requires us to either not commit the CNAME files or build some magic to overwrite their content for a specific environment. This is not the way we build services at WePay. We build them to be non-magical and boring. We prefer the code and setup for staging be exactly like production.

Let’s assume the docs are hosted in a repository docs-repo under the wepay org on github.com. The path to the repo would be, github.com/wepay/docs-repo. The published site would be accessible at wepay.github.io/docs-repo.

We use GitHub Enterprise (GHE) at WePay. It is the on-premise version of Github.com. As a fintech company, we keep our source code and data on our production machines, instead of with GitHub. Most regulated software companies use such on-premise features.

Having GitHub Enterprise greatly simplifies building the staging area for the docs. The URLs for GHE pages don’t follow the same pattern as the public ones. Let’s say the internal website for GHE is github.wepay-internal.com. If we need the path to be the same /docs-repo, then we need an internal organization called docs-repo and GHE pages need to be set up as organization pages. The GHE repo needs to be called docs-repo.github.wepay-internal.com. The published site would be accessible at pages.github.wepay-internal.com/docs-repo.

Routing and SSL

We wanted to route the new developer docs to the subdomain developer.wepay.com. We set that up using an Nginx reverse proxy for https://developer.wepay.com to https://wepay.github.io/docs-repo and a similar setup for staging site. Connections to WePay and from WePay to GitHub happen only over HTTPS.

# production - developer.wepay.com
location / {
      proxy_pass https://wepay.github.io/docs-repo/;
    }

# staging - tst-developer.wepay-internal.com
location / {
      proxy_pass https://pages.github.wepay-internal.com/docs-repo/;
    }

This also has a bonus advantage - we can instantly route to a static site hosted anywhere if GitHub pages are down.

Deployment

Usually, GitHub Pages is attached to projects. This requires a separate gh-pages branch. However, our project is the docs repo. To simplify, we can just have gh-pages as the default branch. Deploying a change is now easy and can be done at any time. Steps involve:

  1. Review and merge pull request to internal repo docs-repo.github.wepay-internal.com.
  2. Verify changes on tst-developer.wepay-internal.com.
  3. git push to public repo github.com/wepay/docs-repo.
  4. Verify changes on production.

GitHub Pages Deployment Diagram

Fig. 1: GitHub Pages Deployment Process

We have complex testing and automation rituals, but the steps above are enough to complete a basic deployment.

Gotchas

Over the last few months, we realized some quirks about this setup.

GitHub vs Enterprise Change Cadence

GitHub Pages change every 5-10 days. On the other hand, GitHub Enterprise is updated every few months. Initially, we tried to protect against this by locking the versions of gems we were using. This constantly resulted in problems until we realized that GitHub Pages can completely ignore the gem versions or gems in the Gemfile.

Now, we simply use the latest github-pages gem and validate locally, instead of relying on GHE pages.

gem 'github-pages', group: :jekyll_plugins

Repository Naming

The difference between GHE Pages and GitHub Pages URLs and repository names is ugly; however, we haven’t found a better solution yet.

Markdown Tables

I feel like I am echoing the pain of the entire universe when I complain about Markdown tables. The Markdown to HTML table rendering in Jekyll is very basic. It is good for showing simple, tabular data but starts getting hairy when you want multiple paragraphs, HTML tags within cells, and specific CSS for individual columns or cells. These caused us so many rendering issues that we simply reverted to using HTML tables for API request and response parameters.

GitHub Downtime

GitHub Pages can and does occasionally go down. This applies to both the content and the build systems. However, their uptime is pretty impressive and meets our expectations. Because GitHub Pages is not part of our network, we need to monitor it externally. We haven’t suffered an extended outage yet, but if we did, we could easily spin up a google cloud storage bucket with a static site and point developer.wepay.com to it. This wouldn’t give all the benefits of deployments and a CDN, though.

Future

Auto-generation

Adding API request and response parameters manually into HTML tables is not ideal. We plan to auto-generate HTML table snippets as Jekyll modules from our existing API modeling specs written in RAML.

Migrating All Docs

Right now, we have only migrated our API reference sections to developer.wepay.com. Ideally, we would migrate all docs hosted on wepay.com/developer to take advantages of a flexible system.

Automatic Failover

We have run into cases where a deployment can break the existing GitHub Pages and a simple revert doesn’t bring them back up. We have also seen outages. During such times, we would ideally like to failover to a public Google Cloud Storage bucket containing a stable static site built by Jekyll. We could just permanently use such a solution if we find a better option for redundancy, easy deployment, and CDN.

Open Source

We plan to open source our developer docs repo, auto-generation, and RAML specs. This allows other developers and our customers to make changes to our docs.

Conclusion

If you already use GitHub and GitHub Enterprise, hosting static websites using them is easy if you can watch for pitfalls and are okay with the tradeoffs. This setup is perfect for developer docs, blogs, support forums, and more. It provides a powerful, flexible way for most employees and non-employees to contribute while making deployments a breeze. More importantly, this setup frees up DevOps teams from managing infrastructure for static sites, leaving resources for mission critical web apps.