--- interact_link: content/advanced/circleci.ipynb kernel_name: python3 has_widgets: false title: |- Previewing or building your book with CircleCI prev_page: url: /advanced/advanced.html title: |- Advanced Topics next_page: url: /01/what-is-data-science.html title: |- Data Science comment: "***PROGRAMMATICALLY GENERATED, DO NOT EDIT. SEE ORIGINAL FILES IN /content***" ---
CircleCI is a continuous integration service that lets you run various commands every time a new change is made to a repository. This can be used to build your book, preview changes, and even push live HTML as you update your book content.
In order to accomplish each of these, we'll use a CircleCI configuration file. This is a YAML file that is used to tell Circle what to do with your repository.
In each case, the expectation is that your master branch holds your book content.
We'll step through each piece of a sample CircleCI configuration to show you how to accomplish this.
First of all, you should set up your CircleCI account to start running CI jobs for your book repository. Follow these steps:
Now, CircleCI will start watching your repository. If it finds a Circle configuration file (more information on this below, it'll run a CI job according to the configuration it finds.
You can copy/paste an empty CircleCI configuration here. This won't actually do anything but we'll add to it later:
# Tell CircleCI which version of its API you wish to use
version: 2.1
# Commands are snippets of configuration we can re-use for convenience
commands:
copy_built_files:
description: "Copy the build site files so we can build the HTML"
steps:
# Grab the the built markdown from the last step
- run:
name: Copy over built site files
command: |
rm -rf _build
cp -r /tmp/workspace/_build .
First you'll build each page's HTML. This is the initial conversion from
ipynb
, md
, etc files. We'll use a Python container for this in order to
use the Jupyter Book command-line interface with jupyter-book build
.
You can build your book's HTML files and preview them using CircleCI artifacts. To do this, you'll need to use two CircleCI jobs:
We'll need to persist the results of this step so that they are available in subsequent steps. Here's the CircleCI configuration that will accomplish this, which you can add to the skeleton configuration you've created above:
jobs:
build_page_files:
docker:
- image: circleci/python:3.6-stretch
steps:
# Get our data and merge with upstream
- checkout
# Install the packages needed to build our documentation
# This will depend on your particular package!
- run: pip install --user -r requirements.txt
# Build the page intermediate HTML files
- run:
name: Build page intermediate HTML files
command: jupyter-book build .
# Persist the specified paths (see https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs)
- persist_to_workspace:
root: .
paths:
- ./_build/
# Note that this
jobs:
# Build the site to store artifacts
html_demo:
docker:
- image: circleci/ruby:2.6
steps:
- checkout
# Attach the workspace that contains the files from our Python step
- attach_workspace:
at: /tmp/workspace
# Copy over the built page files
- copy_built_files
# Build the site's HTML w/ the base_url for CircleCI artifacts
- run:
name: Install Bundler and Jekyll, then build the book
command: |
gem install bundler
bundle install
bundle exec jekyll build --baseurl /0/html/
# Tell Circle to store the documentation output in a folder we can access later
- store_artifacts:
path: _site/
destination: html
You can also choose to automatically push changes to your master branch
to your live textbook. This is accomplished by using the built HTML artifacts
from the previous step. If you include a security key that allows CircleCI push access
to your GitHub repository, then you can use this setup to push newly-built HTML
to your gh-pages
branch. See the below configuration for how to accomplish this.
jobs:
# Finally, push the HTML to the gh-pages branch
html_live:
docker:
- image: circleci/ruby:2.6
steps:
- checkout
# Attach the workspace that contains the files from our Python step
- attach_workspace:
at: /tmp/workspace
# Copy over the built page files
- copy_built_files
# Install the ghp-import tool which we'll use to push the files
- run:
name: "Install ghp-import"
command: |
export PATH=$PATH:/home/circleci/.local/bin
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py --user
pip install -U ghp-import --user
# Tell CircleCI to use the SSH key we've set up with permissions to push to our repo
- add_ssh_keys:
fingerprints:
# The SSH key fingerprint
- "{{ YOUR SSH KEY FINGERPRINT }}"
# Build the site's HTML using the base_url defined in the *book's* configuration
- run:
name: Install Bundler and Jekyll, then build the book
command: |
gem install bundler
bundle install
bundle exec jekyll build
# Push the `_site` folder to our gh-pages branch for it to go live
- run:
name: Push site to gh-pages
command: |
export PATH=$PATH:/home/circleci/.local/bin
ghp-import -p -n -f _site
Now that we've defined several jobs above, we need to tell CircleCI how to use them sequentially (or in parallel). In particular, we want the Python job to run first so that the Page HTML is ready, then we can run the HTML demo job or the HTML live job. Here's the configuration for this:
workflows:
version: 2
default:
jobs:
# First we build the pages
- build_page_files
# Then we build the book HTML to preview
- html_demo:
requires:
- build_page_files
# If we're on master branch, also push the HTML live
- html_live:
requires:
- build_page_files
filters:
branches:
only:
- master
# NOTE: This is an example CircleCI configuration that
# will build your book and preview its HTML content.
# You will probably have to modify it in order to get it working
# just the way you want. See https://jupyter.org/jupyter-book/advanced/circleci.html
# for more information
version: 2.1
# Commands are snippets of configuration we can re-use
commands:
copy_built_files:
description: "Copy the build site files so we can build the HTML"
steps:
# Grab the the built markdown from the last step
- run:
name: Copy over built site files
command: |
rm -rf _build
cp -r /tmp/workspace/_build .
# Jobs define the actual code that CircleCI runs
jobs:
build_page_files:
docker:
- image: circleci/python:3.6-stretch
steps:
# Get our data and merge with upstream
- checkout
# Install the packages needed to build our documentation
# This will depend on your particular package!
- run: pip install --user -r requirements.txt
# Build the page intermediate HTML files
- run:
name: Build page intermediate HTML files
command: jupyter-book build .
# Persist the specified paths (see https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs)
- persist_to_workspace:
root: .
paths:
- ./_build/
# Build the site to store artifacts
html_demo:
docker:
- image: circleci/ruby:2.6
steps:
- checkout
# Attach the workspace that contains the files from our Python step
- attach_workspace:
at: /tmp/workspace
# Copy over the built page files
- copy_built_files
# Build the site's HTML w/ the base_url for CircleCI artifacts
- run:
name: Install Bundler and Jekyll, then build the book
command: |
gem install bundler
bundle install
bundle exec jekyll build --baseurl /0/html/
# Tell Circle to store the documentation output in a folder we can access later
- store_artifacts:
path: _site/
destination: html
# Finally, push the HTML to the gh-pages branch
html_live:
docker:
- image: circleci/ruby:2.6
steps:
- checkout
# Attach the workspace that contains the files from our Python step
- attach_workspace:
at: /tmp/workspace
# Copy over the built page files
- copy_built_files
# Install the ghp-import tool which we'll use to push the files
- run:
name: "Install ghp-import"
command: |
export PATH=$PATH:/home/circleci/.local/bin
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py --user
pip install -U ghp-import --user
# Tell CircleCI to use the SSH key we've set up with permissions to push to our repo
- add_ssh_keys:
fingerprints:
# The SSH key fingerprint
- "4e:f9:a5:62:b0:62:f5:d4:7d:f2:52:0a:53:c9:d6:39"
# Build the site's HTML using the base_url defined in the *book's* configuration
- run:
name: Install Bundler and Jekyll, then build the book
command: |
gem install bundler
bundle install
bundle exec jekyll build
# Push the `_site` folder to our gh-pages branch for it to go live
- run:
name: Push site to gh-pages
command: |
export PATH=$PATH:/home/circleci/.local/bin
ghp-import -p -n -f _site
# These workflows define how the jobs above will run. In our case, we
# want the Ruby jobs to run *after* the Python job has finished so that
# they can use the outputs from the Python page build step.
workflows:
version: 2
default:
jobs:
# First we build the pages
- build_page_files
# Then we build the book HTML to preview
- html_demo:
requires:
- build_page_files
# If we're on master branch, also push the HTML live
- html_live:
requires:
- build_page_files
filters:
branches:
only:
- master