Continuous Integration and Continuous Deployment¶
Overview¶
Each git commit to the master branch in GitHub triggers both the TravisCI automated tests as well as the AWS CodeBuild building. Each git tag applied to a git commit triggers a CodeBuild build.
Travis CI¶
Travis CI runs tests on the MozDef code base with each commit to master. The results can be seen on the Travis CI MozDef dashboard
The Test Sequence¶
- Travis CI creates webhooks when first setup which allow commits to the MozDef GitHub repo to trigger Travis.
- When a commit is made to MozDef, Travis CI follows the instructions in the .travis.yml file.
- .travis.yml installs docker-compose in the before_install phase.
- In the install phase, Travis runs the build-tests make target which calls docker-compose build on the docker/compose/docker-compose-tests.yml file which builds a few docker containers to use for testing.
- In the script phase, Travis runs the
tests
make target which
- calls the build-tests make target which again runs docker-compose build on the docker/compose/docker-compose-tests.yml file.
- calls the
run-tests
make target which.
- calls the run-tests-resources make target which starts the docker containers listed in docker/compose/docker-compose-tests.yml.
- runs flake8 with the .flake8 config file to check code style.
- runs py.test tests which runs all the test cases.
AWS CodeBuild¶
Enabling GitHub AWS CodeBuild Integration¶
Onetime Manual Step¶
The steps to establish a GitHub CodeBuild integration unfortunately require a onetime manual step be done before using CloudFormation to configure the integration. This onetime manual step need only happen a single time for a given AWS Account + Region. It need not be performed with each new CodeBuild project or each new GitHub repo
- Manually enable the GitHub integration in AWS CodeBuild using the
dedicated, AWS account specific, GitHub service user.
- A service user is needed as AWS CodeBuild can only integrate with GitHub from one AWS account in one region with a single GitHub user. Technically you could use different users for each region in a single AWS account, but for simplicity limit yourself to only one GitHub user per AWS account (instead of one GitHub user per AWS account per region)
- To do the one time step of integrating the entire AWS account in
that region with the GitHub service user
- Browse to CodeBuild in AWS and click Create Project
- Navigate down to
Source
and setSource Provider
toGitHub
- For
Repository
selectConnect with a GitHub personal access token
- Enter the persona access token for the GitHub service user. If
you haven’t created one do so and grant it
repo
andadmin:repo_hook
- Click
Save Token
- Abort the project setup process by clicking the
Build Projects
breadcrumb at the top. This “Save Token” step was the only thing you needed to do in that process
Grant the GitHub service user access to the GitHub repository¶
- As an admin of the GitHub repository go to that repositories settings, select Collaborators and Teams, and add the GitHub service user to the repository
- Set their access level to
Admin
- Copy the invite link, login as the service user and accept the invitation
Deploy CloudFormation stack creating CodeBuild project¶
Deploy the mozdef-cicd-codebuild.yml
CloudFormation template
to create the CodeBuild project and IAM Role
The Build Sequence¶
- A branch is merged into master in the GitHub repo or a version git tag is applied to a commit.
- GitHub emits a webhook event to AWS CodeBuild indicating this.
- AWS CodeBuild considers the Filter Groups configured to decide if the tag
or branch warrants triggering a build. These Filter Groups are defined in
the
mozdef-cicd-codebuild.yml
CloudFormation template. Assuming the tag or branch are acceptable, CodeBuild continues. - AWS CodeBuild reads the buildspec.yml file to know what to do.
- The install phase of the buildspec.yml fetches
packer and unzips it.
- packer is a tool that spawns an ec2 instance, provisions it, and renders an AWS Machine Image (AMI) from it.
- The build phase of the buildspec.yml runs the cloudy_mozdef/ci/deploy script in the AWS CodeBuild Ubuntu 14.04 environment.
- The deploy script calls the build-from-cwd target of the Makefile which calls docker-compose build on the docker-compose.yml file, building the docker images in the AWS CodeBuild environment. These are built both so they can be consumed later in the build by packer and also for use by developers and the community.
- deploy then calls the
docker-push-tagged
make target which calls
- the tag-images make target which calls the cloudy_mozdef/ci/docker_tag_or_push tag script which applies a docker image tag to the local image that was just built by AWS CodeBuild.
- the
hub-tagged
make target which calls the
cloudy_mozdef/ci/docker_tag_or_push push
script which
- Uploads the local image that was just built by AWS CodeBuild to DockerHub. If the branch being built is master then the image is uploaded both with a tag of master as well as with a tag of latest.
- If the branch being built is from a version tag (e.g. v1.2.3) then the image is uploaded with only that version tag applied.
- The deploy script next calls the
packer-build-github
make target in the
cloudy_mozdef/Makefile
which calls the
ci/pack_and_copy
script which does the following steps.
- Calls packer which launches an ec2 instance, executing a bunch of steps and and producing an AMI
- Shares the resulting AMI with the AWS Marketplace account
- Copies the resulting AMI to a list of additional AWS regions
- Copies the tags from the original AMI to these copied AMIs in other regions
- Shares the AMIs in these other regions with the AWS Marketplace account
- Creates a blob of YAML which contains the AMI IDs. This blob will be used in the CloudFormation templates
- When ci/pack_and_copy calls packer, packer launches an ec2 instance based on
the configuration in
cloudy_mozdef/packer/packer.json
- Within this ec2 instance, packer clones the MozDef GitHub repo and checks out the branch that triggered this build.
- Packer replaces all instances of the word latest in the docker-compose-cloudy-mozdef.yml file with either the branch master or the version tag (e.g. v1.2.3).
- Packer runs docker-compose pull on the docker-compose-cloudy-mozdef.yml file to pull down both the docker images that were just built by AWS CodeBuild and uploaded to Dockerhub as well as other non MozDef docker images.
- After packer completes executing the steps laid out in packer.json inside the ec2 instance, it generates an AMI from that instance and continues with the copying, tagging and sharing steps described above.
- Now back in the AWS CodeBuild environment, the deploy script continues by
calling the
publish-versioned-templates
make target which runs the
ci/publish_versioned_templates
script which
- injects the AMI map yaml blob produced earlier into the mozdef-parent.yml CloudFormation template so that the template knows the AMI IDs of that specific branch of code.
- uploads the CloudFormation templates to S3 in a directory either called master or the tag version that was built (e.g. v1.2.3).