Inspired by @akoutmos’s tweet on Elixir test coverage ExUnit Test Coverage Tweet, I wanted to show off how you can integrate this into Gitlab CI for tracking test coverage in Gitlab.
At Skylla we have been using Gitlab for source control and CI for a while a now. I have enjoyed using Gitlab and it works well for us. Gitlab CI has a ton of different options and ways to use it so I wanted to showcase an Elixir app pipeline.
This example will walk through setting up CI tests, test coverage tracking, linting, and publishing an Elixir release.
Setup
For this I set up a new mix project with mix new example_gitlab_ci_ex
and set
up a Gitlab repository at
agundy/example_gitlab_ci_ex.
.gitlab-ci.yml
Gitlab CI reads a file in the root directory and can include templates or other files in subdirectories. For now everything well be in one file and we’ll set up which stages our CI pipeline will have and a reusable Elixir helper for different CI jobs.
---
stages:
- test
- publish
.elixir: &elixir
image: elixir:1.11
before_script:
- mix local.hex --force
- mix local.rebar --force
- mix deps.get --only $MIX_ENV
Linting
Elixir has some nice built in code quality tools. I set up this repo to be strict, rejecting code that if is not formated or there are any compilation warnings.
lint:elixir:
extends: .elixir
stage: test
variables:
MIX_ENV: test
script:
- mix compile --warnings-as-errors --force
- mix format --check-formatted
If you want to be less strict you could remove some of the checks or add
allow_failure: true
to the block. Allow failure tells Gitlab CI to run the
job but don’t error if the check fails, just show a warning.
Tests and Coverage
CI with tests does not provide much value so let’s add a test stage. Here’s a
basic Elixir test stage that will run tests in the repo, reusing that same
.elixir
job partial.
test:elixir:
extends: .elixir
stage: test
variables:
MIX_ENV: test
script:
- mix test
Test coverage is a whole other blog post, I do not believe in 100% code test
coverage or hard requirements but use it as a lossy signal of repository
health. I treat it the same way I do a step counter, good for viewing overall
trends but a poor snapshot judging a specific day/merge request. Updating our
script mix test
to mix test --cover
gets us a nice little report for
viewing in the CI output.
Once we have coverage in CI what else can we do? Gitlab has two features we can integrate with to get even more information into our merge requests and project analytics, coverage percentage information and parsed test failure feedback.
Coverage
In the projects CI/CD configuration under “General pipeline settings” is a
“Test coverage parsing” field and the Regex we want to add there is:
\d+.\d+\%\s+\|\s+Total
.
Reports
Gitlab supports junit.xml
reports which are a standard for test reports. Elixir
does not export these by default but we can install a package that helps us and
configure it in two steps.
- In
mix.exs
add{:junit_formatter, "~> 3.1", only: [:test]}
in thedeps
section. - In
test/test_helpers.exs
addExUnit.configure(formatters: [JUnitFormatter])
to the top.
With this setup jobs will show how many tests and an include timing as well as test failures in the UI. Here is a merge request with report summaries visible: Gitlab Merge request and the pipeline test summary.
Summary
Our final test CI job will look like this:
test:elixir:
extends: .elixir
stage: test
variables:
MIX_ENV: test
script:
- mix test --cover
artifacts:
paths:
- _build/test/lib/example_gitlab_ci_ex/test-junit-report.xml
reports:
junit: _build/test/lib/example_gitlab_ci_ex/test-junit-report.xml
Publishing
Elixir has a few different deployment options depending on how and where you are using the code. At Skylla we use releases and so I’ll show that off here, I’m sure there are other ways to do this but one way to deploy is to set up a release, publish it to an artifact and then download it and unpack on whichever server we are running our app on.
Configure the app to compile and then compress the project in the mix.exs
.
releases: [
example_gitlab_ci_ex: [
steps: [:assemble, :tar],
applications: [runtime_tools: :permanent],
include_executables_for: [:unix]
]
]
Once it’s packing up a tar we can run mix release
and have a tar under the
_build/ENV
folder. For CI purposes I added a little script to move it up to the
root level and we store it in Gitlab artifacts.
publish:elixir:
stage: publish
extends:
- .elixir
variables:
MIX_ENV: prod
ARCH: amd64
script:
- mix release --overwrite
- ARTIFACT_NAME="$(find . -name "example_gitlab_ci_ex-*.tar.gz" -exec basename {} .tar.gz \;)"
- mv _build/prod/${ARTIFACT_NAME}.tar.gz ${ARTIFACT_NAME}-${ARCH}.tar.gz
artifacts:
name: "example_gitlab_ci_ex-${CI_COMMIT_REF_SLUG:-CI_COMMIT_SHA}-${ARCH}.tar.gz"
paths:
- example_gitlab_ci_ex-*.tar.gz
rules:
- if: $CI_COMMIT_TAG
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
Results
With this setup we now have an Elixir application that integrates with Gitlab running tests, linting, and publishing a release artifact that is downloadable from any server or device we want. Left as an exercise to the reader is getting the artifact downloaded but you can leverage Gitlab API’s with curl.
You can explore all the code and the integrations over in Gitlab at agundy/example_gitlab_ci_ex.
Some related things we’ve tackled you may find interesting enough to pester me to blog about include bundling static site docs into an Phoenix release and cross compiling Elixir for ARM in Gitlab CI.