Elixir Gitlab CI Example March 10, 2021

Inspired by @akoutmos’s tweet on Elixir test coverage ExUnit Test Coverage Tweet, I wanted to show off how you can integrate this into Gitlab CI for tracking test coverage in Gitlab.

At Skylla we have been using Gitlab for source control and CI for a while a now. I have enjoyed using Gitlab and it works well for us. Gitlab CI has a ton of different options and ways to use it so I wanted to showcase an Elixir app pipeline.

This example will walk through setting up CI tests, test coverage tracking, linting, and publishing an Elixir release.


For this I set up a new mix project with mix new example_gitlab_ci_ex and set up a Gitlab repository at agundy/example_gitlab_ci_ex.


Gitlab CI reads a file in the root directory and can include templates or other files in subdirectories. For now everything well be in one file and we’ll set up which stages our CI pipeline will have and a reusable Elixir helper for different CI jobs.


  - test
  - publish

.elixir: &elixir
  image: elixir:1.11
    - mix local.hex --force
    - mix local.rebar --force
    - mix deps.get --only $MIX_ENV


Elixir has some nice built in code quality tools. I set up this repo to be strict, rejecting code that if is not formated or there are any compilation warnings.

  extends: .elixir
  stage: test
    MIX_ENV: test
    - mix compile --warnings-as-errors --force
    - mix format --check-formatted

If you want to be less strict you could remove some of the checks or add allow_failure: true to the block. Allow failure tells Gitlab CI to run the job but don’t error if the check fails, just show a warning.

Tests and Coverage

CI with tests does not provide much value so let’s add a test stage. Here’s a basic Elixir test stage that will run tests in the repo, reusing that same .elixir job partial.

  extends: .elixir
  stage: test
    MIX_ENV: test
    - mix test

Test coverage is a whole other blog post, I do not believe in 100% code test coverage or hard requirements but use it as a lossy signal of repository health. I treat it the same way I do a step counter, good for viewing overall trends but a poor snapshot judging a specific day/merge request. Updating our script mix test to mix test --cover gets us a nice little report for viewing in the CI output.

Elixir Test Coverage Report

Once we have coverage in CI what else can we do? Gitlab has two features we can integrate with to get even more information into our merge requests and project analytics, coverage percentage information and parsed test failure feedback.


In the projects CI/CD configuration under “General pipeline settings” is a “Test coverage parsing” field and the Regex we want to add there is: \d+.\d+\%\s+\|\s+Total.


Gitlab supports junit.xml reports which are a standard for test reports. Elixir does not export these by default but we can install a package that helps us and configure it in two steps.

  1. In mix.exs add {:junit_formatter, "~> 3.1", only: [:test]} in the deps section.
  2. In test/test_helpers.exs add ExUnit.configure(formatters: [JUnitFormatter]) to the top.

With this setup jobs will show how many tests and an include timing as well as test failures in the UI. Here is a merge request with report summaries visible: Gitlab Merge request and the pipeline test summary.

Gitlab merge request test reports Gitlab pipeline test summary


Our final test CI job will look like this:

  extends: .elixir
  stage: test
    MIX_ENV: test
    - mix test --cover
      - _build/test/lib/example_gitlab_ci_ex/test-junit-report.xml
      junit: _build/test/lib/example_gitlab_ci_ex/test-junit-report.xml 


Elixir has a few different deployment options depending on how and where you are using the code. At Skylla we use releases and so I’ll show that off here, I’m sure there are other ways to do this but one way to deploy is to set up a release, publish it to an artifact and then download it and unpack on whichever server we are running our app on.

Configure the app to compile and then compress the project in the mix.exs.

      releases: [
        example_gitlab_ci_ex: [
          steps: [:assemble, :tar],
          applications: [runtime_tools: :permanent],
          include_executables_for: [:unix]

Once it’s packing up a tar we can run mix release and have a tar under the _build/ENV folder. For CI purposes I added a little script to move it up to the root level and we store it in Gitlab artifacts.

  stage: publish
    - .elixir
    MIX_ENV: prod
    ARCH: amd64
    - mix release --overwrite
    - ARTIFACT_NAME="$(find . -name "example_gitlab_ci_ex-*.tar.gz" -exec basename {} .tar.gz \;)"
    - mv _build/prod/${ARTIFACT_NAME}.tar.gz ${ARTIFACT_NAME}-${ARCH}.tar.gz
    name: "example_gitlab_ci_ex-${CI_COMMIT_REF_SLUG:-CI_COMMIT_SHA}-${ARCH}.tar.gz"
      - example_gitlab_ci_ex-*.tar.gz
    - if: $CI_COMMIT_TAG


With this setup we now have an Elixir application that integrates with Gitlab running tests, linting, and publishing a release artifact that is downloadable from any server or device we want. Left as an exercise to the reader is getting the artifact downloaded but you can leverage Gitlab API’s with curl.

You can explore all the code and the integrations over in Gitlab at agundy/example_gitlab_ci_ex.

Some related things we’ve tackled you may find interesting enough to pester me to blog about include bundling static site docs into an Phoenix release and cross compiling Elixir for ARM in Gitlab CI.

Do you have thoughtful comments or corrections on "Elixir Gitlab CI Example"? Email me. Relevant replies may be included inline.