Getting Started With DevOps for Embedded Systems Using the ATmega328P

DevOps and Agile methodologies have transformed software development by emphasizing collaboration, automation, and continuous improvement. Applying DevOps principles to my designs and projects has been a game-changer, enhancing efficiency and reliability. In this article, we’re going to walk through setting up a continuous integration (CI) workflow for an existing embedded systems project that  uses the ATmega328P microcontroller. By the end of this article, you'll see how these practices can streamline your development process and deliver higher quality products.

Understanding DevOps and Agile for Embedded Systems

DevOps is a set of practices, popularized by the software world, that bridges software development (Dev) and IT operations (Ops) into a continuous flow. In the software world, it used to be common to develop software and “throw it over the wall” to the operations folks for deployment to the customers. DevOps introduced a way to not only tear down that wall but automate the whole process end to end. In the hardware world we find similarities between product development and production, constantly throwing the design “over the wall” to our manufacturing engineering teams to ensure that everything is ready to go for production.

In embedded product design we still have to get our software through production but face the challenge of moving faster than ever and delivering at the highest quality possible. With DevOps principles we aim to solve some of those challenges.

• Hardware Dependencies: Embedded systems depend on hardware and specific revisions of those PCBs. This can make testing and deployment complex if not streamlined to be automated and highly scalable. DevOps practices help automate these processes by using the same setup for both the hardware and software and putting it through automated continuous integration (CI) systems.
• Long Build Times: Building embedded software can be difficult to set up and result in long build times. CI automates and speeds up this process by offloading builds to the cloud utilizing beefier instances that developers don’t normally have access to.
• Manual Testing: Testing on actual hardware is essential but often manual, tedious, and time intensive. Automation through hardware-in-the-loop (HIL) testing improves efficiency and accuracy and can be offloaded to a setup of automated test equipment configured with your CI system.

By applying DevOps principles we’re able to iterate quickly using Agile methodologies within the build-test-deploy paradigm for each additional feature we wish to release to production.

How it all Works

“Build, test, and deploy” is a common set of words you’ll often hear when discussing DevOps. In embedded systems we do the same thing as our deployment also goes to production (and then the end customer). In the project’s repository we’re using Gitlab CI to drive our end-to-end workflow for Embedded DevOps. We use what’s called “pipelines” to create jobs that achieve certain tasks such as compiling the software, running tests on target, or releasing it as an official package. In Gitlab, a pipeline is a collection of jobs that run in a sequential flow like this:

Figure 1: Example pipeline used with the ATmega328P DevOps workflow in Gitlab

Here is a breakdown of the CI script (.gitlab-ci.yml file) to give you an idea of how this works.

• Docker: As discussed in Containerizing Build and Runtime Environments for Hardware in the Loop Testing, this stage builds Docker images to create a consistent environment for building, testing, and flashing the code. This ensures that the build process is reproducible across different machines and architectures (such as a desktop PC versus a Raspberry Pi).
• Test: This stage runs unit tests to verify that your code is doing what you intend it to do. Automated tests are fast and important when modifying or refactoring existing code.
• Build: This stage compiles the source code into binaries. In this project it generates artifacts like .elf and .hex files, which are used to flash the ATmega328P chip.
• HIL (Hardware-in-the-Loop): This stage involves testing the software on actual hardware to ensure it works correctly in real-world conditions. It loads the software onto the hardware and runs tests to verify the functionality that we designed actually works on the end product.
• Deploy: This stage handles the publishing of the built artifacts to a package registry, making them available for use.
• Release: This stage creates a new software release, automating the delivery process to ensure quick and reliable updates. This is what the production team would use to retrieve the version of software needed for their entire assembly of the product.

Fine Details

There are some minor details that take this workflow from a barebones DevOps implementation to a smoothly running, well-documented and easily observed system. There are a few subtle details within the CI workflow that are important to point out.

• Semantic Versioning: Setting a version, whether via an automated mechanism or manually, is extremely important for the release cycle, especially when working with production. As an example, this is set as a variable within the CI script and then used in the publish and release jobs.
• Docker build logic: You’ll notice there is a block of logic that is used for the Docker container builds:

if [ "$CI_COMMIT_REF_SLUG" == "$CI_DEFAULT_BRANCH" ]; then

  export IMAGE_TAG=$CI_REGISTRY_IMAGE/$IMAGE_TYPE:latest

else

  export IMAGE_TAG=$CI_REGISTRY_IMAGE/$IMAGE_TYPE:$CI_COMMIT_REF_SLUG

fi

This logic sets the stage for our “latest” tag to only use the Docker image built on the main branch (i.e. after a merge request successfully passes). This ensures that only successful merge requests publish the latest and greatest docker image that everyone and every pipeline pulls from.

• Reports and Coverage: In modern CI systems such as Gitlab we’re able to extract and display test and coverage reports within our merge requests and pipelines. For example, in this merge request we’ve captured the code coverage:

Figure 2: Code coverage within a merge request

In this screenshot, Gitlab summarizes the tests run on the target using hardware in the loop:

Figure 3: Test summary for tests run on target

In the end, once our code has been validated both with unit testing and on target, the publish and release stages generate a nice package that can be consumed by the production team:

Figure 4: Software package release

With all these automated steps we can release a new feature iteratively in an Agile fashion. There is no need for developing many features, sending them to the QA department, and then a review for package release by the production team. Everything here happens in a single workflow and it’s completely automated.

Conclusion

In this article, we explored how DevOps and Agile methodologies can be applied to embedded systems development, specifically using the ATmega328P microcontroller. We discussed the benefits of implementing a CI workflow in Gitlab, which includes automation, faster build times, and efficient testing. By breaking down the CI script and explaining each stage, we showed how to create a robust and streamlined development process that increases efficiency and product quality. By following this practical guide (and the source code within the repository) you should be able to set up your own Embedded DevOps workflow as well.

The project’s source code can be found here:https://gitlab.com/embedded-designs/atmega328p-serial-led-control.