In CI/CD, a pipeline embodies the process where new code is submitted at one end, undergoes testing across a series of stages (source, build, staging, and production), and then emerges as production-ready code. It can iteratively approach this pipeline if an organization is new to CI/CD. This means that they should start small and iterate at each stage. It helps them to understand and develop the code in a way that will help the organization grow.
Each stage of the CI/CD pipeline structures a logical unit in the delivery process. In addition, each stage acts as a gate that vets a certain aspect of the code. As the code progresses through the pipeline, the assumption is that the quality of the code is higher in the later stages because more aspects of it continue to be verified. Problems uncovered in an early stage stop the code from progressing through the pipeline. The team receives test results promptly, and if the software fails to pass the stage, all subsequent builds and releases are halted.
Businesses can adapt the stages based on their needs. Teams can repeat stages for various types of testing, security, and performance. Depending on the project’s complexity and team structure, teams may repeat some stages multiple times at different levels. For example, the end product of one team can become a dependency on the project of the next team. This implies that the first team’s end product is then staged as an artifact in the next team’s project.
The presence of a CI/CD pipeline has a huge impact on maturing the capabilities of an organization. Organizations should start with small steps and not build a fully mature pipeline with multiple environments, many testing phases, and automation in all stages at the start. Even organizations that have highly mature CI/CD environments still need to continuously improve their pipelines.
Building a CI/CD-enabled organization is a journey, with many destinations along the way. The next section discusses a possible pathway that an organization could take, starting with continuous integration through the levels of continuous delivery.
Continuous Integration
The first phase in the CI/CD journey is to develop maturity in continuous integration. Developers should regularly commit their code to a central repository (such as one hosted in CodeCommit or GitHub) and merge all changes to a release branch for the application. Teams should avoid holding code in isolation. If a feature branch is necessary for a period, it should be regularly updated by merging from upstream. The process encourages frequent commits and merges with complete units of work to foster discipline in the team’s development. A developer who merges code early and often will likely have fewer integration issues down the road.
Developers should create unit tests as early as possible for the applications and run these tests before pushing the code to the central repository. Errors caught early in the software development process are the cheapest and easiest to fix.
When developers push code to a branch in a source code repository, a workflow engine monitors the branch and sends a command to a builder tool to build the code and run unit tests in a controlled environment. To ensure prompt feedback, teams must appropriately scale the build process to manage all activities, including pushes and tests during the commit stage. Other quality checks, such as unit test coverage, style check, and static analysis, can also happen at this stage. Finally, the builder tool creates one or more binary builds and other artifacts, like the application’s images, stylesheets, and documents.
Continuous Delivery – Creating a staging environment
Continuous delivery (CD) is the next phase and entails deploying the application code in a staging environment, which is a replica of the production stack and running more functional tests. For testing, teams could set up the staging environment as either a premade static environment or a dynamically provisioned and configured environment with committed infrastructure and configuration code to test and deploy the application code.
Continuous delivery: creating a production environment
Continuous Deployment
The final phase in the CI/CD deployment pipeline is continuous deployment, which may include full automation of the entire software release process including deployment to the production environment. In a fully mature CI/CD environment, full automation of the path to the production environment enables deploying code with high confidence.
Path Forward and Beyond
As organizations matures, it will continue to develop the CI/CD model to include more of the following improvements:
- More staging environments for specific performance, compliance, security, and user interface (UI) tests
- Unit tests of infrastructure and configuration code along with the application code
- Integration with other systems and processes such as code review, issue tracking, and event notification
- Integration with database schema migration (if applicable)
- Additional steps for auditing and business approval
- Even the most mature organizations that have complex multi-environment CI/CD pipelines continue to look for improvements. DevOps is a journey, not a destination. Feedback about the pipeline is continuously collected. It leads to improvements in speed, scale, security, and reliability. Different parts of the development teams collaborate to achieve this.