It hurts, doesn’t it? Merging feature branches into the main development branch, I mean. The mere thought of continuous integration and having to go through these merges continually, makes you shudder.
But should it? What if you’d no longer have to go through extended periods of integration and validation before a release?
It is possible. Others are doing without it.
And you can too.
Read on to learn what continuous integration is, how it helps teams deliver better software, faster, how its benefit will ripple through your organization, the practices to follow, and the mistakes to avoid.
Better Software, Faster
You can’t avoid integrations.
Unfortunately, postponing them only makes them harder and more painful. Believe me, Martin Fowler is right when he says: “The effort of integration is exponentially proportional to the amount of time between integrations”.
And he showed, that if you integrated everyone’s changes all the time, each integration would be simple and the pain would melt away.
So, do it all the time? Really? All the time?
Yes. All the time.
Let’s clarify what you’re looking at by comparing the old way and the continuous integration way.
Branching the Old Way
Without continuous integration the branches and commits on them in your source control system, look something like this:
Img Src: Bluecast
- The main development branch, or trunk, is where all features and bug fixes come together.
- The release candidate branch is deployed to one or more staging environments for (re-)verification of features after the integration, for preview use, and for further testing such as performance, scalability, and security testing.
- The released branch is where escaped bugs are reproduced and showstoppers (when a customer can’t work because of a bug in your software) are fixed.
- Many development branches in which new features are designed and implemented.
You create development branches when you start work on a new feature by branching off the main development branch. And you remove them after you’ve satisfied yourself that the feature is working as intended and you’ve integrated the changes back into the main development branch.
Feature Branching Heaven or Hell
This feature branching model as it is known, was and is popular because it gives developers a steady base to build on as each feature branch is isolated from the others. The disadvantage is that this allows changes to the same code to diverge. For example:
- You have source code files that you change for every new feature that’s developed and every bug that’s fixed.
- Different solutions to the same requirement may have been created.
- Two features may both have changed the same function in potentially contradictory ways.
- Specific code may have been changed in one branch and moved to a different file in another branch, or even moved within the same file.
- One feature may extensively have refactored code that’s also been changed by other features.
When you finally bring those features together by integrating them back into the main development branch, you’ll have to sort it all out. The longer you’ve waited, the bigger the divergence, the bigger the headaches. And you’ll have to re-test many features to ensure the integration didn’t mess them up.
Continuous integration addresses this by keeping the time between integrations short.
What is Continuous Integration?
Simply put, continuous integration means taking two steps:
- Integrating changes by individual developers back into the main development branch, and
- Automatically approving (validating) the integration result, continuously.
Ideally, you want to integrate and approve every change to your main source code repository. And you want developers to submit their changes at least once a day to the main development branch, preferably more often.
Let’s look at what that means for branching in your source code repository.
Branching the Continuous Integration Way
With continuous integration, your branching model becomes a whole lot simpler. You still have the two or three basic branches: the “main development” branch, potentially a “release candidate” branch, and the “released” branch.
But you won’t have any more feature branches.
Yes, developers still work in separate branches, but these are short-lived. Just long enough to make and verify the code changes needed to implement a small part of a feature. Then the branch is immediately integrated back into the main development branch and removed.
The main development branch is also known as “the trunk”, giving rise to trunk based development as the name for this way of working.
Continuous integration means that code divergence cannot grow out of control and merge conflicts are few and far between. When they do occur, they’re manageable and the developers involved are still fully up to speed with what they changed. So, collaborating to address the conflict is quick and easy.
Continuous Integration Heaven or Hell
When you first start with continuous integration, you’ll hear protests about receiving other people’s changes being disruptive. It’s no fun debugging your code only to find that the strange behavior you’re seeing is caused by changes you just pulled in.
That’s why the approval part of CI is essential. It protects everyone from each other’s slip-ups. By code reviews before integration or as part of a pull request; and through automated tests in the build that’s triggered by an integration commit.
Without the – automated – approval of CI, other continuous practices would be impossible, and Agile development and the close collaboration between developers and operations people in DevOps impractical.
Let’s have a closer look.
What do CI and CD Mean?
Actually, CD stands for two approaches, so let’s define all three.
- In Continuous Integration you build (compile and assemble) and test (unit and integration tests) your software product automatically. Note: “a build” refers to the entire process.
- Continuous Delivery extends CI. With it, you get automated acceptance testing and deployment to staging environments where you can ensure your product’s performance, security, and scaling abilities. More importantly, continuous delivery ensures that every build is ready for deployment to production and making it available for your customers. So you can do that whenever it suits you.
- With Continuous Deployment, you deploy every approved and delivered integration to production automatically. Feature Flags allow you to maintain control over when features are made available to which of your customers.
All three are central to DevOps and Agile. So, what’s that all about?
What is Continuous Integration in DevOps and Agile?
Practicing continuous integration supports three of the four values in the Agile Manifesto:
- working software through automated validation and approval,
- customer collaboration through enabling continuous delivery and deployment, and
- responding to change by shortening the time it takes to put a feature into production.
Developers like to move fast and make frequent changes. Operations people like minimal disruption to meet their service level agreements. As DevOps aims to bridge that gap, it’s no wonder that continuous integration is a DevOps core practice.
The safety net of automated integration approval allows developers, operations people, and the business they work for to move forward with confidence.
Ok, but what’s the mechanism? What are the methods, tools, and practices used in CI?
How Does Continuous Integration Work?
Img Src: patrick-bareiss.com
To start with continuous integration, you need:
- Of course, a single source control repository. Even in the day and age of distributed source control, you need a single repository to hold the single source of truth.
- An integration or build server that keeps tabs on the repository and starts an automated build when it detects an integration to the main development branch.
- A build process (or pipeline) that:
- Compiles and assembles the software components.
- Runs the unit and integration tests to validate and approve the integration.
- Reports the result to everyone involved.
- Makes the software components available so they can easily be deployed.
- If you’re also using continuous delivery: triggers the process to deploy the software components to staging environments and start automated acceptance testing.
That gets you started, but the true purpose of continuous integration is rapid feedback. For the automated approval of integration results ultimately drives all the benefits of continuous integration, delivery, and deployment.
So, your rapid feedback had better be good.
Test-first Development
You can use all the CI tools you like, but without test-first practices, your rapid feedback safety net is likely to be full of holes.
You need development practices that get developers to focus on verification of what they implement.
The most well-known development practice is Test Driven Development (TDD), also sometimes named Test Driven Design when you truly use a test-first approach and use the friction it creates in writing your tests to drive the design of your software. TDD is the main practice for creating unit and integration tests.
Behavior Driven Development (BDD) takes TDD to the business. Technical staff and experts from the business work together to specify how your application needs to behave. In other words, BDD specifies acceptance tests and uses tools in the continuous delivery pipeline to verify that your app acts as expected.
There are many test automation tools for use with these practices. Frameworks for unit testing, also often used to implement integration tests; tools for automated UI testing; specific BDD tools that combine declarative test specification and automated execution; to name a couple.
Small is beautiful
To leverage the rapid feedback from TDD and BDD, you want it often. That means every developer needs to be focused on triggering the CI/CD/CD pipeline often by:
- Making small changes, and
- committing (integrating) to the main development branch frequently, at least once a day.
Out of Sight, out of Heart
“Energy flows where attention goes”, says Tony Robbins.
So, keep CI results top of mind to ensure everyone stays engaged. Use visualization and have some fun with feedback devices.
Benefits of Continuous Integration, Besides Better Software, Faster Ci brings more benefits than better software, faster:
- No longer having to deal with long and arduous integrations, makes everyone involved happier and more confident. And happy, confident people are more successful.
- More people can work and collaborate on the same code base safely by alerting everyone to trouble as it is created. Growing your developer teams is no longer an issue.
- With tedious tasks automated, developers can use their brainpower to delight your customers with features they’ll gobble up and pay for happily.
- You can integrate CI results with project and product management reports, for example from issue trackers and Scrum or Kanban boards, into a single dashboard. The transparency helps communication and reduces the effort you need to put into expectation management.
- Better software reduces the burden on your support staff.
- You can use continuous deployment to enjoy faster feedback from and be more collaborative with your customers. Your customers will feel like co-creators, giving them the Ikea effect for your product.
- With early customer feedback you can stop wasting effort on a bright idea that didn’t land well and focus on ideas that do.
The Common Mistakes That Trip You Up
CI poses challenges that can easily lead you to make mistakes and shoot yourself in the proverbial foot.
- Underestimating the intentional, sustained attention it takes to ditch old habits and grow new ones.
- Not being clear on what quality means for you and wasting time and effort through not having a test strategy before automating tests.
- Keeping or bringing back merge hell by tolerating long-lived branches.
- Foregoing the driving force of transparently showing your aspirations and results.
- Slowing rapid feedback by allowing your automated build process to take more than 10 minutes.
- Weakening your safety net by tolerating brittle unit and integration tests, by bad test code or by using shared testing environments.
You Can Do This
Just imagine how it will feel when you no longer have to suffer through integrating separate feature branches to release the goodness in them to your customers.
Just imagine everything you can create for your customers once you’re able to deliver better software, faster.
Just imagine the customer satisfaction and loyalty once you can get their feedback and collaboration earlier than you ever thought possible.
You now know it’s not some unattainable dream. And you now know how to do it.
So get your teams together and start by getting that alignment on what quality means for all of you and what testing strategy you’re going to use to ensure it.