A branching strategy for Scrum using Git

A branching strategy for Scrum using Git

Agility

A branching strategy in adequacy with the Scrum model.

Scrum is a great framework. If you do not know it, you should check it out and then come back here ! Defining a branching strategy suitable for Scrum is not so simple, here is a proposal.

Scrum is designed to develop product among iterations (called sprints). The sprint backlog contains stories which represents the work to do in the scope of the sprint. At the end of the sprint the Product Owner will decide if the proposed product increment is accepted or not.

It is common for Scrum teams to have strong difficulties with the branching strategy to apply to the product they are building. As a matter of fact, the expected level of collaboration on each story, the need to propose a shippable product increment, the different stories that may or not be sent in the end, all these specificities of Scrum make it challenging to find the proper way to stay clean and consistent with the code repository. That’s the reason why the branching strategy must be in adequacy with the Scrum model.

In this article we will present a branching model for Scrum using Git, oriented towards team collaboration and continuous integration. If you doubt this is worth investing some time, you may be interested in why software engineering matters.

The master branch

The development will start from your reference branch. You call it the master branch, the develop branch or by any other name. It is the one that reflects the level of code approved by the Product Owner as being ready for the production. In the scope of this article we will call it master branch. 

The sprint branch

The sprint branch aims at containing commits of completed stories, the ones that will be presented to the Product Owner during the sprint review. This branch can be seen as the “ready for demo” branch.

At the beginning of each new sprint, its sprint branch is created from the master branch (i.e.: sprint/S01). This operation can be managed by the Scrum Master for instance. It is also helpful to tag the branch at this point (you will understand why in ‘failed stories’).

The story branch (and the relevant-atomic-clean approach)

As soon as the sprint starts, a dedicated branch for each story will be created from the sprint, following a naming convention that has to be decided. It can be of the form story/S01S01, story/S01S02… Which would be read sprint 01/story 01, sprint 01/story 02, …

The important point here is that the story branch is fully part of the model. It can be seen as a  reference branch for the development of a specific feature. Even if it is a short-lived branch (like described in the popular Trunk Based Development model), it should not be considered as a "garbage" branch. More precisely, commits that will be pushed to the remote story branch should be relevant, atomic and clean. For instance, the remote story branch should likely not contain commits description like ‘working on story 2, WIP’, or even ‘fixed typo in MyClass.java’.

The cleaner the commits are in the story branch, the better the code history will be.

Branching strategy for Scrum, the big picture

Commits and interactive rebase

Of course, in the day-to-day work, the developer working on a story wants to commit his ongoing work, like a checkpoint so he knows he can restart from this point if anything goes wrong after. While implementing the story, this can end up with several commits.

Before pushing the commit(s) to the remote story branch, he should ensure that these commits are atomic and relevant. Performing an interactive rebase ("rebase -i") will allow to squash relevant commits between them and write a meaningful commit message if needed. The dev team can agree on the commit message format, like : “Story 654 – Switching from log4j to logback”, where 654 would be the story id in the product backlog, followed by a meaningful description. Only when this operation is done, the commit(s) can be pushed.

In a story branch, a pushed commit is a commit ready for code-review.

Working together, and merge-request for code review

Once a commit has been pushed, the dev team member should create a pull-request (also called merge-request) to the sprint branch in order to ease with the code review. There is no need to wait for the story to be completed to perform the merge-request: the code-review may start before the other parts of the story are implemented. But of course the story will not be merged in the sprint branch before the implementation is complete (i.e. the story is put in the column DONE in the sprint backlog).

TIP: GitLab (for instance) will prevent the branch from being merged as long as the merge-request’s title starts with “WIP:”

The code-review phase can generate one or more additional commits that cannot be squashed to the code being reviewed. That’s an acceptable state.

If several developers are working on the very same topic that requires sharing portion of code being developed, it might not be a good idea to work directly on the story branch because it would break the relevant-atomic-clean approach. Instead, they can consider branching from the original story branch or even forking the project. That way others can contribute to this temporary branch which can be interactively rebased at the end, creating new nice and beautiful squashed commits. 🙂

Story done

When the story finally meets the done criteria and the code-review is completed, it can then be merged in the sprint branch. That means the story is “ready to be shipped” from the development team point of view. All tests (unit tests and integration tests) should be rerun at this stage.

The code level in the sprint branch is the one that will be presented to the product owner during the sprint review ceremony. Uncompleted stories should not be merged into the sprint branch.

It is important to keep in mind that stories in a sprint branch (i.e. in the “done” status) has the purpose of being accepted by the Product Owner. To maximize chances, it is essential to work as closely as possible with the Product Owner. The situation of a story seen as done by the dev team and finally failed at review time should be extremely rare.

The review during the demo

During the sprint review ceremony, the product owner will review all stories of the sprint backlog. He will decide whether each story passes or fails. If you are running Scrum in a strict mode, one failed story will fail the sprint! The consequence is that none of the stories will reach the master branch at this point. The reason is that the product owner acknowledges the stories being ready for production on the very code level being presented during the review. If a failed story is removed from the sprint branch afterwards, the resulting level of code has never been reviewed and thus should not be accepted.

If all stories have passed, the sprint can be merged back to the master branch. This operation is called the delivery.

The Git tree representation of the branching model for Scrum (general case)

Failed stories

If the Product Owner has rejected at least one story, the whole sprint is failed. The way to deal with failed stories may vary a lot depending of the failure type and the Product Owner’s point of view on this. There is actually no unique answer, but here is some global leads:

1st case: the failed stories can be fixed easily.

If the failed story can be fixed easily and quickly, the Product Owner can agree on keeping the story for the delivery. The dev team will fix the sprint branch and will rerun all tests. The Product Owner should accept the fix based on the evidence usually required (test results) or a short informal demo.

2nd case: the failed story has to be removed from the delivery.

That’s the touchy case that should be handled carefuly. Here are three different kinds of approaches:

1/ Commit(s) of failed stories are reverted in the sprint branch.
This is an easy operation if story’s commits are atomic. But if this kind of operation is often done, this will lead to a dirty code base.

2/ A new sprint branch is created from the original one (like ‘sprint/01-take2’), identified by the tag created at the beginning of the sprint. All story branches of approved stories are merged into this new branch.

This is a tedious operation but the code history will be cleaner. This is a safe operation as long as the sprint branch does not contains special commits (last minute fixes for instance). That’s easy to forget those commits, be careful!

3/ A new branch is created locally from the actual sprint branch (like ‘sprint/01-rebase1’) and interactively rebased upon the original sprint branch (identified by the tag created at the beginning of the sprint). That way, only accepted stories are picked and there is no chance to forget any relevant commit.

This is the safer method, but this will generate new commits due to the rebase nature, losing the merge information of the story branch.

In all cases, the strategy should be proposed to and accepted by the Product Owner. He can decide that this is too risky and reject the product increment as a whole. The stories will then be integrated into a new sprint.

Continuous integration: some keypoints

This branching strategy works well for continuous integration.

On story branches

- Every single commit pushed to the story branches triggers at least the run of the unit tests. That’s one of the reason behind the idea of not pushing partial/work-in-progress code to the story branch. If it can technically trigger more tests, it should. The earlier the tests are run, the better

- When the story development is complete, the integration tests should run. The trigger to this operation can be defined in several ways (a tag on the commit can be an option for instance).

On sprint branch

- Once the story is merged in the sprint branch, the whole pipeline should be run (including the integration tests). This will ensure that no story broke another one.

Conclusion

I have implemented this Scrum branching strategy successfully in several companies. Of course this is not the only possible solution, but from my point of view it takes a good compromise between respecting Scrum interactions and maintaining a clean code base. It can cope correctly with the different scenarios a Scrum team can encounter.

Implementing a good branching strategy takes time, and should probably require to convince your Product Owner of the benefits. I hope this article will help you and if not, take a look at why Product Owners should have technical skills.

That is however not the end of the story: not all code changes are necessarily done in the context of Scrum. Maintenance operations and urgent fixes may follow an alternative (and complementary) process. More on this in a future article. You can as well look into this post from Gitlab that provide insights regarding git branching strategies independently of Scrum.

See what's next

Download our guide "How to grow your agile team"
Why software engineering matters

Why software engineering matters

Strategy

From startup to leader, the secret truth of software engineering

What Spotify, Criteo, Uber, Airbnb and Netflix have in common? Of course, they are young tech companies extremely successful in their fields. But beyond that, they have invested a lot in software engineering. Just like other great tech companies like Google, Facebook or Amazon. And they are very vocal about it: Google Developers, Spotify Labs, Facebook Engineering Blog, Criteo Labs (and many more), the engineering is clearly a primary concern.

I believe that there is a trend here: for a tech startup to become a leader, it takes many things, among which good software crafting practices play major role.

You are an entrepreneur? An investor? A CTO? Here are some reasons why you should really focus on software engineering.

First, let’s try to provide a definition of software engineering.

“Engineering” first (well actually second), is “the application of mathematics, empirical evidence and scientific, economic, social, and practical knowledge in order to invent, innovate, design, build, maintain, research, and improve structures, machines, tools, systems, components, materials, processes and organizations.” [Wikipedia]. In short, it means that we try here to use a scientific and rational approach to do stuff.

“Software” is the actual domain on which we apply this, ie any artefact of any size composed of code. That means anything from a simple command in a script to a fully distributed system of million of lines of code, from a desktop application to a web application, from a device driver to a social network etc.

Basically, software engineering focuses on the way software is produced and run by logically implement all the steps involved from the very first idea to operation of the running code.

When you start a software oriented business, or a new IT team in your company, the temptation to just skip all software engineering practices and jump only into the coding itself is strong. And for a (very) short period of time that can actually be OK. But soon enough, I would say you have a proof of concept, when you have few functionalities or when you reach more than few thousands lines of code, you cannot postpone anymore. Here are the reasons why.

#1 Because it saves time

Software creation, in many minds, is typing lines of code, and for this you do not need much than a text editor (we do not recommend MS Word). But many other activities are around, such as:

  • Building my code and working with different versions
  • Testing my code
  • Publishing or distributing the code
  • Assembling peace of codes together
  • Installing the code on-to where it should run
  • Documenting what the code change I just did is meant for

Software engineering has defined further the concepts around these needs. Packaging, releasing, loading… All these processes have to be carefully designed so that your R&D will not spent most of its time handling low level tasks.

A common solution is to invest in Continuous integration, Continuous delivery or even Continuous deployment. In a nutshell:

  • Continuous integration: build, test
  • Continuous delivery: build, test, publish
  • Continuous deployment: build, test, publish, assemble, deploy

Without entering into too much details (more on this in later articles), the concept in all cases is than the developer’s actions are limited to writing the adequate test and code and all the rest is just automatically done. And it saves time. A lot. There is no much figures around it, but it can hardly be much less than 50% gain. Some companies have reported moving from months’ to minutes’ time-to-market releasing on their services implementing correctly Continuous Deployment.

Most of the time it makes a huge difference in any market.

#2 Because you want to get the right product at the right time

Two years plans never met, extremely complex requirement phase, very long and painful test phases, a final product arriving after the users have changed their mind ten times… This model, inspired by industrial projects, is nowadays generally replaced by agile methods.

Collaboration, short feedback loop, constant inspection and adaptation is now supposed to be the norm. If well applied, the agile methods will bring you a lot. If badly applied, not much.

But not wondering how to work together is by far the worst choice.

#3 Because your product is here for a while

In our industry, it is pretty rare to work on short-lived products. It means that as long as you take shortcuts, you accumulate technical debt (in your coding and architecture) and “engineering debt” (in your practices).

Debt that you will pay at some point, either as large payment (large refactoring) or as interests (more and more difficult to change, maintain, deliver new versions).

Many companies, even among the largest editors in the world, have suffered very heavily by procrastinating the reduction of debt.

Investing in test coverage and test strategy and maintaining a good code sanity is vital for the durability of your products and by extension your company.

#4 Because you want the service to be up

Sadly, it is not enough to create the exact application your users need. The quality of service is as important as the provided functionalities.

How Google beat Yahoo? Among other things, the googlers invested in infrastructure as code and by this practice were able to scale, rationalize their hardware, to be faster and always up and running.

Scalability, availability, operability, response time… The so called non-functional requirements are nowadays key to any software provider that hopes to disrupt its market.

Once again, it is a serious investment that has to be done, but it can very well be the X factor between success and failure.

#5 Because it saves money

Good engineering practices dictates an automatic application of software changes. The reasons: faster, more reliable. There is no leader in this industry that apply changes manually.

Globally, many engineering practices, especially in the areas of operations and tests, advocate for automation. The era of agility encourages efficient methods to do low added value actions and to focus your energy on high added value actions.

It sounds obvious, but it is still common to see entire systems tested manually (which is not always better than not tested at all) or code deployment triggered by clicks from overloaded operators.

A lack of efficiency in these areas can quickly become a major issue: while you spend energy (ie money and resources) doing this, you do not invest in operational concerns, exploratory testing, non-functional requirements testing or other among many relevant practices where these professionals can help so much.

It is not a surprise that Netflix, followed by Amazon or Github are investing in Chaos Monkey testing[1]. They understand the importance to go further in automation to test even the one thing that no-one wants to see in a datacenter: crashing hardware.

#6 Because it is simpler

Say you are launching a startup or a new software product in your company. You recruit the team to work on this new awesome stuff and let them self-organize to build it.

The team has then two choices: define together how to work from scratch or reuse methodologies most adapted to the setup and elaborate from there the team unique ways. Guess which one goes faster and is more efficient?

Obviously, there are such alternatives only if in the team or elsewhere you find some strong knowledge of software engineering. Otherwise you are stuck with reinventing the wheel.

And that is actually the point: it is a lot simpler to rely on decades of accumulated wisdom on how to build a product or how to test or how to ship than to have to figure it out from scratch. It allows to focus on delivering the product and not on which branching strategy is the best suited for this or that purpose.

#7 Because you want your teams to be happy

So far we have covered time to market, durability, quality of service and obviously, money. All what an executive is concerned with for the profitability of his company.

I believe that the very first reason why some significant investment should be done on software engineering is for your staff.

As Richard Branson said, Companies Should Put Employees First[2]. The reason is easy to get: happy employees are doing a better job in all areas, including customer care, quality of service etc.

Can you imagine how frustrating it can be for a developer to see that his organization is so underperforming and so slow that the products he builds every day are condemned to mediocrity? That no matter what, it will always be a nightmare to see value reaching users and customers?

I know it sounds over-dramatic, but the last thing you want to see as a manager is your team turning into a disillusioned self-interested organization where nobody seem to care about what is effectively delivered at the end of the day.

And yes, a software company scaling with bad practices in its core will leave plenty of room for such situation to develop. Target happiness, use expertise in software engineering, you will get the rest.

Conclusion

In this article we have seen that software engineering has to be at the heart of the creation and maintenance of any software development unit. It is critical to put in place as soon as possible the right environment in which the team can continue to perform. It is crucial to balance the urge of deadlines and other business-related constraints with the mid term impacts shortcuts can have on your ability to deliver value.

We do not say that shortcuts are impossible and that to be efficient you have to follow every methodology around. We say that software engineering should follow a strategy, aligned with your business strategy, and that any choice needs to be carefully taken as it could impact the company viability pretty fast.

In this journey, you may need some support. In form of consulting to define the way to go or tools to simplify the implementation of a given methodology, Stenusys can help! Do not hesitate to contact us for more details.

[1] https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey

[2] https://www.inc.com/oscar-raymundo/richard-branson-companies-should-put-employees-first.html