Microservices development and CI-CD with dockers and git-submodules — GS3D

Develop and deploy to Kubernetes with GS3D using GitLab CI/CD, and Harbor

This article is based on the concepts that were presented on “Managing microservices with dockers and git-submodules — GS3D”. It assumes that the git repositories were arranged in the same way it recommends and will tell about the actual development and the ci cd flows. To read the first part:

How to manage microservices with dockers and git

Since we work with release cycles we decided to base and inspire the development and ci-cd by the Gitflow development model. This is of course optional and can be changed to other development models.

Since each micro-service is independent, the Gitflow principles will be applied in each of the microservices repositories.

Main concepts of Gitflow:

• develop branch will represent the currently developed release
• master/main branch will represent the current release
• develop and master/main branches will record the history of the project
• feature branch will contain the feature-developed changes. each feature will usually have a feature branch.
• release branch will represent the next phase of developed release before it is completed and merged to master/main branch

for further reading

A successful Git branching model

Gitflow Workflow | Atlassian Git Tutorial

Development and CI-CD with GS3D

The ci-cd flows we use are pretty bound/inspired by the same gitflow flows

Develop flow

Microservices repoitories — automatically build and deploy latest changes to dev environment.

Every change that will be pushed to develop branch on the microservices repositories will be seen automatically deployed to the dev environment.

CI-CD based on dockers implementation description:

1. build (automatic) — Any commit to develop/main/master branch will trigger a docker image build that will be pushed to a docker container registry. If the change will be pushed to develop branch the docker image will be tagged with a tag :latest, otherwise, it will be tagged with the :commit-id

2. deploy (automatic)— The image tagged with :latest will be pulled automatically to the dev environment.

Release flow

Integration repository — deploy a release version to remote environments.

In a traditional mono-repo, a release will be represented by a git tag. Since we work with multi-repo it is split and each repository can be on a different version, but the final picture (like we do on the mono-repo) can be represented on the integration repository which can point to any version of a microservice repository (Used usually to deploy to stage and production environments).

Every git tag that will be pushed to the master/main branch on the integration repository will be available to be deployed to the remote environments such as stage and prod by the following.

CI-CD based on dockers implementation description:

1. build (automatic) — A git tag on the integration repository will represent all the submodules commit ids (by the submodule reference) to define each micro-service version. Images for these commit ids are already made and wait to be deployed on the container registry since their development passed on develop flow. In kubernetes we use helm representing these images with these commit-ids that will be built and pushed with the git tag to a remote helm registry.
2. deploy (manual)— Images with the relevant commit ids will be pulled to the remote environment. In kubernetes we deploy by the helm from the helm registry to a remote environment.

Note: Every commit on the integration repository should be merged to master before it is being used on this flow. Otherwise, it will not be recognized.

Summary

• The Develop flow is based on gitflow with develop and master/main brancher. Changes are automatically deployed to the remote environments. It doesn’t require you to deploy a complete application as a composition of microservices and just to work on one microservice.
• Many companies that we worked with, requested the develop flow for their production as well based on master (because they don’t have dependencies between microservices and they don’t need benefits from the composition)
• The Release flow requires extra steps to create the composition description and manual triggering. However, it also gives you many advantages, such as granular version control and delivering the application with a specific microservice version.
• The huge advantage that we see in our Ci-Cd mechanism is that we mix these 2 approaches and have benefited from them. However, it is up to you to decide which approach you want to use for your product.

Credits:

These methodologies were developed for a long time, and they were fine-tuned based on real-life cases. I got great assistance with the thinking, implementation, writing this article, and more from my team which I would like to thank a lot. They are the best!

Sergey Zavadski, Andrei Gordienko, Lior Kesos, Tova Weiss, Lev Levenberg, Itay Wolfish, Israel David Dahan, Racheli Krakover, Sara Weber