Version control procedures

Introduction

The purpose of this document is to provide guidelines and technical steps involved in various actions related to PunchPlatform configuration management.

The overall principles of PunchPlatform configuration management are explained in Version Management, to be read first.

This document provides some insight on the use of git tool, but does not replace the need for reading and understanding git concepts and commands, as provided by public git documentation (see ref::https://git-scm.com/documentation)

Git concepts

git tags:

Tags are used to mark/identify a past status in a git repository. In PunchPlatform configuration management, two kind of tags are used :

  • release version tag ( ‘e.g. ‘ AHMAD_V2_1_2), that are assigned once, and never change
  • major releases floating tag ( ‘.e.g.’ ‘AHMAD’ ), that are kept updated to always provide access to the latest version of the release (i.e. ‘AHMAD’ will allow retrieving the same thing as ‘AHMAD_V2_1_2’, if this is the last stable version existing for release Ahmad.)

Note that a tag stores not only the current state of all files in the filesystem, but also the state of the list of all successive changes that have been made previously (the ‘git log’) to arrive at this point.

git branchs:

A branch is a working space that is used by one or several users to commit new changes. It can be seen as a ‘floating tag’ that is automatically updated each time one of the team members pushes a modification to it. This way, one of the team member can easily retrieve the “last status” of this working space.

In PunchPlatform team daily work, the following branches are used :

  • major release common development branch. (e.g. ‘ahmad-dev’). These branches are used for integrating developed features, and running integration tests. They normally contain all developed features, as soon as the developer has satisfied basic stability criteria (unitary tests, documentation update, non-regression test on a standalone environment).
  • feature development branch (e.g. ‘feature-es-extractor’). These branches are temporary work space for one or more developers working on the same feature. There is no stability constraint on this versions, except if mutually decided by the co-workers using the branch
  • local workspace. IMPORTANT : Because of its distributed nature, any developer work space is in itself an ‘implicit’ branch separated from the workspaces of other developer computers (even tough it has the same git branch name as on other computers). To allow merging of multiple people work, the team git server is used as a storage space into which common work reference is stored. Each developer has to merge/test on his computer the work of the team with his new ‘local’ changes, and then update the team git server with the result.

git merge:

Action of bringing into the working git branch, some changes coming from an other branch, or from any repository state (tag) that includes changes not yet imported in your working branch state.

When merging, git tries to find which “common” version existed in the past (git log) of YOUR branch and the past of the imported branch or tag. Then git tries to determine if changes made since this command on your branch and the imported context can be merged automatically (for example, changes made on different sections of the same file). If not, the user will have to manually analyze this changes to decide what the final ‘merged’ version of the file should look like.

Main configuration procedures HOW TO’s

Developing a feature on multiple major releases

Example context

I have to develop a feature for punchplatform. I am a member of PunchPlatform development team allowed to integrate features in the coming major release. Two major releases are currently maintained with non-frozen feature scope : Ahmad, and Art (which comes after Ahmad). The team designer have agreed that this feature can and must be developed so as to be compatible with both releases.

Procedure

  • Clone the development repositories :

    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-core
    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-log-resources
    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-packaging
    
  • Request last state of common development branch for Ahmad major release (to be done in each of the repositories):

    git checkout ahmad-dev
    
  • Create a feature branch to work independently on common branch, in order to avoid its unstabilization

    git checkout -b feature-my-new-feature
    
  • (Repeatedly) do some changes on files, and commit and secure your changes by storing them on the team git server

    pp-core/poms_housekeeper --snapshot <path to my .pom> # each time you change some file that mean the artifact associated to the maven .pom file is changed as compared to last release version
    
    git add <files....>
    git commit -m "<Comment including the reference to the #<number> of the jira (or formerly redmine) issues associated to the feature or fix>"
    git push origin feature-my-new-feature
    
  • After having reached a stable status of the feature (unit tests OK, updated product documentation, some non-regression testing done at standalone deployment level at least), integrate your feature inside the common development branch of the major release

    git checkout ahmad-dev
    git pull origin ahmad-dev    # to retrieve last changes of the team from the team git server
    git merge feature-my-new-feature
    git mergetool                # if some merge could not be made automatically by git
    git commit
    # And, after retesting stability of the integrated version
    git push origin ahmad-dev
    
  • Prepare for merge of the new Ahmad development into Art major release. this needs to insure all merged artifacts are NON-SNAPSHOT to ensure unicity of maven artifacts version in a multiple-releases context :

    pp-core/poms_housekeeper.sh --release    # To remove 'snapshot' status marker to all maven artifacts versions
    pp-core/poms_housekeeper.sh --check
    
    vi pp-core/RELEASENOTE.rst # ensure that all issues that have been merged into Ahmad-dev are listed in the description rubric of the currently being integrated ahmad version
    cd pp-core
    git add RELEASENOTE.rst
    cd ..
    
    for repository in pp-core pp-log-resources pp-packaging ; do
            cd ${repository}
    
            git add ./\*.pom.xml     # to add all modified maven .pom files
            git commit -m 'Releasing artifacts'
            git push origin ahmad-dev
    
            cd ..
    done
    
  • Merge into Art common development branch TO BE REPEATED for each repository (pp-core, pp-log-resources, pp-packaging) :

    cd pp-core
    git fetch origin art-dev
    git fetch origin ahmad-dev
    git checkout art-dev
    git merge ahmad-dev
    git mergetool                   # here, you may have to solve manually conflicts
    
    # EDIT and update all .pom files of artifacts that have been modified by merge, in order to increment
    # version of artifact, and report development status by appending '-SNAPSHOT' to the maven version
    # This can be done recursively by the script '../pp-core/poms_housekeeper --middle-increment --snapshot <path to pom>' so as not to forget any indirect modification of dependent PunchPlatform artifacts.
    
  • And, after retesting stability of the integrated versionCheck stability of locally integrated version, commit merged major release to the team git server : TO BE REPEATED for each repository (pp-core, pp-log-resources, pp-packaging) :

    git commit
    git push origin art-dev
    

Developing a non-urgent maintainance fix on a production-deployed major release version

Example context

A bug has been diagnosed on version Ahmad V2.1.2, which is deployed at a customer site The last stable version of Ahmad is V2.1.3. No migration issue is identified in release notes preventing migration from V2.1.2 to V2.1.3 requiring any change in customer site configuration, so PunchPlatform maintainance team decides to build the fix to be compatible with the major release Ahmad. The bug is not of critical nature, so that it looks possible to release a new version of Ahmad including both the required fix, and the new features or other fixes that have been integrated in the ahmad development branch.

Procedure

  • Clone the development repositories :

    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-core
    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-log-resources
    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-packaging
    
  • Request production-site state of common development branch for Ahmad major release (to be done in each of the repositories):

    git fetch origin --tags   # to retrieve all tags definition from reference git server
    git checkout AHMAD_V2_1_2  # use git tag --list if you need to check existing version tags
    
  • Create a ‘fix development branch’, first to reproduce the bug in a site-identical version, then to develop fix

    git checkout -b feature-my-ahmad-bug-fix
    
  • Generate the version and reproduce the bug with it

  • Develop the bug fix in yout fixing branch by (repeatedly) do some changes on files, and commit and secure your changes by storing them on the team git server

    git add <files....>
    git commit -m "<Comment including the reference to the #<number> of the jira (or formerly redmine) issues associated to the bug>"
    git push origin feature-my-ahmad-bug-fix
    
  • After checking the bug if fixed , and having reached a stable status of the fix (unit tests OK, updated product documentation, some non-regression testing done at standalone deployment level at least), integrate your feature inside the common development branch of the major release as decided by the maintainance team

    git fetch origin ahmad-dev  # to retrieve last changes of the team from the team git server
    git checkout ahmad-dev
    git merge feature-my-ahmad-bug-fix
    git mergetool                # if some merge could not be made automatically by git
    git commit
    # And, after retesting stability of the integrated version
    git push origin ahmad-dev
    
  • Follow Releasing major version for delivery procedure to produce a new ahmad release version and provide the associated distribution packages to the customer site for deployment

Developing an URGENT maintainance fix on a production-deployed major release version

Example context

The context is the same as for Developing a non-urgent maintainance fix on a production-deployed major release version, except that the team prefers to release a new version of Ahmad major release without taking into account the new ahmad features and other fixes that had already been prepared in ahmad-dev common development branch.

The fix will of course have to be also integrated in subsequent Ahmad updates that will include these new features…

Procedure

  • Clone the development repositories :

    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-core
    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-log-resources
    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-packaging
    
  • Request production-site state of common development branch for Ahmad major release (to be done in each of the repositories):

    git fetch origin --tags   # to retrieve all tags definition from reference git server
    git checkout AHMAD_V2_1_2  # use git tag --list if you need to check existing version tags
    
  • Create a ‘fix development branch’, first to reproduce the bug in a site-identical version, then to develop fix

    git checkout -b feature-my-ahmad-bug-fix
    
  • Generate the version and reproduce the bug with it

  • Develop the bug fix in yout fixing branch by (repeatedly) do some changes on files, and commit and secure your changes by storing them on the team git server

    git add <files....>
    git commit -m "<Comment including the reference to the #<number> of the jira (or formerly redmine) issues associated to the bug>"
    git push origin feature-my-ahmad-bug-fix
    
  • After checking the bug if fixed , and having reached a stable status of the fix (unit tests OK, updated product documentation, some non-regression testing done at standalone deployment level at least), follow Releasing major version for delivery procedure to produce a new ahmad release version and provide the associated distribution packages to the customer site for deployment BUT package, test tag the AHMAD_V2.1.3 fixed release directly from the feature-my-ahmad-bug-fix branch, instead of doing it from the ahmad-dev branch (as is described in the generic releasing major version for delivery procedure )

  • Integrate the new fix in the Ahmad release versions yet to come (to be repeated in all pp-core, pp-log-resources and pp-packaging git repositories)

    git fetch origin ahmad-dev  # to retrieve last changes of the team from the team git server
    git checkout ahmad-dev
    git merge feature-my-ahmad-bug-fix
    git mergetool                # if some merge could not be made automatically by git
    
    
    ## edit maven .pom files and RELEASENOTE.rst to increase the version number of PunchPlatform packaging artifacts to match next version to come (Ahmad V.2.1.5-SNAPSHOT).
    git add RELEASENOTE.rst # this, of course just in pp-core
    git add ./\*.pom.xml
    git commit
    # And, after retesting stability of the integrated version
    git push origin ahmad-dev
    
  • The team can go on developing the next version of ahmad release…

Developing my own additions to the PunchPlatform product, without being a team member

Example context

I want to prototype/develop a feature in addition to last PunchPlatform stable release I am (not yet?) a member of PunchPlatform development team allowed to integrate features in the coming major release. I do not want to be bothered by new features being integrated by the main PunchPlatform development team for the coming next major release version.

Procedure

  • Clone the development repositories :

    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-core
    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-log-resources
    git clone https://git.gp.sef.cloudomc.ts-agora.fr/punchplatform/pp-packaging
    
  • Request last state of stable Ahmad major release (to be done in each of the repositories):

    git checkout ahmad  # This tag retrieves last stable,maintained version of the ahmad major release.
    # Alternatively, you can do 'git checkout ahmad-stable', but please refrain from comitting changes in this branch : it is dedicated to holding only official PunchPlatform distributions (see `Releasing major version for delivery`_)
    
    # At this step, you have everything to build PunchPlatform and run it !
    
  • Create my personal work branch, to avoid mixing my development with someone else…

    git checkout -b my-own-work-space-branch
    
  • (Repeatedly) do whatever you want to code, test… and commit it on PunchPlatform team git server to secure your development and ease its integration in future releases of punchplatform

    git push origin my-own-work-space-branch
    
  • call us so that we know about your work, and discuss integrating it in next release !

Releasing major version for delivery (or internal version, for integration purpose)

Example context

The team has integrated a set of features and fixes in a common development branch for one of the ptunchplatform major release (Ahmad) We want to package and deliver the PunchPlatform distribution to the outside world, and to maintain it afterwards. This new version will be called Ahmad V2.1.6

Procedure

  • After having reached a stable status of the development branch (unit tests OK, feature tests OK, updated product documentation, migration procedure testes, non-regression testing done at standalone deployment level at least),

        git fetch origin ahmad-dev
        git checkout ahmad-dev
    
        pp-core/poms_housekeeper.sh --snapshots-check <PREVIOUS_RELEASE_TAG> # this allows to detect any artifact which version should have been snapshotted because of file changes since the last version
    (use  pp-core/poms_housekeeper.sh --snapshot if there are some inconsistencies)
    
        pp-core/poms_housekeeper.sh --release    # To remove 'snapshot' status marker to all maven artifacts versions. This needs to be done once only because it will apply to all the PunchPlatform git repositories.
    
        pp-core/poms_housekeeper.sh --check
    
        vi pp-core/RELEASENOTE.rst # ensure that release note includes all information about the version (including new features numbers, fixed bugs numbers, known bugs numbers, migration documentation reference)
    
        cd pp-core
        git add RELEASENOTE.rst
        cd ..
    
        for repository in pp-core pp-log-resources pp-packaging ; do
                cd ${repository}
    
                git add ./\*.pom.xml     # to add all modified maven .pom files
                git commit -m 'Releasing artifacts'
                git push origin ahmad-dev
    
                cd ..
        done
    
  • REBUILD the version (this IMPORTANT step will genenerate among other things the PDF documentation, to ensure it can be built, because it is deactivated when building a SNAPSHOT version)

  • Proceed to validation tests on this rebuilt version (standalone redeployment, integration tests…)

  • Tag this version as an official one ( to be repeated in each of the pp repositories) - if the version is to be for internal use only (at least temporarily) then the tag should be AHMAD_INTERNAL_V2_1_6 instead :

    git tag "AHMAD_V2_1_6"
    git push origin AHMAD_V2_1_6 --tags
    
  • Tag this version as the last stable one (in each of th pp repositories)

    git tag "AHMAD" --force    #This updates the "AHMAD" tag to register this new version as the last
    git push origin AHMAD_V2_1_6 --tags
    
  • (Alternate to (re)defining floating tag “AHMAD”) Update the main branch of the major release, that only contains official stable versions of the release :

    git fetch origin ahmad-stable
    git checkout ahmad-stable
    git merge AHMAD_V2_1_6
    git push origin ahmad-stable
    
  • Ensure you have rebuild locally on your station all the artefacts, then deploy new release artefacts to the release repository :

    pp-core/poms.housekeeper.sh --deploy-from <previous release tag>
    

    where <previous release tag> identifies the last deployed release artefacts bundle (for example : AHMAD_INTERNAL_V2_1_5). This will identify all maven artifacts that have increased their version number between the previous release and the new one, and will deploy the release artifacts to the LMC Software Engineering “Releases” repository

  • Publish the development documentation :

    connect to jenkins builder, and run the “dev-doc-publishing” job, :

    gittag = AHMAD_V2_1_6
    productbranch = ahmad
    
  • Publish the product documentation and download links (NOT FOR INTERNAL RELEASES) :

    connect to jenkins builder, and run the “product-doc-publishing” job, :

    gittag = AHMAD_V2_1_6
    productbranch = ahmad
    

    This will publish the distribution documentation and packaging URLs in the PunchPlatform documentation site.

  • IMPORTANT : re-snapshot the development repositories to avoid accidental local regeneration of new artifacts content with the same version number :
    cd pp-core
    ./poms_housekeeper.sh --snapshot punchplatform-doc/pom.xml
    ./poms_housekeeper.sh --snapshot ../pp-log-resources/pom.xml
    
    # EDIT RELEASE NOTE to create a new "-SNAPSHOT" version section a top of document
    
    # And for each of the repositories :
    git add .
    git commit -m "snapshotting for next release"
    git push origin art-dev
    
  • Manage your tickets in your ticketing tool: close tickets, create the new version

Useful git tricks for overall CM

How to know what files were altered during the automatic merge I just did ?

Purpose

I have issued a git merge myOtherBranch and everything went “automatic”. I would like to know what was changed in my branch by the merge.