This post describes FTS SDLC and how does it integrates other important flows like Git and JIRA ones.
Delivery Platform & SDLC
We will follow this structure:
One of the most important processes for software companies is their Software Delivery Life Cycle because it will force methodologies, technologies to be used, hierarchies and most important, human relationships.
SDLC says a lot about software companies, it talks about the trust they have in their engineers, about the comfortable they are about technology, it reflects their adaptability and most important WHO rules the software the company creates.
In FTS, the Delivery Platform is the one in command of ensuring the SDLC strictly followed and to integrate tools like JIRA critical for management and customer information.
The challenges we face for SDLC definition and implementation in the Delivery Platform are related to:
Our delivery flow includes multiple Cloud Provider technologies as you can see in the previous post Delivery Platform and Automatic Creation of API Endpoints, it includes coordinating integration with third party tool APIs like Gitlab, Jenkins, JIRA, Slack…
SDLC is all about orchestration, when to do what, how to fail, who has to be informed, how to create reports and the proper timing of these topics will be critical in the SDLC’s success.
Evangelization & Trust in Automation
As humans, it is natural to keep coming back to what we are use to do, to what it is comfortable and not threatening. Overcoming that and convincing engineers and managers to change the SDLC takes time, data and planning.
Another critical topic is trust in automation, because it is difficult to assume the role of engineers and start thinking in ways of delivering a solution from inception to deployment in production. And when we don’t want to be engineers, we start relying in manual procedures such QA in human testing, manual software packaging, manual deployments and this is because we can blame others instead of taking responsibility
- Describe FTS Software Delivery Life Cycle
- Describe FTS SDLC integration with Git flow
- Describe FTS SDLC integration with JIRA flow
- Detail Delivery Platform mechanisms to ensure SDLC is followed
FTS SDLC follows continuous delivery procedures to ensure quick delivery of new features to live environments. In other words, we want to decrease fear to change by releasing small features in a continuous way rather than delivering at the end of each sprint.
We follow TDD methodology so that our CD can be trustworthy. The meaning of this is that we perform tests definition code review right at the moment of the requirements gathering. Once that review is completed and a success, the engineer can continue to develop new software pieces.
The following figure describes our CD flow:
If any of these steps fail, the delivery pipeline will be aborted and notifications will be sent so that the affected team members can work over the problem.
Continuous Integration Stage
Assuming the source code is ready, there will be a merge request that will trigger our Jenkins continuous integration pipeline over the feature branch (please refer to the Git flow section to know more about repository branches and merges).
There is a build step that ensures the code compiles as expected and unit and integration tests will run. The Delivery Platform runs static code analysis in SonarQube warrantying coverage, bugs, vulnerabilities, repeated code and code debt are in the proper thresholds.
After that, Jenkins generates the snapshot artifact and uploaded it to the Nexus repository.
This stage starts by merging code to master branch with the new feature and running tests and quality gates before pushing. This warrantees the alignment between feature branches and master branch is ok. Then, as in the following picture, Jenkins pushes the generated artifacts and/or docker images.
Of course, this doesn’t end here, beta stage requires functional tests to be run over a production-like environment. In the case of some of the software we develop they will be pods deployed in Kubernetes clusters, in other cases they will be Notebooks to be deployed in Azure Databricks. It can be anything and the Delivery Platform includes a set of templates to operate with multiple technologies so the delivery flow is the same for each of them.
Functional tests technology will depend on the actual software, e.g. Angular applications will include Protractor, HTTP endpoints will run JMeter tests, but all of them generate XML Junit format tests results so Jenkins can read them and stop the pipeline if something fails.
If a new feature reaches Candidate stage, it means that at least, from the functional perspective the code its good and we only need to validate it from the performance and the security perspective. This becomes critical when we talk about components of the Central API because poor performance there means multiple customers affected so the requirements here are high. Low latency for get operations, stability after thousands of requests, components to scale if the load grows over defined thresholds…
Regarding tools, Performance tests for HTTP endpoints will rely in JMeter run by Jenkins spawning a set agents so it can scale as if thousands of users were accessing the services.
Finally, ZAP tool is run by Jenkins so that we can identify OWASP vulnerabilities in our HTTP endpoints getting reports as the following:
Live stage is the final step to make a functionality available to the public and its just about deploying the candidate artifacts in live clusters and update the data structures if needed.
SDLC relies in Delivery Platform to make this flow to be followed. As we can see, feature branches will be triggered to merge in develop for the Continuous Integration Stage. If everything goes well, the develop will be merged to master in Beta stage and then, the same code tested in beta, candidate and deployed in live.
Release fixes are done branching from master as consequence of missing quality gates in master. Another case can be failing functional tests that cannot be reproduced in feature branches or stress/security check.
Finally, one of the most important aspects of our delivery process is to include the tool that links resources assignment, tasks, requirement cards with the actual work done in the code and tests.
Typically, managers and customers will look into JIRA to get a view about the project status and progress. For that reason, the Delivery Platform integrates with JIRA so the status in the cards match the actual results of the CD pipelines.
The following statuses are included in the Jira Workflow [1 to 4 are actually the stages pre-continuous delivery]:
- Initial Step: New JIRA Card
- First Status: Open
- Test in Progress
Task ready for the implementation of the automatic teststhat will be executed after the deployment or execution of the final component.
- Tests Ready for Review
- Ready for Implementation – Jenkins Continuous Integration Pipeline
Once the code review for tests is met the development engineer starts the functional implementation of the software component meeting the previously written tests in the previous phases
- Ready for Beta – Jenkins Beta Pipeline
- Ready for Candidate – Jenkins Candidate Pipeline
- Ready for Production – Jenkins Live Pipeline
This way, right after the development of a functionality has started, the Delivery Platform will take care of progressing JIRA cards linked to the tests and requirements described and the actual results of the pipelines.
In this post, we have described FTS SDLC linking major elements of the company such as Delivery Platform, JIRA and Gitflow.
The work done to get all of this ready has been complex since it required to get together many topics at the same time and agreeing about complex topics and procedures such as automated testing, requirements for delivery, requirements for performance and security and many more aspects.