A 360 Review on Continuous Integration, Delivery, and Deployment

The increasing amount of literature on Continuous Integration (CI), Continuous Delivery (CDE), and Continuous Deployment (CD) has stimulated the need and desire of secondary studies. Recently, there have been several reviews published on CI, CDE, and CD, mostly in Software Engineering journal. Whilst these reviews have made important contributions to the body of knowledge about CI, CDE, CD, there was no significant effort to perform an integrated review of these three practices areas, which are so much intertwined that it is critically important that the key pieces of work on these topics are reviewed in an integrated fashion. Such an integrated review is expected to provide the researchers and practitioners with a 360 degree view on the approaches, tools, and challenges in these areas. We have tried to address this need through a Systematic Literature Review (SLR), Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges, which has been recently published in IEEE Access. We are really excited to have this work done as it would fill important gaps left by other secondary studies on these important topics. Our review also critically comparing the existing reviews and our work that is an important part of progressing the state of the art in an emerging area. I’m copying/pasting the abstract for the readers’ interests as a teaser leading to a full read of the article. We also welcome comments.

Continuous practices, i.e., continuous integration, delivery, and deployment, are the software development industry practices that enable organizations to frequently and reliably release new features and products. With the increasing interest in the literature on continuous practices, it is important to systematically review and synthesize the approaches, tools, challenges, and practices reported for adopting and implementing continuous practices. This paper aimed at systematically reviewing the state of the art of continuous practices to classify approaches and tools, identify challenges and practices in this regard, and identify the gaps for future research. We used the systematic literature review method for reviewing the peer- reviewed papers on continuous practices published between 2004 and June 1, 2016. We applied the thematic analysis method for analyzing the data extracted from reviewing 69 papers selected using predefined criteria. We have identified 30 approaches and associated tools, which facilitate the implementation of continuous practices in the following ways: 1) reducing build and test time in continuous integration (CI); 2) increasing visibility and awareness on build and test results in CI; 3) supporting (semi-) automated continuous testing; 4) detecting violations, flaws, and faults in CI; 5) addressing security and scalability issues in deployment pipeline; and 6) improving dependability and reliability of deployment process. We have also determined a list of critical factors, such as testing (effort and time), team awareness and transparency, good design principles, customer, highly skilled and motivated team, application domain, and appropriate infrastructure that should be carefully considered when introducing continuous practices in a given organization. The majority of the reviewed papers were validation (34.7%) and evaluation (36.2%) research types. This paper also reveals that continuous practices have been successfully applied to both greenfield and maintenance projects. Continuous practices have become an important area of software engineering research and practice. While the reported approaches, tools, and practices are addressing a wide range of challenges, there are several challenges and gaps, which require future research work for improving the capturing and reporting of contextual information in the studies reporting different aspects of continuous practices; gaining a deep understanding of how software-intensive systems should be (re-) architected to support continuous practices; and addressing the lack of knowledge and tools for engineering processes of designing and running secure deployment pipelines.