D2D CRC Project on Reference Architecture for Defence Big Data Systems


, ,


Big Data systems (i.e., data-intensive applications) have become one of the key priority areas for all sorts of organizations (i.e., private or public). Big Data is usually characterized by huge amount of diverse set of data that cannot be easily (or timely) processed using the conventional data processing approaches and technologies. For example, the data coming from 9100 tweets per second or the data emitted from the jet engines of the aircraft of the 2500 flights that cross Atlantic every day.  Data to Decisions (D2D), a new initiative under the Cooperative Research Centres Program, is aimed at inventing new and innovative techniques and technologies for supporting several functions of Big Data systems such as data capture and storage, data transmission, data curation, data analysis, and data visualization. One of the first projects of D2D CRC is designing a Reference Architecture (RA) for Defence Big Data Systems. Whilst there are already several dozens of references architectures for big data systems, one can question the needs of such an effort. A domain specific reference architectures for big data systems are expected to play a key roles in the design, development, and evolution of big data systems in different domains. one of the key reasons of having reference architectures is that the inventions and innovations for leveraging Big Data opportunities and/or addressing Big Data challenges needs to be harnessed (or productized) into software intensive systems (or services) that need to be supported by robust and reliable architectures and frameworks. Big Data systems in a given domain can have architectures that share similarities but also demonstrate differences, it is important to have a Reference Architecture (RA) that can abstract the peculiarities of architectures for individual systems in a particular domain. It is quite common practice to design and validate domain- and/or technology specific reference architectures for providing an overall blueprint of the functionality that is needed in a given domain (e.g., Defence), how the functionality can be divided, and how the required information flows among different components of a system. I’m very excited to be involved in the Working Group set up for Reference Architecture for Defence Big Data Systems. Our team’s extensive knowledge and experience of designing and evaluating architectures will make valuable contribution to support the process of designing and evaluating a reference architecture for Big Data applications. As the project progress and there is no classified information, I’ll be sharing the process and outcomes from the project for sharing knowledge and helping the interested readers to learn and apply from this project’s efforts. Of course with due acknowledgement to the D2D CRC.