Project Overview

JOURNAL #ONE [PRJ701]

Project Overview

Background

The approved project proposal is intended to showcase the required knowledge and expertise relating to a major in information systems and completion of a bachelor of information technology degree. The project scope is required to meet the standards of a level 7 course, whilst exhibiting the learning achievements thus far. The learning goals and outcomes will be tracked and documented throughout the project.

It is important that the project is planned, implemented and completed of sufficient quality and depth to not only meet the discussed requirements for completion of the course but to reflect the 45 credits assigned and produce a body of work that provides a material benefit to the company.

Summary

To use and deploy analytic engineering to produce clean data sets for end-users employed in key departments of the company, modelling data in a way that empowers them to answer questions to support the business operation and future growth plans.

Furthermore, personal goals will include developing an understanding and gaining further practical experience with cloud-based data-warehouses such as ‘Snowflake’ to make data processing and analytics faster and more affordable. In addition, there is an important research element to the project that will require investigating business intelligence (BI) tools that can enhance the ability of stakeholders to self serve the reporting function across business units.

The intention is to conduct any research, where possible, to improve personal understanding whilst also gaining an ability to contribute to the decision-making process within the company. The approach being taken is very much an agile one, as such, it is anticipated that the project may change and take on new responsibilities and direction as it progresses, and as further clarity is gained with regards to the products and services available, including any changing user requirements within each department.

Scope

The primary focus of the project is the technical, research and analytic aspects. This will not however restrict the scope simply to these areas and topics. The nature of IT and specifically data is such that areas not normally considered the purview of IT professionals, will be taken into consideration. The reason for this is unavoidable as the steps required to meet corporate decisions on data privacy and security will be carried out by IT professionals, therefore it is deemed necessary in relation to this project.

Data privacy and security considerations will include, but are not limited to, regulatory concerns such as the EU General Data Protection Regulations (GDPR) and California Consumer Privacy Act (CCPA), in addition to any security controls that form part of SOC 2 audits and ISO 27001 accreditation.

Approach

The following guidelines are a high-level approach to the project. The purpose is to offer direction for the project, however, they are not intended to act as a set of inflexible rules, it is expected that these will develop, change and grow with the project.

1- Getting Started

• Investigate products and services that will lead to a successful project outcome within the given time constraints.
• Gain an understanding of the current infrastructure and data storage systems, including the applications used to collect data stored by the company.
• Clarify and document the problem domain by analysing user requirements.
• Document the data lake strategy, clarify where data is stored and what is needed to answer use cases.
• Conduct research into problem-solving around data warehouse vs. streaming warehouse, specifically whether a modern data warehouse is a stream and how this research can be best deployed to meet the use case.
• Gain familiarity with SQL in a PostgreSQL environment.

2- Project Planning & Deliverables

A systems analysis and design study will be conducted that will include the collection and interpretation of information to aid the creation of data analytics that support the business requirements. Problems can be identified that might require the project objectives to be reconsidered where necessary.

Data mapping will take place across multiple platforms to create the database and tables necessary to meet reporting requirements, perform any required data cleansing and write queries to display data in a meaningful way, Extract, Transform and Load (ETL). Finally, the project deliverables are intended to produce options for report building and/or the implementation of a dashboard display of the data retrieved.

Consideration must be made to the security and privacy of data that is sensitive, an increasingly global business concern, particularly when handling data stored in a cloud environment. A wider organisational data mapping process may have to be undertaken to track and record the legitimacy of data collected by the various company systems and where it is stored geographically, which will have wider implications on the company’s ability to comply with the GDPR and CCPA.

Data must be encrypted without overtly increasing complexity, the database will need to be securely backed up to ensure its trustworthiness (Li et al., 2010). The on-demand benefits of cloud database services should not be adversely affected through the implementation of security principles comprising data integrity, confidentiality, and availability, (Sun et al., 2014).

3- Report Deliverables

Reporting deliverables must capture the needs of the following business departments as they relate to company data:

• Production
• Product
• Marketing
• Sales
• Tech

Consideration will be made to:

• Marketing, Finance and Sales data
• Site usage data – website and cloud platform
• Events based data to drive automated actions
• Orders
• Customers & Studio Usage
• Recurring customers
• Assets

Production reporting will be prioritised to meet the demands of the business and split as follows:

• High
• Medium
• Low

Cloud-based Data Warehouse. (n.d.). SnapLogic. Retrieved August 1, 2021, from https://www.snaplogic.com/glossary/cloud-based-data-warehouse

Giulio, C. D., Sprabery, R., Kamhoua, C., Kwiat, K., Campbell, R., & Bashir, M. N. (2017). IT Security and Privacy Standards in Comparison: Improving FedRAMP Authorization for Cloud Service Providers. 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), 1090–1099. https://doi.org/10.1109/CCGRID.2017.137

Markel, M. (2006). Safe harbor and privacy protection: A looming issue for IT professionals. IEEE Transactions on Professional Communication, 49(1), 1–11. https://doi.org/10.1109/TPC.2006.870462

What Are Business Intelligence (BI) Tools | Microsoft Azure. (n.d.). Retrieved August 1, 2021, from https://azure.microsoft.com/en-us/overview/what-are-business-intelligence-tools/

What every IT pro needs to know about privacy. (n.d.). Retrieved August 1, 2021, from http://technologydecisions.com.au/content/information-technology-professionals-association/article/what-every-it-pro-needs-to-know-about-privacy-202184559

What is Data Engineering? - Responsibilities and Tools. (n.d.). Retrieved August 1, 2021, from https://www.dremio.com/data-lake/data-engineering/