Agile Data Logo

Agile Data Architecture

Follow @scottwambler on Twitter!

Data architecture is the foundation of a data strategy that supports your organization's goals and priorities. Agile data architecture does so in a collaborative and evolutionary (iterative and incremental) manner. The goal of this article is to explore what agile data architecture is and why it is important. Other articles in this series present heuristics for successful data architecture.

This article explores the following topics:

  1. What is agile data architecture?
  2. Why agile data architecture?
  3. Related resources

1. What is Data Architecture?


This is an important question because it depends on context. In Agile Data, as we show in Figure 1, we recognize that data architecture refers to three interrelated concepts:

  1. The actualization of the data aspects of an organization or solution. This includes data assets such as databases and files, data transport, and data technologies. It is important to note that data architecture addresses a portion of your overall architecture. For example, the data architecture of an organization is a subset of your overall enterprise architecture whereas the data architecture of a solution (a system or application) is a subset of its overall solution architecture. With an agile approach, your way of working (WoW) must be tailored to be fit-for-purpose for the situation that you face - agilists adjust to their context. See Agile Data Architecture in Context for greater details.
  2. The description of the implementation. Data architecture is described in logical (what and why), physical (how and where), and organizational (who, when, and value) terms. Data architectures can be described by models, documents, data conventions, executable tests, and ideally reference architectures in the form of working examples. These artifacts will address a range of architectural concerns such as storage, security, flow, and more. Architectural descriptions may address the current architecture, proposed or desired future architecture, and even your past architecture (often captured for regulatory reasons). See potential data architecture artifacts. With an agile approach these artifacts are concise, fit-for-purpose, sufficient, and ideally executable. To do so we adopt strategies from the Agile Modeling (AM) method.
  3. The activities around data architecture. This includes exploring, describing, building, supporting and evolving the implementation and description. As you would imagine, the Agile Data (AD) method describes many effective strategies for doing exactly these things in a collaborative and evolutionary manner. See the description of the Agile data architect role for details.

Figure 1. What is data architecture?

Data architecture definition

2. Why Agile Data Architecture?

Agile data architecture enables:

  1. A roadmap for effectively working with data. Data architecture creates a better understanding of how your organization's systems ingest, process, and produce data. With an agile approach, this is done so collaboratively via a fit-for-purpose approach.
  2. Data to become a shared asset. Clean data architecture eliminates departmental data silos to provide a complete view of your organization's data. This provides an opportunity for a wider range of people to work with your existing data and hopefully not create yet another copy of it.
  3. People to make data-driven decisions. For people to make effective decisions, they need ready access to current data that pertains to that decisions. Clean data architectures provide interfaces that make it easy for people to consume data using tools fit for their jobs. This includes data warehouse (DW) and business intelligence (BI) platforms that deliver useful information and insights.
  4. System evolution. Clean data architectures are extensible and easily evolved to meet new business needs. They do this by indicating where data is stored and how it flows through your enterprise, and by providing consistent guidance for doing so.
  5. Data security. Data security is one aspect, in many cases the most critical aspect, of your organization's overall security strategy. Clean data architecture addresses data security issues such as access, encryption, integrity monitoring, and more.
  6. Technical quality. Clean data architecture helps improve data quality by reducing the number of places source data is stored, streamlining data integration, and streamlining data transport.
  7. Agile data management. A clean data architecture is a crucial part of the data management process, the overall goal of which is to ensure that data is used properly and meets business needs for information.
  8. Lean data governance. Data governance guides and monitors the collection, storage, arrangement, integration, and use of data in organizations, the goal of which is to ensure that your data is accurate, consistent, and used appropriately.

3. Other Articles in This Series

This article is the first in a series. The other installments in this series are:

  1. Critical success factors in agile data architecture
  2. Agile data architecture in context
  3. The agile data architecture process
  4. Clean data architecture
  5. Potential data architecture artifacts
  6. The agile data architect

4. Related Resources