Agile Data

Data and the Unified Process (UP)

Follow @scottwambler on Twitter!

The Rational Unified Process (RUP), OpenUP, Agile Unified Process (AUP), and the Enterprise Unified Process (EUP), are instantiations of the Unified Process (UP). A fairly common question that many organizations face is "how do data professionals fit into a RUP project?" Very well, actually, when the data professionals choose to modify their approach to meet the realities of the RUP. The RUP takes an evolutionary (iterative and incremental) approach to development. Unfortunately many data professionals prefer a more traditional, serial approach to development where comprehensive data models are typically developed very early in a project, reviewed and accepted, and then baselines and put under change control.  Although this is very convenient for the data professionals it puts projects at risk because it hampers their ability to react to change. Luckily, as this site amply shows, data professionals can in fact work in an evolutionary manner if they choose to do so. 

To understand why there is an issue, you need to understand the way that the RUP works. The RUP lifecycle is based on four consecutive phases (the EUP adds two more so as to cover the entire system lifecycle, not just development).  The fact that there are four serial phases can often motivate traditional developers to believe that the RUP is similar to the serial processes that they are familiar to. This isn't the case. Instead, I like to say that the RUP is iterative in the small, serial in the large, delivering incremental releases over time.  Let's look at each phase in order, discussing the potential role for data professionals:

  1. During the Inception phase the goal is to identify high-level requirements for the system so that the scope may be identified and project funding obtained. Little, if any, data modeling is performed at this point although some conceptual modeling may occur. Detailed data models are clearly not needed at this point, nor should you be sidelined by some needless quest to ascertain the "one truth".
  2. During the Elaboration phase the goals are to flesh out the requirements to the 80% level, to envision an architecture for the system, and to prove that the architecture works via an end-to-end technical prototype. Enterprise architects may be involved with this effort, and one of several issues that they are likely to address is data architecture. See Agile Enterprise Architecture for some thoughts on how architects can work in an agile manner. Once again, detailed data models are not needed at this point. 
  3. During the Construction phase the goal is to build a working system that is ready to be put into production. It is at this point that data professionals are likely to be needed. In the RUP development proceeds in iterations, typically several weeks in length, during which a subset of the requirements are implemented. For example a 6 month construction phase might be organized into 6 iterations of four weeks each, or even 12 iterations of two weeks each. During each iteration your analysis models, design models, source code, and test suite are fleshed out to implement that iteration's requirements. In other words your development artifacts, including your data-oriented artifacts, evolve over time. The implication is that data professionals need to be prepared to grow their models, including their database schema over time. Furthermore they may discover that they may need to refactor their schema to reflect the needs of new requirements. If your system needs to interface with existing legacy systems, and it often does, data professionals will likely be needed to be involved with legacy integration efforts. Data models are very likely needed, but they're not needed up front.
  4. During the Transition phase the goal is to system and user test the system to ensure that it is ready to be deployed into production, to make needed corrections, and to actually deploy the system. The need for data professionals decreases during this phase.

So how can data professionals involved with their their first RUP project learn to work in an evolutionary manner via the philosophies and agile database techniques of the Agile Data method? My advice is to do the following:

  1. Read The Agile Data Method to learn the philosophical underpinnings of the evolutionary database techniques.  
  2. Read The roles on Agile Data Projects to learn about the two roles that they are likely to fall into, Agile DBA or Enterprise Administrator, and how they can effectively interact with evolutionary teams.
  3. Read Evolutionary Development to gain an overview of basic development techniques for evolutionary database development. Repeat after me: comprehensive data models are not required up front, comprehensive data models are not required up front, comprehensive data models are not required up front, ...
  4. Read Agile Model Driven Development (AMDD), posted at, to learn about how it is in fact possible to model in an evolutionary and documentation-light manner, instead of the traditional serial and documentation heavy one that many data professionals are accustomed to.
  5. Read The Process of Database Refactoring to learn how to evolve a database schema after the fact.  It is a myth that you must have the database schema in place, or at least mostly in place, before coding begins.
  6. Read Why Data Models Don't Drive Object Models (And Vice Versa), The Object-Relational Impedance Mismatch, and Mapping 101 to overcome any misconceptions about how to effectively use object and relational technologies together.
  7. Read the rest of the articles at this site.
Note that there is an assumption in the writings at this site that project teams are taking an agile approach to software development.  This may not be completely true of a RUP project team, although hopefully should be, but what is true is that the team will be taking at least an evolutionary approach. Therefore the techniques described at this site are still applicable although may be a little harder to accomplish in environments where the RUP has been instantiated in a non-agile manner.