Data and the RUP AgileData.org: Techniques for Disciplined Agile Database Development
Home  |  Agile DBAs  |  Developers  |  Enterprise Architects  |  Enterprise Administrators  |  Best Practices  |  Agility@Scale Blog  |  Announcements  |  Contact Us

The Rational Unified Process (RUP), OpenUP, Agile Unified Process (AUP), and the Enterprise Unified Process (EUP), are instantiations of the Unified Process (UP).  A fairly common question that many organizations face is "how do data professionals fit into a RUP project?"  Very well, actually, when the data professionals choose to modify their approach to meet the realities of the RUP.  The RUP takes an evolutionary (iterative and incremental) approach to development. Unfortunately many data professionals prefer a more traditional, serial approach to development where comprehensive data models are typically developed very early in a project, reviewed and accepted, and then baselines and put under change control.  Although this is very convenient for the data professionals it puts projects at risk because it hampers their ability to react to change.  Luckily, as this site amply shows, data professionals can in fact work in an evolutionary manner if they choose to do so.

To understand why there is an issue, you need to understand the way that the RUP works.  The RUP lifecycle is based on four consecutive phases (the EUP adds two more so as to cover the entire system lifecycle, not just development).  The fact that there are four serial phases can often motivate traditional developers to believe that the RUP is similar to the serial processes that they are familiar to.  This isn't the case.  Instead, I like to say that the RUP is iterative in the small, serial in the large, delivering incremental releases over time.  Let's look at each phase in order, discussing the potential role for data professionals:

1. During the Inception phase the goal is to identify high-level requirements for the system so that the scope may be identified and project funding obtained.  Little, if any, data modeling is performed at this point although some conceptual modeling may occur.  Detailed data models are clearly not needed at this point, nor should you be sidelined by some needless quest to ascertain the "one truth".
2. During the Elaboration phase the goals are to flesh out the requirements to the 80% level, to envision an architecture for the system, and to prove that the architecture works via an end-to-end technical prototype.  Enterprise architects may be involved with this effort, and one of several issues that they are likely to address is data architecture.  See Agile Enterprise Architecture for some thoughts on how architects can work in an agile manner.  Once again, detailed data models are not needed at this point. 
3. During the Construction phase the goal is to build a working system that is ready to be put into production.  It is at this point that data professionals are likely to be needed.  In the RUP development proceeds in iterations, typically several weeks in length, during which a subset of the requirements are implemented.  For example a 6 month construction phase might be organized into 6 iterations of four weeks each, or even 12 iterations of two weeks each.  During each iteration your analysis models, design models, source code, and test suite are fleshed out to implement that iteration's requirements.  In other words your development artifacts, including your data-oriented artifacts, evolve over time.  The implication is that data professionals need to be prepared to grow their models, including their database schema over time. Furthermore they may discover that they may need to refactor their schema to reflect the needs of new requirements.  If your system needs to interface with existing legacy systems, and it often does, data professionals will likely be needed to be involved with legacy integration efforts. Data models are very likely needed, but they're not needed up front.
4. During the Transition phase the goal is to system and user test the system to ensure that it is ready to be deployed into production, to make needed corrections, and to actually deploy the system.  The need for data professionals decreases during this phase.

So how can data professionals involved with their their first RUP project learn to work in an evolutionary manner via the philosophies and agile database techniques of the Agile Data method?  My advice is to do the following:

1. Read The Agile Data Method to learn the philosophical underpinnings of the evolutionary database techniques.  
2. Read The roles on Agile Data Projects to learn about the two roles that they are likely to fall into, Agile DBA or Enterprise Administrator, and how they can effectively interact with evolutionary teams.
3. Read Evolutionary Development to gain an overview of basic development techniques for evolutionary database development. Repeat after me: comprehensive data models are not required up front, comprehensive data models are not required up front, comprehensive data models are not required up front, ...
4. Read Agile Model Driven Development (AMDD), posted at www.agilemodeling.com, to learn about how it is in fact possible to model in an evolutionary and documentation-light manner, instead of the traditional serial and documentation heavy one that many data professionals are accustomed to.
5. Read The Process of Database Refactoring to learn how to evolve a database schema after the fact.  It is a myth that you must have the database schema in place, or at least mostly in place, before coding begins.
6. Read Why Data Models Don't Drive Object Models (And Vice Versa), The Object-Relational Impedance Mismatch, and Mapping 101 to overcome any misconceptions about how to effectively use object and relational technologies together.
7. Read the rest of the articles at this site.

Note that there is an assumption in the writings at this site that project teams are taking an agile approach to software development.  This may not be completely true of a RUP project team, although hopefully should be, but what is true is that the team will be taking at least an evolutionary approach.  Therefore the techniques described at this site are still applicable although may be a little harder to accomplish in environments where the RUP has been instantiated in a non-agile manner.

References and Suggested Online Readings

Adopting Agile/Evolutionary Database Techniques Agile Database Best Practices Agile Database Techniques plug-in for the Eclipse Process Framework (EPF) Agile/Evolutionary Data Modeling Agile Modeling and the RUP Agile Testing and Quality Strategies: Discipline Over Rhetoric Agile Unified Process (AUP) Architecture Envisioning: An Agile Best Practice Choosing a Primary Key: Natural or Composite? Comparing the Various Approaches to Modeling in Software Development The Cultural Impedance Mismatch Between Data Professionals and Application Developers Data Modeling 101 Introduction to Data Normalization Mapping Objects to Relational Databases On Relational Theory The "One Truth Above All Else" Anti-Pattern Prioritized Requirements: An Agile Best Practice The Process of Database Refactoring Rationally Unified Data (Software Development, May 2005) Requirements Envisioning: An Agile Best Practice A RUP-Based Approach to Developing a Data Warehouse Part 1: Setting the Stage Part 2: Building and Deploying It Survey Results (Agile and Data Management) UML Data Modeling Profile When is Enough Modeling Enough? The EUP is an extension to the RUP, turning it into a full IT lifecycle. The AUP is an agile instantiation of the Unified Process.  You can download the AUP product free of charge.

	This book describes, in detail, how to refactor a database schema to improve its design. The first section of the book overviews the fundamentals evolutionary database techniques in general and of database refactoring in detail.  More importantly it presents strategies for implementing and deploying database refactorings, in the context of both "simple" single application databases and in "complex" multi-application databases.  The second section, the majority of the book, is a database refactoring reference catalog.  It describes over 60 database refactorings, presenting data models overviewing each refactoring and the code to implement it.
	This book describes the philosophies and skills required for developers and database administrators to work together effectively on project teams following evolutionary software processes such as Extreme Programming (XP), the Rational Unified Process (RUP), the Agile Unified Process (AUP), Feature Driven Development (FDD), Dynamic System Development Method (DSDM), or The Enterprise Unified Process (EUP).  In March 2004 it won a Jolt Productivity award.
	This book presents a full-lifecycle, agile model driven development (AMDD) approach to software development.  It is one of the few books which covers both object-oriented and data-oriented development in a comprehensive and coherent manner.  Techniques the book covers include Agile Modeling (AM), Full Lifecycle Object-Oriented Testing (FLOOT), over 30 modeling techniques, agile database techniques, refactoring, and test driven development (TDD).  If you want to gain the skills required to build mission-critical applications in an agile manner, this is the book for you.

Let Us Help

We actively work with clients around the world to improve their information technology (IT) practices, typically in the role of mentor/coach, team lead, or trainer.  A full description of what we do, and how to contact us, can be found at Scott W. Ambler + Associates.