Modern software development processes, including
Disciplined Agile Delivery (DAD), Extreme Programming (XP), and
Scrum are all evolutionary
in nature. The implication is that if data professionals are going to be
effective members of such teams, then they need to adopt tools and
techniques which enable them to do so. There is nothing special
about the data aspects of an IT system: they can be developed in an
evolutionary manner just like non-data aspects. Even
data warehouse and business intelligence projects
can be developed in an evolutionary manner (and quite frankly this is
the preferred approach). This article overviews a
collection of best practices for agile/evolutionary database
These "best practices" for evolutionary/agile database
Database refactoring. A
refactoring is a small change to your database schema which improves its
design without changing its semantics (e.g. you don't add anything nor do
you break anything). The process of database refactoring is the
evolutionary improvement of your database schema so as to improve your
ability to support the new needs of your customers.
Agile data modeling. With an
evolutionary approach to data modeling you model the data
aspects of a system iteratively and incrementally. With an agile
approach you do so in a highly collaborative manner. You will envision
architecture early in the project at a high-level and then
storm the details on a just in time (JIT) basis when and if they're
needed -- no more "big
up front modeling (BMUF)".
Database regression testing. You should ensure that
database schema actually meets the requirements for it, and the best way to
do that is via testing. With a
test driven development
(TDD) approach you write a unit test before you write production
database schema code, the end result being that you have a
test for your database schema.
provides the concrete feedback that you need to ensure data quality.
Configuration management of database artifacts.
Your data models, database tests, test data, and so on are important project
artifacts which should be
configuration managed just like any other artifact.
Developer sandboxes. Developers need their own
working environments, called
sandboxes, where they can modify the portion of the system which
they are building and get it working before they integrate their work with
that of their teammates.
normalization is a process in which data attributes within a
data model are organized to increase the cohesion of entity types.
In other words, the goal of data normalization is to reduce and even
eliminate data redundancy, an important consideration for application developers because
it is incredibly difficult to stores objects in a relational database that
maintains the same information in several places.
Set a realistic primary key strategy. The
fact is that sometimes it makes sense to use
natural keys and
sometimes surrogate keys. As a professional you need to understand
when to apply each strategy, and to be prepared to
if you discover that you've made the wrong choice.
Database encapsulation. A
database encapsulation layer hides the
implementation details of your database(s), including their physical schemas,
from your business code. In effect
it provides your business objects with persistence services – the ability to
read data from, write data to, and delete data from – data sources.
Ideally your business objects should know nothing about how they are
persisted, it just happens. Database
encapsulation layers aren’t magic and they aren’t academic theories;
database encapsulation layers are commonly used practice by both large and small
applications as well as in both simple and complex applications.
Database encapsulation layers are an important technique that every agile
software developer should be aware of and be prepared to use.
Train developers in basic data skills.
This enables developers to both improve their data-oriented work and to
interact with data professionals more effectively. Fundamental skills
relational database fundamentals,
to RDBs (O/R mapping),
working with legacy data,
integrity and shared business logic,
objects from an RDB,
how to implement
access control, and
and concurrency control.
Train DBAs in basic development skills.
Similarly, DBAs need to gain an understanding of application development so
that they can play active roles on the team(s) which they support.
Common development guidelines. Having a
common, usable set of development standards which are easy to understand and
to comply to can greatly improve the quality of the systems that you
develop. These guidelines may include, but not be limited to,
guidelines, modeling style
guidelines, data naming conventions, and user interface conventions
(including report design conventions).
Lean data governance. The goal of
governance is to ensure the quality, availability, integrity, security,
and usability within an organization, and the goal of
data governance is to enable development teams to do these things
effectively within your overall IT ecology.
Many traditional approaches to data governance seem to
struggle in practice, I suspect in part because of the
cultural impedance mismatch but also in part because traditional IT
governance struggles in general. The command and control approach typical of
traditional governance strategies is a lot like herding cats, you do a lot
of work but nothing much gets accomplished in the long run.
Lean governance, on the other hand, is focused on enabling people and
motivating them to do the right things. A lean data governance
approach promotes a healthy, collaborative
relationship between data professionals and the teams that they're
Agile master data management (MDM). If
you're going to adopt an
MDM strategy within your organization, it should at least be an
agile one. Many organizations struggle when it comes to MDM, typically
because they adopt a traditional, command-and-control strategy. Your
MDM efforts can in fact be very agile and streamlined if you choose to.