Agile Enterprise Architecture
AgileData.org: Techniques for Disciplined Agile Database Development
|When project teams work under the assumption that they can do anything that they want, that they can use any technology that they want, chaos typically results. Functionality and information will be duplicated and reuse will occur sporadically if at all. Systems will not integrate well. Systems will conflict with one another and cause each other to fail. Costs will skyrocket because similar products from different vendors, or even simply different versions of the same product, will be purchased and then operated within production. Although each individual project may be very successful, as a portfolio they may have serious challenges. It doesn’t have to be this way.||
Why are enterprise issues an important aspect of the Agile Data (AD) method?
Because data is an enterprise asset. It isn’t your only enterprise
asset, but it is an important one. My philosophy is that to be effective
as a developer you must consider enterprise issues, which is why
Data’s 2nd philosophy “development teams must consider and act
appropriately regarding enterprise issues” exists. However, to remain
agile your organization must find a way to streamline their enterprise
activities to support agile software development teams in this endeavor.
Agile Data’s 3rd philosophy “Enterprise groups exist to nurture
enterprise assets and to support other groups, such as development teams, within
your organization. These enterprise groups should act in an agile manner
that reflects the expectations of their customers and the ways in which their
In this article, I discuss:
This article has been written with the following assumptions:
For our purposes, enterprise architecture consists of the various structures and processes of an organization, including both technical structures and processes as well as business/domain structures and processes. Following this definition, an enterprise architecture model is a representation of those structures and processes. A good enterprise architecture model will depict the organization both as it is today and as it is envisioned in the future, and will map the various views representing the architecture to one another. These views include both business-oriented perspectives as well as technical perspectives. In many ways enterprise architecture models are a communication bridge between senior business stakeholders and senior IT professionals.
Unfortunately, within the IT industry the terminology isn’t used in quite this way. Sometimes we use the term “enterprise architecture” to refer to the group of people responsible for modeling and then documenting the architecture. Other times we use the term to denote the process of doing this work. More commonly we are referring to the models, documents, and reusable items (components, frameworks, objects, and so on) that reflect the actual architecture. Unless otherwise noted, this is the way that I will use the term at this site.
In the Enterprise Unified Process (EUP) I point out how some organizations choose to separate the business/domain side of EA from the technical side of it, something which is reflected in the EUP's enterprise business modeling and enterprise architecture disciplines respectively. If your organization chooses to think of the EA as encompassing both, which I recommend, then your EA strategy encompasses the scope of those two EUP disciplines.
The benefits of enterprise architecture can be summed up
using three words: better, faster, cheaper. It is important to realize that the
better, faster, cheaper (BFC) benefits
come at a price. You must be
willing to invest in the underlying organizational and cultural structures to
support them. You need to recognize
that these costs exist and ensure that the BFC benefits you achieve outweigh
them. Better yet, adopt Agile
ROI and strive for
As a consultant I have the privilege of working in a wide range of organizations across the world. The result is that I get to see and try many different approaches to software development, including enterprise architecture efforts. Over the years I have observed a common set of problems that organizations seem to experience. I have yet to see an organization that experiences all of these problems, although I have seen some that experience all but one or two. These common problems are:
There isn’t an enterprise architecture effort.
Project teams don’t know the enterprise architecture exists.
Project teams don’t follow the enterprise architecture.
Project teams don’t work with the enterprise architects.
Narrowly focused architecture models.
Dysfunctional “charge back” schemes.
A “do all this extra work because it’s good for
the company” attitude.
A common thread behind these problems is a focus on processes and tools over individuals and interactions, the exact opposite of the Agile Alliance’s first value. If your organization experiences one or more of these problems then you may want to consider taking an agile approach to enterprise architecture.
The Agile Data Method's third philosophy is "Enterprise groups exist to nurture enterprise assets and to support other groups, such as development teams, within your organization. These enterprise groups should act in an agile manner that reflects the expectations of their customers and the ways in which their customers work." Let's explore what that actually means.
First and foremost, the values, principles, and practices of Agile Modeling (AM) should help to guide your enterprise architecture modeling and documentation efforts. This is just a good start though. My experience is that to be successful at enterprise architecture you need to rethink your overall approach and address five fundamental issues. These issues are connected in a synergistic manner; you must address all of them otherwise you will put your effort at risk. These issues are:
Focus on people, not technology or techniques
Keep it simple
Work iteratively and incrementally
Roll up your sleeves
Look at the whole picture
Make enterprise architecture attractive to your customers
Fred Brooks (1995) wrote that “The quality of the people on a project, and their organization and management, are much more important factors in success than are the tools they use or the technical approaches they take.” The reality is that enterprise architectures are developed, evolved, and followed by people. All of the arguments over “which model is right”, “which notation is right”, and “which paradigm is right” are meaningless if you don’t have a viable strategy for working together effectively. You could create a perfect enterprise architecture model but it doesn’t matter if project teams can’t or won’t take advantage of it.
Effective enterprise architects will work with their customers, very often application developers and agile DBAs, in the most effective manner possible. Sometimes this will be face-to-face drawing sketches with them at a whiteboard, sometimes they will work with project teams via video conferencing, sometimes they will answer questions via email, and sometimes they will publish models and documents. I highly suggest following the AM practice Display Models Publicly for your architectural models and documents – publish them online at an internal web site and even consider putting up paper versions of critical diagrams in their workspaces. A common mistake that I’ve seen architecture groups make is to put their diagrams on the walls within their work areas but not the work areas of the application developers. Better yet, as I argue below there shouldn’t be separate work areas to begin with.
Enterprise architects also work as “boundary spanners” between project teams and the people within your organization that have the long-range vision – very often senior IT and senior business executives.
An interesting observation is that enterprises have two organizational structures – the formal one documented by your organization chart and the informal one that people use to get things done. Within IT departments there are always the “go to guys” that developers work with to get things done, very often other developers that have critical skills or knowledge. Agile enterprise architects understand this and actively seek to become “go to guys”.
A critical concept is that your enterprise architecture models and documents just need to be good enough, they don’t need to be perfect. It is naïve to assume that you can produce perfect artifacts, you’re only human: you will never get it perfectly right and nobody expects you to anyway. Furthermore, even if you did manage to create perfect artifacts they’d be out of date the day after you published them because something within your business or technical environment would change (Embrace Change is also an AM principle). Why is this important? My experience is that a hand-drawn sketch today on a plain old whiteboard (POW) can often be far more valuable than a fully documented and validated document several months from now. Timely guidance from an enterprise architect who understands the current environment and the future vision for the enterprise, even when this guidance is imperfect and based on incomplete information, is often far better than the uneducated guesses that a project team will make on their own while they wait for the official architecture to be published.
By keeping your enterprise architecture artifacts simple you increase the chances that your audience will understand them, that project teams will actually read them, and that you will be able to keep them up to date over time. Overly detailed documents might look impressive sitting on a shelf, but a simple model that project teams actually use is what your true goal should be. You might find When is Enough Modeling Enough? to be interesting.
Agile enterprise architects work in an iterative and incremental manner. Agile modelers will follow the practice Apply the Right Artifact(s) and use a wide variety of modeling techniques as required (more on this later). They will also follow the practice Iterate To Another Artifact when they realize that the model that they are working on either isn’t the appropriate one with which to depict a concept or because they are simply stuck and need to break out of their “analysis paralysis”. They will also follow the practice Create Several Models In Parallel so that it is easy for them to iterate back and forth between artifacts. Instead of holding a use case modeling session an agile modeler will focus on requirements modeling, or even just modeling, and work on use cases, business rules, change cases, and whatever other artifact they require to get the job done. The advantage of these practices is that the right model is used for the task at hand.
Agile modelers will also follow the practice Model in Small Increments, modeling just enough to fulfill the purpose at hand and then moving on to the next task. They don’t try to create complete models, instead they create models that are just good enough. When they find that their models are not sufficient they work on them some more. The advantage of this approach is that they evolve their models incrementally over time, effectively taking a just-in-time (JIT) model storming approach that enables them to get the models in the hands of their customers as quickly as possible.
On the preceding advice, some readers may say to themselves “All of this sounds great, particularly for a project team, but enterprise architecture is different. It’s more complex. It’s more important. It requires significant modeling up front to get it right.” Aaarrrrggghhh!!! That’s old-style thinking. Enterprise architects can work in an iterative and incremental manner if they choose to.
Figure overviews how to take an Agile Model Driven Development (AMDD) approach to enterprise architecture. The enterprise architecture team would define the initial architectural vision and models, a process that would likely take several days or even a week or two. Any longer and you’d be at risk of developing architectural models that don’t actually reflect something that would work in practice. Remember, your models need to be just good enough, not perfect. The idea is that your enterprise model(s) start out small and are fleshed out over time based on the feedback you receive from both the business community and from project teams.
Figure 1. Agile Model Driven Development (AMDD) at the enterprise level.
Although an important part of the job of an enterprise architect is modeling and documentation, that should not be your primary focus. Supporting the architecture within project teams should be, coaching developers in the architecture and architecture skills. The best way to do this is to get involved with project teams, to work with them to understand the enterprise architecture and to try it out in practice. In other words, the enterprise architects will often take on the roles of "architecture owners" on the application teams. This approach has several benefits:
You quickly discover whether or not your ideas work, and if so then how well.
You improve the chance that project teams understand the architecture because you’re working with them face-to-face.
You cross-fertilize ideas, particularly technical ones, across teams, quickly sharing good ideas and strategies.
You improve the chance that a common infrastructure, both technical and business, will be built and reused over time because the project teams will be working towards the enterprise architecture.
You gain experience in the tools and technologies that the project teams work with, as well as the business domain itself, improving your own understanding of what it is that you’re architecting.
You obtain concrete feedback that you can then act on to improve the architecture, enabling it to evolve over time to meet the actual needs of your organization.
You gain the respect of your primary customers, the application development teams, because they see that you’re participating and not simply pontificating.
You actively help to build software-based systems, the primary goal of IT professionals.
You can mentor the application developers and agile DBAs on the project teams in modeling and architecture, improving their skill sets.
You provide clear benefit to application teams because you’re helping them to fulfill their immediate goals, forsaking the “do all this extra work because it’s good for the company” attitude for a more attractive “let me help you achieve your goals, and by doing so together we’ll do something good for the company” attitude.
You work closely with the development teams and enterprise administrators to ensure that your overall data management (including Master-Data Management (MDM)), security management, network management, ... efforts support and enhance the development teams efforts instead of hinder them.
My experience is that the enterprise architects need to be active members of project teams, and to do that they must be co-located with them. When architects are in a different location, perhaps a different corner, on another floor, or even in another building, their ability to communicate with and work together effectively with the project team(s) they are trying to support is dramatically diminished. The implication is that enterprise architects may need to become nomadic, moving between their “home base” to the work areas of the project team(s) that they support. When you work side by side with someone you pick up more information (often just by overhearing something) and you make yourself easily available to them thus increasing the likelihood that they will take advantage of your expertise.
Agile enterprise architects will ensure that their technical ideas actually work before they advice project teams to try them by writing a small system to validate the idea. This is called a spike solution in eXtreme Programming and a technical prototype or skeleton in the Rational Unified Process (RUP). The idea is that you write just enough code to verify that what you think will work actually does. This helps to reduce technical risk because you’re making technology decisions based on known facts instead of good guesses.
Agile enterprise architectures, agile modelers in general, believe in the principle Multiple Models and thus strive to look at the whole picture. They don’t just focus on data models, or object/component models, or business models, or whatever type of model might tickle their fancy. Instead they strive to model from several points of view so that their understanding and description of the architecture is more robust.
Agile enterprise architects realize that they need to make their work, including their services, attractive to their customers (developers and business stakeholders). If your customers perceive that you have value to add, that your enterprise architecture efforts will aid them in their jobs, then they are much more likely to work with you. If, on the other hand, they think that you’re wasting their time they won’t work with you. They’ll find ways to avoid you, to cancel or postpone meetings with you, to find ways around you.
Introducing the techniques and philosophies described in this article will prove difficult in many organizations, particularly those that have an established enterprise architecture group that follows a traditional approach. Adoption agile techniques requires a change in mindset – agile enterprise architects are service oriented, realizing that it is their job to help project teams to succeed and to work with senior stakeholders to define and evolve the corporate vision. Agile enterprise architects realize that they need to make it as easy as possible for other people to work with them and that they must provide perceived value to the teams that they support. They realize these things, and act accordingly, because they know that the people they are supposed to serve will ignore them if they don’t. In the end it’s all about people and the way that you interact with them.
My experience is that the best way to introduce agile architecture techniques into an organization is to start small at first and grow your strategy over time. This approach allows you to gain some initial successes, to learn from your experiences (because everything won’t go perfectly right), and to evolve your strategy based on those learnings. A common mistake is to try to put an enterprise architecture team in place and have all teams start to follow the new vision. The typical result is that existing project teams become confused, the enterprise architecture team becomes swamped trying to play catch up, and fighting ensues over “which is the best way”. The fundamental mistake that is made with this approach is that it doesn’t allow the enterprise architects to build a solid foundation from which to work, to build up the proof that their approach actually works, and basically to get ahead of the project teams.
If you’re hoping for an exact list of deliverables here then you need to go back and re-read this article because you haven’t gotten it yet. Sorry for being harsh, but sometimes you just need to say it as it is. However, it is important to define a set of goals that should be achieved. In priority order, these goals are:
Customer support for the enterprise architecture.
A vision and plan to achieve that vision.
A collection of models and documentation describing the architecture.
No approach is perfect, including this one. I would be remiss if I didn’t identify these known issues:
It does not include an explicit way to ensure compliancy (although having enterprise architects embedded on the teams goes a long way towards this).
It depends on people being responsible.
It requires you to actively strive to keep things simple.
It requires you to accept an agile approach to modeling and documentation.
The approach described in this article works incredibly if you let it. Your most important take away point is that it’s all about people. Fancy tools based on theoretically sound frameworks, metamodels, or modeling languages are great to have but they won’t do anything for you if developers don’t use them. It’s all about people. Sophisticated models and documents are interesting to create, but they offer little value if nobody reads them. It’s all about people.
|The EUP is an extension to the industry standard IBM Rational Unified Process (RUP). Whereas the RUP defines a software development lifecycle, the EUP extends it to cover the entire information technology (IT) lifecycle. The extensions include two new phases, Production and Retirement, and several new disciplines: Operations and Support and the seven enterprise disciplines (Enterprise Business Modeling, Portfolio Management, Enterprise Architecture, Strategic Reuse, People Management, Enterprise Administration, and Software Process Improvement).|
|This book describes the philosophies and skills required for developers and database administrators to work together effectively on project teams following evolutionary software processes such as Extreme Programming (XP), the Rational Unified Process (RUP), the Agile Unified Process (AUP), Feature Driven Development (FDD), Dynamic System Development Method (DSDM), or The Enterprise Unified Process (EUP). In March 2004 it won a Jolt Productivity award.|
This book describes, in detail, how to refactor a database schema to improve its design. The first section of the book overviews the fundamentals evolutionary database techniques in general and of database refactoring in detail. More importantly it presents strategies for implementing and deploying database refactorings, in the context of both "simple" single application databases and in "complex" multi-application databases. The second section, the majority of the book, is a database refactoring reference catalog. It describes over 60 database refactorings, presenting data models overviewing each refactoring and the code to implement it.
|This book presents a full-lifecycle, agile model driven development (AMDD) approach to software development. It is one of the few books which covers both object-oriented and data-oriented development in a comprehensive and coherent manner. Techniques the book covers include Agile Modeling (AM), Full Lifecycle Object-Oriented Testing (FLOOT), over 30 modeling techniques, agile database techniques, refactoring, and test driven development (TDD). If you want to gain the skills required to build mission-critical applications in an agile manner, this is the book for you.|
We actively work with clients around the world to improve their information technology (IT) practices, typically in the role of mentor/coach, team lead, or trainer. A full description of what we do, and how to contact us, can be found at Scott W. Ambler + Associates.