The Architecture Process for Agile Organisations

This article attempts to describe an example of an implementation of an architecture process in organisations that strive towards greater agility. Implementing an architecture process can be a daunting exercise, and one of the pitfalls you will encounter is making it too rigid and top-heavy. Especially in situations where you already have teams implementing more agile processes, your architecture process should not hinder them. By creating an agile architecture process you will reap the benefits of working agile on a much broader scale in the enterprise, leading to more synergy between business and IT.

The article is applicable for medium to large sized organisations, where a robust architecture process is mandatory.

The RACI matrix

Before diving into the architecture process itself, we think about roles and responsibilities in organisations. For this we often employ a matrix that we find helpful: the RACI matrix.

Person/Role	Accountable	Responsible	Consulted	Informed
Jackson	x
Smith		x
…			x
…				x

The matrix as shown above shows a possible situation for a specific project or organisational unit. We see that Smith is Responsible, Jackson is Accountable, and some others who are Consulted or Informed.

Various variations on this basic model exist, for more information see the links below this article.

What is the meaning of these terms?

• Responsible – means a person or role with the skills and or expertise to understand and explain, and to peers (other Responsible persons or roles) defend a specific policy or action
• Accountable – means a person or role with the assigned final right to decide, with the mandate and usually the funding to start or stop a certain organisational activity such as a project. Equivalent to an executive responsibility.
• Consulted – means a person or role that is consulted for the activity, because their input is necessary or desirable. The input is in the area of knowledge, skills or expertise, rarely on the executive level.
• Informed – means a person or role who should be informed of the proceedings or decisions taken, but is not directly involved in the process.

We focus in this article on the first two roles, because they are the most interesting with respect to the architecture process. Also we try to give a deeper meaning to the Responsible role, since we think that role has been undervalued in the literature, and we link this role to that of the architect.

We state that, for an organisation to be operating in clear, auditable and transparent processes, it is imperative, in the context of a project or policy, for any person or role in the organisation to be exclusively assigned to one of these aspects in the RACI matrix. This is especially critical for the Responsible and Accountable roles:

Responsible ↔ Accountable

We are talking about roles here, and in the context of a specific project or policy implementation (for example an organisational unit). This means that concrete persons can have more than one role, when looking at their activities across the boundaries of projects or organisational units.
Usually the boundaries between executive responsibilities (which we call Accountable), and those responsibilities that are based on knowledge or skills on the content of a strategy or decision (which we call Responsible), are vague, undefined, or not deemed important in any way.
In fact, most organisations have a relatively well defined executive process, implemented in an executive hierarchy, but pay no attention at all to skills and knowledge management of the highly trained professionals on which the entire organisational operation is depending. This leads to what is often called “brain drain”. Expertise is often hired, the bearers of which are part of the organisation for a delimited period. These individuals may learn a lot and thus increase their value to the organisations that hire them, but their knowledge is not incorporated into the organisations that hire them, except indirectly in the implemented systems or policies.
Often, after these individuals have left the organisation, the system they implemented begins its slow decay as if the heart has been taken from it.
Another symptom is that there is no well defined career path for persons who do not desire or lack the skills for managerial functions. These persons can be exceptionally gifted and productive, and the lack of a career path for these persons implies that the organisation will eventually lose these valuable assets, either because these persons choose their career paths elsewhere, or because they eventually choose to become managers against their preferences.

Actually many organisations have realised already the need for “flattening” the organisational pyramid, which can only succeed if they also realise the need for “responsible” roles and career paths.

In most organisations there is a severe imbalance between Accountable and Responsible behaviour.
We state that it is important for the successful enterprise to operate from a balance between the two.

Architects and Managers

I can now introduce the role of an architect: an architect is a person or role with responsibilities based on skill and/or knowledge:

Accountable 🔁 Manager

Responsible 🔁 Architect

To implement the role of an architect, or any role or person which is Responsible, we need to pay close attention to the limiting condition that this role does not have executive responsibilities. An Architect is not a Manager!
The vice-versa is equally important: a Manager is not an Architect. In an organisation with an undefined or immature architectural process we find that there is a high demand for “explanations”: documents, presentations, written by knowledgeable experts. These explanations are almost exclusively targeted at an audience lacking the expertise to understand or appreciate the actual content of these documents. They function solely as a smoke screen for policy makers to enable them to defend themselves when problems arise: “look here, we really looked at the problem thoroughly!”. These explanations are usually provided with a so-called “management summary”. These chapters are often an illuminating read. They treat the reader as an imbecile. They employ a language that is suited for children (and even often openly referred to as “children’s language”!), and by a long reach do not do justice to the contentual quality of the documents. Usually the real content is not read at all, except by the writers themselves.
The lack of an architectural process introduces the real danger that the content of these documents is insufficiently challenged, and does not reach the desired quality.

Mixing executive and contentual responsibilities is a real problem, and one that is consistently underexposed. When a manager tries to collect knowledge to appreciate the expert advice as a peer expert, he is trying to do the impossible. His responsibilities never give him enough time and opportunity to gain the required level of expertise, and worse: his daily activities are for the most part not directly related to this knowledge area, so it is consistently underexposed.
There are many more problems. An example is the way the reward system in most organisations, specifically in executive hierarchies, is implemented. Managers are judged to the extent they reach the formulated goals, usually only expressed in terms of time and money. There is no valuation of the qualitative aspects of his work or decisions. Nor, and I want to emphasise this here explicitly, should there be. When the architectural process is implemented in a way resembling the one set forth in this document, there does not need to be, because in that case processes are set in place to guarantee the quality of the information on which decisions are based.

When an architect is required to spend time on executive actions, such as evaluating persons in his team or allocating funds to activities, he is handicapped in spending sufficient time on the real content of the activities. But a worse problem is the fact that he is creating conflicting interests. The architectural roles in his environment, especially the architects below or above him, tend to get a skewed advice, because he has two interests to take into consideration: the content and intrinsic quality of his work, and the power structures, the political and executive boundary conditions that have been set. These aspects usually conflict. It makes the work of an architect, as well as the work of the executive managers involved, a lot easier, and avoids the conflicting interests, if we separate the Responsible and Accountable roles.

Architectural Process

The architectural process describes roles and responsibilities of knowledge workers and how they interact with the executive (Accountable) hierarchies.
For the architectural process to be introduced in organisations, it is necessary to start by creating awareness of architectural decisions and architectural aspects of the work that is being done by the organisation.
The definition of an architectural activity is:

Any activity or decision that has side effects, that is rippling effects that directly or indirectly influence components outside the direct area that is under consideration.

Rob Vens

What we are talking about here is systems theory, or the well known paradigm of coupling. We strive toward systems that are loosely coupled (minimal side effects) and highly cohesive (performing complex tasks efficiently). In fact another rationale for the implementation of an architectural process is to make this explicit. Enterprises are by definition complex systems, and complex systems need an architecture to stay competitive and successful. Actually they have an architecture whether they are aware of it or not!
What happens in the architectural process is that participants in the process are aware of their responsibilities and how to act in a variety of situations that may arise. They consistently and constantly work toward minimising coupling by employing architectural techniques, and maximising cohesion by explicitly positioning the goals that have been formulated for the enterprise in all activities that are undertaken on all levels of the enterprise.

The architecture process and project management

From the arguments explained above, we still need to sketch out the actual structure of an architecture process as it will function in a project organisation. This will illustrate the structure of the architecture process, and in fact will be a reflection of the implementation of the architecture process on enterprise levels.

A project has a definite boundary in time and money. In fact the responsibility of a project manager is to assure that these boundaries are respected, and that the primary goal of delivering a solution to a customer is reached within these boundaries.
In fact in these requirements we can already recognise the two main players:

1. Time and money
2. Business value

Within the structure of a project, the argument of this article is that the two should not be combined in one role or person, i.e. the project manager. We designate two roles for these two main considerations:

1. Time and money — Accountable — Project Manager
2. Business value — Responsible — Project Architect

For many projects a comparable structure has already been defined, but most of these suffer from an immature implementation:

1. There is no process in place for the project manager to validate the quality of the project architect — hence she will constantly try to understand and validate this work herself (mix of roles). This is a form of waste (Lean).
2. As a side effect of the previous point, the project architect is constantly forced to explain his decisions to the project manager, in a language that is by definition not rich enough to do that since the project manager lacks the expertise for that. In The Netherlands we even have an expression for these attempts: “Nijntje taal”, after a character in pre-school books. This is a serious symptom because it will gradually undermine the necessary respect the different roles in the process will need to have for each other.
3. The project architect has mixed interests because he was assigned to define the architecture of the project, but also has to obey to the rules of a departmental or enterprise architecture or architects. In fact he may very well be recruited from a pool of enterprise architects and himself have played a role in defining this enterprise architecture. Thus: conflict of interest with a direct impact on the effectiveness of processes.
4. There is insufficient awareness in the project team of the meaning of architecture, causing a strong increase in workload for the project architect.
5. There is no process in place to handle a conflict between project manager and project architect, usually resulting in the project manager taking the decision without consent of the architect, which may be detrimental for the quality of the work. In a similar vein, a project manager may yield to the decisions of the project architect too easily because she lacks the instruments to validate his work, which may endanger the time and money boundaries of the project.

We summarise the various aspects of the architecture process within the scope of a project for your convenience:

1. A project architect has the interests of the project as his primary concern and responsibility. Any defined architecture outside the project scope will be secondary to it. Note that the defines architecture process is in place to manage possible negative consequences of this.
2. An escalation path has been defined when the previous point will create a conflict. For this the project architect will not take it upon himself to defend the enterprise architecture but will defend his case in front of a group defined outside the project scope (usually something like a steering committee) and delegate the responsibility of defending the enterprise architecture to another person representing the enterprise.
3. A project manager will not rely too much on her own understanding to validate the quality of the architectural decisions, but is willing to delegate this to the architecture process itself.
4. All members of the project team have architectural awareness and function within an implemented architectural escalation process.
5. All members of the project team are aware that they all are architects at some points in their work, or take architectural decisions more or less often.
6. The architectural escalation process describes in unambiguous terms the way people function as architects.

You can read more on the separation of concerns regarding responsibilities in the article: The Responsible Architect. Also this article explicitly refers to a political model that served as an inspiration for me: the Trias Politica, originally formulated by Montesquieu, who was inspired to write about this by the fledgling Republic of The Netherlands, and which served as a major inspiration for the United Stated Declaration of Independence.

Conclusion

In the literature the architecture process itself has often been undervalued. We try to remedy that with this article and show the value of an implemented architecture process.