Observations from the Field – The Hard Part of SOA

Where have I been? It’s been over a month since I posted. I’d wish I could say I’ve merely been enjoying the first springtime in Maine I’ve witnessed in the dozen or so years I’ve lived here. But, in actuality I’ve been working – consulting, chatting with architects and lead technologists, writing, reading, tinkering and thinking. It’s the last one (thinking) combined with a period of silence that used to simultaneously intrigue and frighten my old (awesome) team at LLB. It was hard to predict the outcome of such a cycle, but it was bound to be interesting! At the moment, even I’m not sure what will transpire, so I’ll refrain from wasting more space on that here (for now).

In the meantime, I want to share some information and insights I’ve collected over the last month. Instead of dumping everything into one post, I thought I’d start with an excerpt from the Observations from the Field: SOA report I just wrote for PSGroup.

In this excerpt, I share the “Hard Part of SOA” as identified by enterprise practitioners and lead thinkers/technologists (not marketers) at application infrastructure vendors.

The illustration I’ve included is one I first drew for a SOA workshop gig in 2004. I’d be curious if readers see the relationship between re-use and granularity differently.

Since real-world insights are invaluable, I invite enterprises to share their SOA tips, concerns, and stories with me. In turn, I’ll share that information back to the broader community. Together, we can cut through the hype, and develop pragmatic practices for SOA success.

Excerpt: The Hard Part of SOA

Enterprise architects and technical leaders consistently state the hardest part of SOA isn’t the technology. Rather, the real work is in service definition, semantics, and establishing a SOA program.

1. Service Definition

Many enterprises find it difficult to determine the correct bounds (job and tasks) and granularity (collaborations) of business services. Correctness is viewed in terms of both re-use and agility. Business services should be reusable (shared) within a line of business, across lines, and in some cases, at the enterprise level. Business services should be easily changeable (replace, augment, compose) to meet business demands.

Enterprises have erred equally on creating services at too fine a grain, and too large a grain. Erring on the fine-grained side still allows for re-use, through service collaboration, but creates performance inefficiencies to perform actual work (bounds). Erring on the large-grained side creates application- or use-specific services, rather than common business services.

Business Service Re-Use Diagram [click on diagram to enlarge]

This illustration shows the general relationship between granularity and service re-use. As service granularity increases, the opportunity for re-use of the resulting service decreases. However, what’s important to note is the reliance on collaborators when creating a large-grained service. A large-grained service, such as a process-oriented service, should be composed of multiple function and information-oriented services. The proper use of appropriately bounded (defined) collaborators is the re-use payoff.

Industry Specifications. Architects most confident in their business service definition have been able to leverage industry-specific specifications, such as the Open Travel Alliance (OTA). OTA specifications include schemas and WSDL implementation guidelines for common travel-related interactions, such as Request Rail Availability.

Unfortunately, industry specifications for true service interactions, rather than information hand-offs, are the exception rather than the norm. In most cases, architects and business analysts are relying on
service definition practices from SOA technology and/or solution providers, or adopting best practices from experience, gut feel, and a wide range of published methods, including PSGroup’s Service Discovery.

Lack of Methodology.
What’s missing—and desperately needed—is a publicly available, cohesive, services definition methodology. Cohesive doesn’t refer to degree of documentation. [I believe a methodology should contain the right amount of methodology to accomplish the task, no more, no less.] Rather, with the word “cohesive”, I am referring to the service definition activities that span business modeling, analysis, and design. You find a service (and the need for it) during business modeling. You define and refine the service during analysis. You further refine and provision the service during design.

In such, the business definition of a service is derived independent of technology implementation—current or future. As notation and tooling are used to transition the business service definition to technical implementation, the business intent must be retained.

Since the emergence of a good methodology happens over time, from real-world experience, I encourage practitioners to share their tips and challenges publicly, and join or create practitioner-led forums to advance real-world SOA.

2. Semantic Understanding

For humans, machines, and the combination, semantics has always been a challenge. While it is relatively easy to receive information (conversation, message, data exchange) it is not always easy to discern the sender’s true meaning. For example, when the word “customer” is used, does it mean end consumer, prospect, or purchaser?

As information exchange between applications and businesses became prevalent, industry and company dialects were defined to assign common terms, syntax and meaning to information elements. Given the initial purpose (technical integration) of these terms, the definition and administration has largely been the purview of technologists (geeks). As such, many of the terms are cryptic, derived from the application and information sources, rather than a true business dialect.

With SOA, the need for semantic understanding becomes even more pronounced, because of the multitude of service interactions, and the self-describing nature of service-oriented languages for service descriptions, contracts, policies, and orchestrations.

Use Business Terms.
Similar to the service definition (above), semantics must be expressed in business terms. These terms must be recognized and understood by both business and technology professionals and machines, within an enterprise, and often between enterprises.

Most architects are approaching the challenge of semantics incrementally. A first priority is to achieve a common language between human parties in the enterprise, to enable proper business service definition.
Another is to adopt a common XML dialect to enable semantic interoperability between services.

A common practice is to describe interfaces and message payloads with a common XML dialect, and leave the underlying providers (applications, data stores) in their current formats.

The adoption of industry standard dialects varies by enterprise. Some enterprises adopt industry standards such as ACORD for insurance, throughout the enterprise. Others limit the adoption to customer- and partner-facing services. A common complaint from architects is the industry dialects are overly complex. And the problem worsens as functional dialects (human resources, banking) are added to the mix.

Vendors agree. The semantics problem is very difficult to resolve. While most offer support for industry dialects in their SOA products, none have tackled the big semantic problem. Although the problem is difficult, it can’t be ignored. To ease your way, be sure to define and/or adopt a common business lexicon for semantic compatibility of services, applications, events, processes, information stores, and of course, people.

3. SOA Program

The mission of an SOA program is to articulate and execute an SOA strategy that makes sense for your business. Common activities include evangelizing SOA, setting and enforcing standards (technical, methods), selecting application infrastructure and tooling, building out common (system-oriented) services, finding and conducting pilot projects, and stewarding the service catalog.

Architects, technical leaders, and business solution project leaders say the biggest challenge in developing a SOA program is balancing the natural tension of executing business projects (today, on-time, on-budget) with the need for architectural integrity.

Obviously, the simplest thing to do is give your architecture team a head start. Fund, staff, and start the SOA program three to six months ahead of the first business project that needs it. The shorter the head start, the less critical that first business project should be.

Barring the luxury of a head start, build iteration, tolerance, and refactoring into the execution of your architecture and business project. Tips from the field include:

•Re-Use. Be smart about re-use mandates. While every service has re-use potential, focus initial development efforts on business and infrastructure critical services. Everywhere else, use good design practices—well-defined interfaces, separation of concerns—to allow for non-invasive replacement of “non-enterprise class”
assets.

•Increments.
Plan and deliver your SOA environment—practices, tools, technology, and infrastructure—incrementally. Plot your plan to balance business project delivery and architecture delivery. Better to build an incremental architecture, than a bottleneck, or bypass route.

•Refactor.
For tolerant re-use and incremental architecture, you must plan and allocate time and resources to refactor and re-release code/assets.

•Simplify.
Trust your architect instincts. Think about the standards, technology, and products you need in a service-oriented manner, and then map options to them. Don’t buy a box of SOA. Leverage tools and practices you have in place today.

•Communicate.

Early and often.