I am slowly (painfully) making my way cross country to BEAWorld. I hope to gain some insight into the methodologies behind their SOA Practice, and of course talk with some enterprise folks that are on (or considering) BEA’s SOA journey.
Archives for September 2005
This week I wrote a BPEL Primer for the PSGroup research service. The Primer answers 4 questions:
What is BPEL?
What is it Good For?
How does it work?
What does it look like?
I wrote the primer because I think BPEL is an important spec for IT professionals to understand, especially those pursuing SOA and/or integration strategies. For that same reason, I’m excerpting sections (and subsections) here.
In thinking about how to excerpt the report for my blog, I ended up creating a new illustration that shows the whole “A BPEL process is a service orchestration, used to describe/execute a business process (or large grained service), which is implemented as a stateful service” thing. In addition, the picture covers the “A process is comprised of steps, steps have activities AND activities are BPEL language elements, and the basic activity elements are the ones used to interact with
the collaborating services
I wish I thought of it earlier, when I was working on the report. But, that goes to show the real-life nature (and power) of the blog…iteration!
If you want the full report, you can get it for free here. Before bailing for the free report, take a look at the new illustration; it ties into the “What is BPEL” section on page 4 of the PDF.
The BPEL Primer Excerpt: What is BPEL, What is it Good For, How does it Work?
Note: The service terminology is consistent with my SOA definitions post.
What is a Business Process? A business process is a flow of work, broken out as a series of steps, or activities, which are required to complete a business transaction. The business process activities may require application and/or human behavior for completion. A business process is typically long-running in nature, involving multiple parties and/or applications within an enterprise or across enterprises.
What is a Business Process in BPEL? In BPEL, a business process is a large-grained stateful service, which executes steps to complete a business goal. That goal can be the completion of a business transaction, or fulfilling the job of a service. The steps in the BPEL process execute activities (represented by BPEL language elements) to accomplish work. Those activities are centered on invoking partner services to perform tasks (their job) and return results back to the process. The aggregate work, the collaboration of all the services, is a service orchestration.
As you can see, BPEL focuses on a very specific (but important) problem: the orchestration of services, into large-grained services and/or processes. The delineation of large-grained service versus process is based on the specific job being performed. That said, let’s go deeper.
[click ON Picture to enlarge]
What is BPEL?
BPEL, also known as BPEL4WS and WSBPEL, is an XML-based language, built on top of Web Services specifications, which is used to define and manage long-lived service orchestrations or processes. Let’s decompose that in reverse. Orchestrations first, followed by the technical base.
Orchestration. As stated in the SOA terms, in an orchestration, a primary service controls (provides) the collaboration.
When you use BPEL for orchestration, the primary service is a BPEL process, because BPEL processes are implemented as services.
The BPEL process controls the overall sequence and invokes the collaborating services.
To perform the orchestration, the BPEL process contains the logic (sequence, activities, invocations, assignments, and case logic) to invoke other services (collaborators), to complete their combined job.
This BPEL process may be asynchronous or synchronous. In an asynchronous model, the BPEL process is managing a long-lived flow of work; in other words, a process.
In a synchronous mode, the BPEL process is more likely playing the (logical) role of another service type, such as request/reply, worker, agent, or monitor.
BPEL’s technology underpinnings are XML and Web Services. XML, as most people know, is a standard (tags-based) markup language used in document and message definition and processing. In addition, XML serves as the language base (format and syntax) for many special purpose languages such as WSDL (Web Services Description Language), and RSS (Really Simple Syndication).
BPEL follows this same model. The BPEL process description is encoded using XML language constructs. BPEL also incorporates XPATH to write expressions and queries.
Web Services Specifications. Web Services are important to BPEL in two ways. First, BPEL processes follow the WSDL service model—which is why BPEL processes are implemented as services. More specifically, BPEL processes have a WSDL definition, and are implemented as Web Services (WSDL, SOAP, UDDI).
Along with the basic functions of service definition, discovery and invocation, the Web Service implementation allows BPEL execution engines to leverage additional Web Services standards at runtime, such as WS-Addressing and WS-Coordination.
Second, the collaborating services are also Web Services, described in WSDL. So, BPEL processes and collaborators are Web Services, and BPEL execution engines contain (and use) Web Services
runtime components. (Natively, the collaborating services (and the process service) are Web Services. However, BPELJ, an extension of BPEL, allows for a mix of Java Services and Web Services.
What Is BPEL Good For?
So far, this has been fairly technical, talking service orchestration, business process services, interaction models and technology bases. Before going much deeper, it is important to understand the best uses (and limitations) of BPEL.
BPEL and Service Orchestration. BPEL offers a nice model to abstract orchestration logic from the participating services, and configuration using BPEL over (hard core) coding of service interactions is enticing. However, there is processing overhead and infrastructure expense, so BPEL might not be the best choice for simple orchestrations. As a rule of thumb, a simple orchestration is comprised of two to five services and has static interaction patterns.
BPEL AND PROCESSES. As a language to develop processes, BPEL is good at executing a series of activities, which occur over time, and interact with internal and external services. These processes may represent IT scenarios, such as integration, or business scenarios, such as information exchange, or flows of work.
As for limitations, BPEL does not account for humans in a process, so BPEL doesn’t provide workflow—there are no concepts for roles, tasks and inboxes. In addition, BPEL does not support really complex business processes, those which evolve during their execution, branching out to incorporate new parties and activities. Lastly, BPEL does not have native support for business activity monitoring (BAM). There isn’t a data model for measurement and monitoring.
Of course, for every limitation there are industrious groups trying to resolve it. In late August 2005, IBM and SAP AG proposed an extension to BPEL (BPEL4People), to incorporate humans (initiators, approvers, managers, and task doers) in BPEL processes. This is an important issue to resolve, and I’ll be watching the BPEL4People proposal closely.
How Does BPEL Work?
[In this excerpt, I only cover the BPEL environment. In the report, I also speak to the Design to Execution Procedure]
[click ON Picture to enlarge]
BPEL Design, Development, and Deployment. At the far left of the diagram is the BPEL design, development, and deployment environment. Here are the tools for business analysts and developers/integrations to describe processes (orchestrations), as a series of steps (activities), associate services, and add any special business or validation logic to the process flow.
These tools vary from plug-ins for drawing tools or integrated development environments, to full-blown standalone environments. Most tools support a toggle between the graphical design view and the BPEL source code view. Changes made in one mode should be reflected in the other.
Supporting the design and development tools should be your service repository.
BPEL Execution. In the center of the diagram is the BPEL execution environment. The BPEL Server contains the executing process instances, a Web Services framework, and the BPEL execution services. The BPEL execution engine services collectively provide the process lifecycle requirements (instantiation, communication, dehydration/hydration, correlation, transaction management, compensation, and termination) as spelled out in the specification. Similar to J2EE containers, the engine implementations need to meet the specification requirements, but how that is accomplished is up to the individual BPEL engine providers.
To the right of the BPEL server, I show the Process DB used to hold information on process instances and state, and the BPEL server’s management console.
I drew the BPEL server simply, in alignment with the specification; however, implementations vary, with vendors adding features for non-Web Service invocations, and business performance reporting. As with any technology, be sure you ascertain which functions are provided as part of the specification, and what are vendor-supplied (proprietary) extensions.
Underneath the BPEL server, I show four different host environment options: J2EE container, .Net container, Java Business Integration (JBI) container, and an enterprise service bus. Example (not endorsements) implementations for
each host environment (respectively) are: Oracle’s BPEL Process Manager, Microsoft’s BizTalk Server, FiveSight Technologies’ PXE, and CapeClear’s ESB.
Above the BPEL server, I show the client service that initiates the process, and the collaborating (partner) services.
Bridge To Execution. The hand-off from development to execution contains the BPEL process definition, the WSDL of the BPEL process, and all engine-specific deployment files.
For more on BPEL, check out the OASIS cover page, play a little – I started with this, or read my full report 🙂
Last week I was on vacation, and this week I’ve been desperately trying to catch up, as well as taking some briefings in advance of what appears to be a big month of tech announcements in September. Obviously, I can’t speak to the briefings
(yet), but one of the things I’ve been catching up on is the open source enterprise service bus (ESB) release announcement from ServiceMix, and the ESB-like Apache Synapse Web Services Mediation Framework project announcement.
For those that are counting, this brings the number of open source ESB (and like) announcements this summer to five. Earlier announcements came from Iona (Celtix project), Sun (Java System Enterprise Service Bus (Java ESB)), and Mule.
Open Source in the Enterprise
I haven’t spoken (here) at length about open source, but in brief, I believe open source absolutely has a place in the enterprise; and that enterprises have some responsibility to contribute back (resources, code) to the open source community. Of course, on both ends, you need to be smart. Understand the implications of licensing, IP (yours and theirs), support (yours and theirs), code quality, security and stability, project adoption and longevity, and total cost – because nothing is really free.
That said (more another time); a great (no-brainer) use of open source is in experimenting with new technology. In almost all of my ESB writings for PSGroup, I encourage our clients to “play” with open source solutions, to get a feel for
the technology, and to contemplate technology (not necessarily product) fit for their businesses.
One of the open source ESB solutions I’ve mentioned is ServiceMix, the project backed by LogicBlaze. Last week, ServiceMix announced the first major release of their open source ESB, which is based on the JBI specification. Let me emphasize the word release, as in there is real software to download and try out.
The Key Technical Features (taken from ServiceMix materials):
- Normalized Message Service and Router
- JBI Management MBeans
- Ant Tasks for management and installation of components
- Support for the JBI deployment units with hot-deployment of JBI components
Service components (Service Engines):
- Rules-based routing via the Drools rule engine.
- BPEL support for WS-BPEL via PXE
- Cache, for caching service invocations using a Map cache or a JCache provider
- Groovy, for clean integration with the Groovy scripting language as a component, transformer or expression language
- JCA, which allows the Java Connector Architecture to be used for efficient thread pooling, transaction handling and consumption on JMS or other resource adapters
- Quartz, to support enterprise timer integration via the Quartz library
- RSS component for integrating and processing RSS news feeds in JBI
- Scripting support for any JSR 223 compliant scripting engine to be used to easily create a component, perform a transformation or be an expression language.
- Transformations using XSLT
- Validation, for schema validation of documents using JAXP 1.3 and XMLSchema or RelaxNG
- XPath and XSLT based routing and transformation engine.
- XSQL for working with SQL and XML via Oracle’s XSQL library
- ActiveSOAP, the lightweight StAX-based SOAP stack
- Reflection, to allow POJOs to be deployed in ServiceMix
- SAAJ for Soap With Attachments and Apache Axis support
- XFire for clean integration with POJOs via the XFire SOAP stack
- WSIF for integration with the Apache Web Service Invocation Framework
- Email support via
- File-based components for writing messages to files and polling directories and sending files into the JBI
- FTP support via the Jakarta Commons Net library
- HTTP, for both client-side GET/POST with the Jakarta Commons HttpClient, and server side processing with servlets or Jetty
- Jabber to provide bindings to Jabber network via the XMPP protocol
- JMS via the Java Messaging Service plus highly reliable and scalable transports in ActiveMQ, including persistence, recovery and transaction support.
- VFS via the Jakarta Commons Net library, which provides access to file systems, jars/zips/bzip2, temporary files, WebDAV, Samba (CIFS), HTTP, HTTPS, FTP and SFTP, among others.
As you can see, ServiceMix includes many important ESB features/functions. One optional feature that I was happy to see is the RSS component for integrating and processing RSS feeds. On (more than) a few occasions, I’ve written/spoken to the role of RSS for individual subscriptions in the enterprise, especially for dissemination of business and system events. For me, SOA, event-driven architecture (EDA) and integration are all inter-related, and this services-oriented integration solution, with capability for RSS-based event syndication is a good example of things to come.
I haven’t taken the product for a spin yet so, but fully intend to, and will write about my experience. I suggest readers do the same. As Robert Strachan, ServiceMix’s project architect, points out in the product announcement:
“What we’re saying with this release is, you could go online and read what a lot of people are saying about JBI, or you can download ServiceMix and get hands-on with JBI right now…”.
…of course I also advocate mixing in some reading!
Apache Synapse – A Web Services Mediation Framework (ESB-like)
The Apache Synapse project announcement generated a lot of conversation last week, but at the core, is a straightforward goal. The project intends to provide an implementation of a distributed services mediation framework, which is based on Web Services specifications. The specifications at the center of the framework are SOAP, WSDL, WS-Addressing, WS-Policy, WS-Security and WS-ReliableMessaging.
The distributed services mediation framework implementation aims (from the project proposal) are as follows:
- a messaging and communications infrastructure;
- based on the principles of service oriented architectures;
- that provides a way of connecting disparate systems;
- that manages the connections and the routing of messages between systems;
- that intermediates and transforms messages and service interactions independently of the endpoint applications;
- that is neutral with respect to the languages and platforms that are integrated – providing first class support for C/C++/COBOL/Java/.NET and other application platforms.
Why is this needed? Well, to achieve enterprise scale SOA (reuse and loose coupling) you often need to employ non-invasive middlemen between requestors and services for routing, transformation, security, and management
tasks. Without mediators, unnecessary complexity would have to be built into requestors and services, which defeats the purpose of SOA.
Why is this project so interesting? First, it’s Apache, and it is being built to work with Apache Axis 2.0, the next generation of the hugely popular (and successful) Apache Web Services implementation. Second, as Phil Waineright of Loosely Coupled points out here, the implementation in C will be designed to take full advantage of the Linux environment, and should result in an “Indigo for Linux”. Third, enterprise developers will have a standard Web Services mediation framework to use in sandbox and development environments. Fourth, vendors will be able to implement that same mediation framework in robust SOA environment and integration solutions (Fabrics, Web Services Management, ESB, JBI containers etc.), which enterprise developers can use in test, quality and production environments.
Why all the conversation? The name: Web Services Mediation Framework rather than ESB, or services broker. (Honestly, who cares as long as the right functions are there, and it is standards-based.) The code base: The code is being donated by vendors, rather than originating in the open source community. The players: The vendors have good mind share in their spaces (Blue Titan, Infravio, Iona, and Sonic Software) but aren’t industry giants. The driver, WSO2, while long active in Apache, is a startup. And, the lack of detail: What exactly do they mean by “mediation”, what does JBI support as an add-on mean?
What now? Well, it is early. It only makes sense there are more questions than answers. So, for the majority, Synapse is something to watch. For those that are passionate (with the proper expertise), you might want to get involved. If you have current SOA and integration projects, continue to pursue existing solutions that fit into your business and architecture, but (as always) keep an eye forward.
Speaking of Getting Involved…Breaking News – Impending Alliance of ServiceMix, Apache Synapse and Celtix
As I was researching and writing today, I saw this article on c|net that speaks to an impending alliance of three open source integration projects: ServiceMix, Celtix and Synapse. A snippet from the article:
Those involved in talks told CNET News.com that the partnership calls for close technical ties and code sharing among ServiceMix, Apache Synapse and Celtix, which is hosted at France-based consortium ObjectWeb.
The goal of the planned alliance, the sources said, is to create a more cohesive integration offering and attract software developers in the increasingly cluttered field of open source, where new projects seem to appear weekly.
According to c|net, an official announcement should be forthcoming in September. Digging a little deeper, I saw this on the Apache mail archives that James Strachan of ServiceMix approached Sanjiva Weerawarana of Apache Synapse to
collaborate on making Synapse fully JBI-compliant. Perhaps that covers the JBI add-on question.
In the long run, cooperation is good, for the projects, and the potential consumers. But as we all know from our own project work, merging of ideas and parties, typically slows things down.
So, play with ServiceMix and watch Synapse. At least, that’s my plan.