Archives for September 2008

Before I fully immerse myself in the SOA Consortium this week, I wanted to augment my “reporting” on last week’s EPTS symposium with a little analysis.

My EPTS Take-Aways

True Industry Collaboration – As Opher mentioned, 41 of the 70 attendees were event processing vendors, representing a many of the technology providers in the space: Aleri, Coral8, Event Zero, IBM, iLog, Nastel, Oracle, Progress Apama, RuleML, Streambase, Tibco, and West Global.  What impressed me is these vendors attended the symposium not to one up each other, but to work together to move both the technology and market forward.  While I didn’t attend the follow-on business meeting, I learned that several new working groups have been proposed to explore important topics including event processing languages, reference architectures, benchmarks, university curriculums and interoperability.  These are in addition to the current EP glossary and use case efforts.

Cross Industry Applicability – While Capital Markets and “secret agencies” are fertile grounds for event processing applications, the market doesn’t begin and end there.  Many participants, myself included, brought forward event processing examples related to business intelligence, business visibility & responsiveness, passenger transportation, freight & logistics, information dissemination, healthcare (epidemic, patient monitoring, RFID in-body monitoring), ambient intelligence (smart homes, smart cities, personalized mobile information systems), workplace safety, homeland security (storm evacuation, first responder systems, terrorism situational awareness, power / fuel / water system monitoring, and environmental systems.

Hype is Relatively Low – I made this point in my panel presentation, and frankly, I think it surprised many of the attendees who are immersed in event processing, and therefore partake in high, but localized hype.  So, for my practitioner readers, I apologize in advance for sparking a hype wave.  However, if the EPTS sticks to its mission, and truly collaborates on (as Seth Grimes mentioned) bringing some ‘precision’ to the space, then the hype could very well be productive, rather than destructive. 

Forthcoming Battles – Speaking of destructive, two battles are imminent.  One, the language wars: (a) SQL vs. procedural vs. Rules and (b) mine vs. yours.  Second, what are the boundaries of event processing?  Is information dissemination in or out?  Same with sensor applications?  Does it depend on the use of an event processing engine?  As Mani Chandy asked, is event processing the engine, or the result?

Early for Standards – While standards are good for practitioners and market adoption, standards for standards sake are a mistake.  And yes, I am thinking of the ‘early and often’ delivery of WS* standards as I type this.  So, in a way, I was relieved to hear that EPTS isn’t knocking down the doors of standards bodies with ‘must haves’.  However, I hope the group doesn’t back burner the standards conversation, because if they don’t take the lead, someone else might, and that could be troublesome for all parties.

 

[Disclosure: None of the vendors mentioned in this post are current clients of Elemental Links, but that can change 😉 ]

Standards keynote: The role and influence of standards Chris Ferris,IBM Distinguished Engineer, CTO Industry Software Standards, IBM Software Group

Objective to discuss standards in general, as informational to EP society as it considers standards.  As for EPTS’ role in standards:

“Work with existing standards development organizations such as OMG, OASIS and W3C to disseminate and where necessary incubate a set of consistent; standards in the areas of: event formats, event processing interoperability, event processing (meta) modeling and (meta) languages; EPTS does not plan to develop standards itself.”

Back to Chris’ talk:

Open Standards

• improve information sharing
• simplify integration of disparate technology
• promote interoperability

What is a meaningful gauge?

• Broad adoption
• public interfaces/public input
• path to long term stewardship

What to watch out for

• “based on standards”
• “leverages open standards”
• “standardized on a particular product”

Why are standards important?  For technology adopters, standards create choice, flexibility, speed, agility and skills.  “Open standards give us the most options and the best chance to gain real interoperability”.  “Taking care of your customers first will take care of your business!” (obligatory dig on vendors with “closed standards”)

Chris then shared IBM’s approach to standards development, from idea generation to innovation to standardization to ecosystem exploitation.  He provided industry standards development tips to EPTS members, which were helpful to the group, but not relevant to the majority of my blog readers. 

IBM notes the following standards as relevant to EP:

• WS-Eventing* 
• WS-Notification*
• StreamSQL

*the joint IBM-Microsoft work to bring WS-Eventing & WS-Notification together as WS-EventNotification (pdf) has stopped.  New WS-Eventing work has been proposed to W3C.

Jerry Baulier, CTO of Aleri

Current cutting edge work:

– Evolving authoring paradigms & languages / computational model

• convergence of relational data flow & rule based paradigms
• multiple authoring languages – SQL, Visual directed graphs, lower level textual
• enhancements to the relational paradigm to include pattern matching & procedural programmable streams
• increased state management constructs

– Hybrid Analytics

• real-time ep integrated with historical analytics
• extending into OLAP

– Visualization, Sensor data & authoring extensions

Mani Chandy, Cal Tech, sensor detection projects, are they also CEP?

• Nuclear detection – handheld sensor devices to detect dirty bomb prior to explosion
• predict shaking from earthquake – once it starts
  • crowd sourcing data collection, people attaching sensors to their computers at home;

Malcolm Lockhart, CTO, Event Zero

“Event processing is ‘Dessert topping and floor wax'”, many event processing domains:

• Industry – utilities and manufacturing
• Business class – BAM, BPM, MDM
• Speciality – intelligence and defense
• Future Internet – semantic web, massive pub-sub, content based networking

Event Processing Topologies:

• Monolithic CEP, few endpoints, single location
• Event Processing Network, many known endpoints — This is Today
• Event Cloud, unknown numbers of endpoints

John Bates, Progress Apama

– Monitor, analyze and act on everything.

– Event processing for “augmented life”, (graphic of windshield notifying driver of location based information)

– Event processing for “extended life” – RFID pill that updates on all human vitals, used during fun run; could extend to medication delivery

Technology Visions for EP

• new application paradigm
• Events as “first class” objects
• Event-driven SOA, millions of modules producing & consuming events
• real-time BI
• graphical programming
• voice-based adhoc programming
• next generation “pattern discovery”
• globally federated applications
• complex rules tracking millions of objects
• federated event schemas across application boundaries
• EP that “follows you in the cloud”

Albert Mavashev, Nastel, Applying CEP & Monitoring to Complex SOA

• CEP can observe how businesses actually behave, observe pattern of activity, determine if normal, or not, and act
• CEP in the cloud
• Virtual grid of event processing engines to deliver QOS

John Morril, Coral8

directions customers are dragging them into:

• More power, CEP GRIDs — Wall Street firm for risk management, risk as a service, (started a bit late)
• Sophistication requires increased control – stats, optimization, tuning
• deployability – managing a large network of CEP engines/applications
• user empowerment & integration – data streams, external apps and downstream tools – metadata exchange notion in adapters
  • example of metadata exchange with existing BI tools, leverage existing tools and reports

Q&A

1. Event processing/CEP as a service, real or work-in-progress?  Event processing platforms are work in progress.  But, event processed results are available as a service — airlines as an example.

2. How are vendors dealing with integration problems? John Bates – traditional adapters.  Need a semantic type integration to better inter-operate events and services. 

3. What is event processing? Mani Chandy – are we focusing on technology, or the results?  what do we care about most?  We, as EPTS.

4. What academic work needs to be done on GRID & Event Processing? Panelists & audience point to Google’s MapReduce as foundation.

5. Projects like Active Badge and protecting privacy?  John Bates was on that project, and mentioned how issues arose such as who should know how long was so-and-so in bathroom.  (no).  But, pervasiveness of cameras and sensors bring up need for new regulation.  Jerry Baulier, many good RFID based applications such as child protection and location.  Albert Mavashev, privacy issues increase as move to cloud (outside bounds of enterprise).  Malcolm Lockhart, spend a lot of time on event integrity, including who can use, store and see.

6. Unique challenges for security in CEP? Malcolm – describing security model is biggest challenge, access control per event, but it is data in motion, not at rest.

7. Should event processing engines stand alone? Or, will engines be embedded in other software (infrastructure or application)?  Panelists agree that hybrid model will exist.  Today, mostly stand alone, but embedded in some software such as systems management.

8. Event as first class citizen? John Bates, generally means not being an adjunct of databases.  Events can be generated, acted upon by any programmer or business user. 

9. David Luckham asks, “Is content based routing on the Internet (Internet scale) a feasible goal? Routing messages more efficiently, only get the messages that you want, brings precision.”  Not yet.