If a soda can falls off a pallet in a warehouse, should it be considered a business event?

The answer is yes; this is an event, but only one of hundreds of thousands — or even millions — that may typically occur across an enterprise every day. The trick is being able to determine what particular events make a material difference to the business, and then applying analytics against those more important events.

I recently had the opportunity to join two experts in the complex event-processing arena — Brenda Michelson and David Olson – in a thoughtful online roundtable discussion on how business events need to be processed, and the role of event processing in what has been a very eventful year for businesses.

As the world progresses toward complex event processing – and it will take time – the role of enterprise data integration will grow even more critical. The ability to sense and respond in near real-time fashion to events across the organization will require a robust and infallible data integration infrastructure, that delivers quality data to any part of the enterprise that needs it.

A soda can shifting off a pallet obviously is not a big event that needs to be monitored and analyzed. But something like the collapse of credit markets is a significant event that has huge ramifications for a business. Recognizing important events from the “event cloud” that rolls through every day is an artful science known as “complex event processing,” or CEP.

CEP is not a black-and-white exercise, as there is a wide spectrum of events that may or may not have an impact. Some events that may even be hidden or innocuous may have long-term impacts. The trick is identifying those events that need to be tracked and understood. For example, sensors in a production facility may sense errant patterns in output that have matched the lead-up to shutdowns in the past. The CEP engine may proactively notify facility managers via alerts, as well as sales managers downstream that delays could be imminent.

A little while back at a previous session, Dr. David Luckham, considered the father of complex event processing, responded to a question on what an example of a “complex event” may be. His answer: World War I.  The war was a complex event, with many different and unseen components that converged, and had a lasting impact to this very day.

Organizations may not be dealing with World Wars, but still are nonetheless complex organisms attempting to survive and thrive in complex environments. There are many things going on at once, with many different relationships, with many different impacts – again, coming out of the event cloud.

From a management perspective, the most important part of the CEP exercise is developing a capability to identify both threats and opportunities among the voluminous streams of data coming in from internal and external sources. This is where business intelligence comes into the picture. Brenda observed that data flowing in from across the enterprise – from sensors, from applications, and from external sources – is part of “one large flow,” of which essential pieces need to be captured and subjected to analysis. The business needs, through rules development, to determine what events are and aren’t important – and when an insignificant event suddenly becomes a significant one.

“At the edge, where the events are being generated, and specifically, with RFID, if you think every time the can of soda moves on the pallet in the warehouse, events are generated,” she states. “So you do need to filter some of those at the edge. And there will be business rules as well applied at the edge. So there is some type of analytic processing in there. Maybe it says, ‘I only really care when a high-volume item leaves the warehouse. I want to be notified when the pallet of TVs goes out, make sure I know about it. The pallet of soda I don’t care about as much.’”

Why does business intelligence need complex event processing behind it? CEP provides a real-time snapshot of developments as they occur across the enterprise, and elevates these views to decision makers. Event processing, in conjunction with business intelligence, enables decision makers to expand their knowledge relating to hundreds of thousands of events that are flowing in and out of most businesses on a daily basis. This means more than implementing a dashboard or decision algorithm, Brenda said.

“Think about it in the context of information flow, and the patterns that emerge from this flow. Analytics and business intelligence is often about making informed decisions for your business tomorrow. What you’re really doing is you’re trying to make informed actions for your business right now.”