Using Analytics to Modify In-Flight Processes

I recently had the pleasure of presenting the Oracle Key Note at the Butler Business Process Management & Integration symposium. The subject of the keynote was to look at how we can use business analytics to enable us modify processes already in-flight.

When you consider the traditional closed loop BPM Lifecycle, as illustrated below, the emphasis has always been very much on using analytics about how processes have performed in the past, in order that we can modify the actual process definition in order to improve / optimize future versions of the process.



Whilst this approach has many benefits, the challenge was how we could modify processes already in-flight. In order to achieve this we need to overcome a number of challenges; firstly we need to collect analytics in (near) real time in order to base our decisions on up to date information; secondly we need a controlled way of modifying the in-flight process based on the data.

Collecting Real Time Business Analytics

Business Activity Monitoring (BAM) provides us with the tool to collect the near time analytics on which we can base our decision on how we wish to modify our in-flight processes.

For those of you who are not familiar with Oracle BAM, it enables the business to gain a real-time view of what’s happening with the business. To achieve this it provides the following key components:

• Capture Real Time Data - Within a BPEL process you can place a sensor on any activity with a process, when triggered this generates an event which is picked up by BAM in real time.
  
  Note: You can actually use events generated pretty much by any event based system (e.g. database triggers, messaging systems, etc), it’s just that BPEL makes it very easy.
  
• Analyze Processes, Trends, and Context - Next Oracle BAM is able to correlate all these events and synthesize them into meaningful data objects within it’s active data cache
  
• Interface for Business Users - The business can then build interactive real-time dashboards and proactive alerts on top of this active data to enable them to monitor the business processes.

For more information on BAM see http://otn.oracle.com/bam.

Modifying In-Flight Processes

Once we have the real time analytics, the next challenge here is to use the data to modify the processes already in flight.

Now when we talk about modifying In-Flight Processes; most people assume that this involves quickly writing a new version of the process, testing it, deploy it, and then migrating the existing in-flight processes to this new improved version. The reality is that is rarely as quick as required!!!

Rather what’s really required from a business perspective is to be able to modify the flow through a process (and its sub-processes) in order to obtain the desired business outcome. There are three basic patterns here which provide a way of achieving this, these are:

• Modify Process Flow – This uses business rules, which can be modified externally to the process to change the flow through a process instance.
• Exception Management – This uses BAM to launch a process to manage an exception which is (typically) occurring across multiple processes.
• Dynamic Process Assembly – Here we use BAM as source of real time data against which to evaluate Business Rules and use the results to dynamically call sub processes and assemble the end to end process on the fly.

Modify Process Flow

With this approach we are not looking to modify the actual process, rather modify the “path” through the process. If we look at any process, then at various points the process will hit a decision point which will determine which route it should take the through the process. Rather than building these decision points directly into the process we can externalise them in a rules engine such as Oracle Business Rules or iLog JRules, as illustrated below.



This then enables the business to modify the rules based on what’s currently happening within the business (as indicated through BAM) and thus get the process to take a different route.

An excellent case study for this is Cattles (a UK company which lends money to the secondary market), which uses a combination of BPEL, BAM and Rules for this purpose.

Exception Management

BPEL already provides a comprehensive way to handle exceptions, within the context of a process. However on some occasions we want to take a more holistic view to managing exceptions. This is certainly the case where we are suddenly “hitting” a common exception across multiple processes.

For example, if we have a loan flow process that’s going out to an external credit rating agency, and for some reason that credit rating check fails. Within the individual process BPEL process, we could have built in a simple re-try mechanism that simply waits a period of time before re-trying the service.

However if we start having a high number of failures (e.g. we may 1000’s of these processes running at any time), rather than handle the same exception multiple times (at least the same root cause) we can use BAM to detect that we have an issue and kick-off a single process to handle that exception (as illustrated below).

Dynamic Process Assembly

Rather than use BAM to feed a Dashboard, we can use it as a real time data source. A BPEL process can then query this real time data and pass this to the rules engine; enabling us to evaluate rules based on real time data.
At this point we could just use the result from the rules engine to dictate the flow through a process (as we did with the first example – Modify Process Flow).

However (as the title implies) we can take it a step further and dynamically assemble the process. The trick here is to have multiple “sub” processes all with the same WSDL definition. The rules engine rather than returning a “decision” now returns the end-point of the sub process to call, which the main BPEL process can then call dynamically (as illustrated below).



For more details on how to implement dynamic routing, see my previous blog; Using BPEL to Implement Dynamic Content Based Routing.

Final Thoughts

You can download a pdf of the full presentation from here. In the mean-time I would be curious to learn of any other strategies that are being adopted for modifying in-flight processes.