Thursday, 26 July 2012

Scheduling a Web Service within a SOA Composite


We get a lot of questions from customers about how to schedule a BPEL process or web service to run at regular intervals. For example, how to schedule an account billing composite which is required to be run once every night.

Whilst the Oracle SOA Suite doesn’t provide a native scheduling component; Rubicon Red provides an extension to the Oracle SOA Suite (and Oracle BPM Suite) that allows you to schedule the regular invocation of a web service or the publication of an EDN Event.

The Rubicon Red Scheduler provides a Web Service API, which allows you to schedule the execution of a service from another service, such a composite. Once you can do this, it introduces a number of interesting patterns that can be implemented using the Oracle SOA Suite.

For example, we may have a customer satisfaction process, which we want to execute one month after the customer received delivery of ordered goods. Alternatively, we may want to schedule a repeating process for a specific period of time; for example to track the status of a shipped item until it is delivered.

Whilst this can be done using BPEL, this can cause issues when managing the dehydration store, often a better approach is to get the BPEL process to create a scheduled job to manage this separately. In this blog we cover how to schedule a web service from within a BPEL process. We will use this as a foundation for a series of future blogs to cover some of the patterns that this allows.

How to do it...

To configure a scheduled job via the web service api, we will need the WSDL for the Scheduler, this can be found at: http://localhost:7001/RXRScheduler_2.0/util.sch.evs.Job?WSDL

For our purpose, we will be configuring the scheduler to invoke our ParcelTracker process (this will be the subject of a future blog), so to follow the example you will need to download and deploy the ParcelTracker composite contained within the example code for this blog (code sample coming soon).Once deployed you will need the WSDL for the Parcel Tracker, this should be available at: http://localhost:7001/soa-infra/services/default/ParcelTracker/ParcelTrackerService?WSDL

Create a SOA Application, with a project containing an empty composite (named ScheduleParcelTracker in the example below). The ScheduleParcelTracker is designed to receive a request containing the orderNo for a parcel to be tracked and creating a scheduled job to invoke the ParcelTracker every 15 seconds to track the status of the specified order. 
Drag a Web Service from the SOA Component Pallet onto the External References swimlane within our composite. This will launch the Create Web Service wizard, specify Scheduler as the name and for the WSDL URL enter the location of the Scheduler WSDL.

Ensure the Port Type is util.sch.eve.Job and click OK. JDeveloper will add a reference to the Scheduler to our composite. Next drag a wire from the BPEL process to the Scheduler external reference.




Open the ScheduleParcelTracker BPEL process and rename the default inputVariable to orderNo.

Next drag an Invoke activity onto our BPEL Process; double click on it to open the Edit Invoke window. Give it the name putJob, for the Partner Link select Scheduler, and select the operation putJob. For the input variable, click on the auto-create variable (green plus icon) to launch the Create Variable window; give the variable a meaningful name (e.g. putJobInput).  Do the same for the output variable.

Drag an Assign activity onto our BPEL Process, just before the Invoke activity; double click on it to open the Edit Assign window; give it the name setPutJob.

First we need to populate the job element with the unique identity of our job; this is a composite key defined by the elements jobId and jobGroup.  Use the assign activity to set jobId to contain the orderNo of the parcel being tracked and the jobGroup to ParcelTracker (the name of our process).


We have also set jobDescription to hold details of the parcel being tracked. 

Next we need to specify the schedule for when the job will be run. For the purpose of this blog we will use a basic cron schedule to run our job every 15 seconds. Use the assign activity to specify the following values for the job.

Job element Source
ns9:startDatexp20:current-date()
ns9:endDateXp20:add-dayTimeDuration-to-dateTime(xp20:current-date(), ‘P14D’)
ns9:active true()
ns9:jobDefinition/ns9:cronSchedule '0/15 * * * * ?'

We have specified the job will only be active from the current date up to 14 days into the future (this is optional – but will prevent the job running for ever in case we forget to cancel it), and set the job to be active (otherwise it won’t run).

Next we need to initialize the Job Definition element, which specifies the web service to be invoked and the content of the payload to be passed. Use the assign activity to initialize the attributes of the job in the element $putJob.payload/ns8:body/ns9:job/ns9:jobDefinition, as detailed within the following table:

Element Value
@jobClass‘WebService’
@responseInterface'One-Way'

We also need to use the assign activity to initialize the content of the element ns9:jobDefinition/ns9:webServiceJobDefinition, as detailed in the following table:

Element Value
ns9:service/ns9:URI'http://rubiconred.com/ckbk/svc/ParcelTrackerService'
ns9:service/ns9:localName'ParcelTrackerService'
ns9:port/ns9:URI 'http://rubiconred.com/ckbk/svc/ParcelTrackerService'
ns9:port/ns9:localName 'ParcelTrackerService_pt'
ns9:endpointAddress 'http://localhost:7001/soa-infra/services/default/ParcelTracker/ParcelTrackerService'

We have now configured the scheduler to send a correctly formed SOAP message to a web service endpoint; the next step is to provide the payload to put in the message.
The putJob element contains an element called soapRequestBody which is defined as xs:anyType. This is where we specify the request message to send to ParcelTracker when the scheduler invokes it.

To do this, we must create a variable of the same type as the message to be sent to ParcelTracker; this is defined in ParcelTrackerService_1.0.wsdl.

Create a new global variable, named syncParcelLocation. Select Message Type as the variable type, and click on Browse Message Types … to open the Type Chooser. From here select Import WSDL file and locate the Parcel Tracker Service WSDL file; ensure copy to project is selected and click OK.
Within the Type Chooser, expand the ParcelTrackerService_1.0.wsdl and select the Message Type syncParcelLocation.


syncParcelLocation contains the element orderNo, set this to be the orderNo contained within the variable used to invoke our BPEL Process. Next, within the Assign activity use an Append rule to copy the entire syncParcelLocation message into the soapRequestBody.


When creating the mapping, you may have noticed a couple of choice elements, for example simpleSchedule or cronsSchedule, and ednJobDefinition or webServiceDefinition. By default, at run-time BPEL will create empty elements for these alternative choices, so we need to remove them to produce a valid message. In the Assign activity, select the simpleSchedule element, right click and select remove. 



Repeat this step for the element ednJobDefinition. Now our process is complete so deploy it and run it to see the created job in action. When our scheduleParcelTracker composite is executed it invokes the putJob operation against the Scheduler, the content of the soap message should look something like the following: 
This will cause the Scheduler to invoke ParcelTracker every 15 seconds.  If you log in Enterprise Manager you should thebe able to see a list of instances of the Parcel Tracker being started every 15 seconds.

If we click on any of the ParcelTracker instances we’ll see the audit trail for a single instance of this repeatedly executed process.





Saturday, 16 October 2010

OTN Podcast

About a month ago Antony Reynolds and myself were interviewed for ArchBeat which provides an online forum for the Oracle Technology Network (OTN) Architecture Community. The interview was carried out by Bob Rhubart around our latest book the Oracle SOA Suite 11gR1 by Bob Rhubart.

The interview not only covers the book, but takes an opportunity to examine how things have changed in SOA since we released our original book covering the 10g platform. In addition to covering what we believe are the key changes in the Oracle SOA Suite, we also give our views on how SOA has matured as an approach as well as its adoption by customers.

The first part of this interview, has now been released as a pod cast and can be found here. Have a listen see what you think, and see whether you agree with Antony and myself!

Friday, 16 July 2010

SOA Suite 11gR1 Developer’s Guide Published!

Well I'm both delighted (and relieved) to announce the publication of the 11gR1 version of the SOA Suite Developer’s Guide. As with the original version, this was jointly written by Antony Reynolds and myself; in fact in many ways this is the book that Antony and I originally intended to write, when we first put pen to paper (or finger to keypad) back in May 2007. At this point the 11gR1 version of the Oracle SOA Suite was still in the initial stages of development, with the ‘goal’ being to time the publication of the book with the release of 11gR1.

Then in early 2008 Oracle announced the acquisition of BEA, which it finalized in July; at this point future timings around the release of 11gR1 were very much up in the air. So we re-focused the book on 10gR3, of course no sooner had this been published and Oracle released 11gR1! So after a brief pause to catch our breath and restore our sanity, Anthony and I pulled out the original manuscripts and started again.

As with most books, there is a huge amount of support and effort behind the scenes. We had a great team of reviewers who provided invaluable feedback and encouragement:

In addition we had great support from the SOA product management team at Oracle, and the editorial team at Packt Publishing who played a key role in keeping us on schedule!

The new book contains approximately 40% new content and is slightly larger than the original. But we made a very conscious effort to try and stop the book from getting to big, so as we introduced new content, we tried to condense (and in some cases remove) material in other areas. We hope we have the balance right, and that readers will get as much (or more) benefit from this version as they did the last.

Monday, 6 July 2009

Downloading the SOA Design Time for JDeveloper 11G

Oracle Fusion Middleware 11G Release 1 was officially launched by Oracle on the 1st July 2009; included in this release are the 11G version of WebLogic, SOA Suite, WebCenter and Identity Management. You can download all of the required components from http://www.oracle.com/technology/software/products/middleware/index.html.

Installing the Oracle SOA Suite is pretty straight forward; however the one gotcha is the install of JDeveloper 11G does NOT install the SOA Design Time. This is not really an issue as JDeveloper automatically checks for updates (including the SOA Design Time) and will prompt you to download and install them (alternatively select Help->Check for Updates).

However the SOA Design Time is 200+ MB in size. So not a trivial download, particularly if you are installing JDeveloper in multiple environments. Ideally it would be nice to be able to download a local copy of the SOA Design time extension which can then be installed in each environment that is required. Well fortunately you can do this by going to the Oracle JDeveloper product updated center shown below. From here click on the download link for the SOA Oracle Composite Editor (circled below).



To install the design time, within JDeveloper select Help->Check for Updates. Click Next and Select Install from Local File and browse to where you saved soa-jdev-extension.zip. Click Open, click Next and then Finish. JDeveloper will then install the extension; finally restart JDeveloper when prompted.

Saturday, 13 June 2009

Arch2Arch Podcasts

The Oracle Technology Network Arch2Arch Podcast brings together architects and other experts from across the Oracle community and beyond to discuss the issues, tools, and technologies that are a daily part of the software architect's ever-changing world. I've always enjoyed these podcasts and find them an extremely useful and convenient source of information.

Their latest podcast, is part one of a two part interview by Bob Rhubart (the host of Arch2Arch) with Antony Reynolds and myself about our book the Oracle SOA Suite Developer's Guide. The interview is truly international with Bob carrying out the interview based in California, Antony in the UK and myself from a hotel room in Sydney, Australia (about 8am in the morning).

It's quite strange hearing yourself speak, but what struck me is how English Antony sounded! Have a listen and see what you think (and whether I’ve picked up an Aussie twang) and whilst you’re their check out some of the other excellent podcasts as well.

Saturday, 18 April 2009

Top 5 Insights for Maximizing Returns with SOA

Oracle are hosting a live webcast on some of the key insights gained by customers who have successfully implemented SOA based solutions within their organization.

They will be covering areas such as; building a business case for SOA, strategies for adopting SOA, critical success factors as well as some of the efficiency drivers and cost savings achieved through the deployment of SOA based solutions.

Participating executives include:
  • Job Simon, Senior Director, NetApp
  • Dan Goerdt, Director, Schneider National Inc.
  • Jennifer Briscoe, CTO and VP, Collect America
This is an excellent opportunity to gain some valuable insights into how you can leverage SOA within your own organization.

The webcast is scheduled for April 23 8:00 a.m. PT / 11:00 am ET / 4.00 pm GMT. To register click
here.

Saturday, 11 April 2009

Writing a 'Singleton' BPEL Process

A typical use case in BPEL is connecting to an external resource / system which only support a single connection at a time. In such a case we need to protect against parallel BPEL instances submitting concurrent requests to that resource.

This implies that we need some way to serialize requests for that resource (probably on a first in first out basis). A common design pattern for achieving this is the Singleton, first documented in the Gang of Four's patterns book (where they use a print spooler as an example).

Now BPEL doesn’t explicitly support the notion of a Singleton, however it does allow you to simulate one, which for the purpose of what we are trying to achieve is good enough.

The basic idea is to have an explicit initiateSingleton operation which is used to instantiate a single process instance, and then have a separate invokeSingleton operation which is used to submit each actual request. The invoke operation is then placed in a loop, so as soon as it’s processed one request it loops back round for the next one.

For this example, I’ve just created a simple process which keeps track of the number of times it’s been called. It takes a single parameter, name and returns a string of the following format:

Hello <name>. You are caller number <count> of this singleton.

Pretty simple, but it illustrates the point. The basic process is shown in the screenshot below:


The first activity in the process is a receive activity, InitiateSingleton, which is used to instantiate the process. We then have a simple assign which just sets the count to zero. The remainder of the process, which handles the invokeSingleton operation is contained within the ProcessRequest scope, which itself is placed inside a while loop, that loops for ever. This contains the following activities:
  1. InvokeSingleton – This receives the next request to be processed by the singleton.
  2. Wait – This waits for a period of 5 seconds. It is just to simulate going out to an external system. It also enables us to see how multiple requests for the singleton are queued up by the Oracle BPEL PM Server.
  3. SetResult – Here we just set the result and increment the count by one.
  4. CallbackClient – Finally we just send back the result to the client; before looping back round to process the next request.

Message Correlation

At first glance this all looks deceptively simple, however the complexity comes when you try and call the invokeSingleton operation.

The reason is that when we invoke an operation on a BPEL Process, the message is received by the Oracle BPEL PM Message Handler, depending on the operation it will either invoke a new process to handle it (as with the initiateSingleton operation) or route it through to the appropriate instance of an already running process.

This is where it gets a bit convoluted, when we receive the message for the invokeSingleton operation, even though we only have a single instance of the BPEL process running how does BPEL PM know that it is the correct instance to route the message to? The answer is it doesn’t.

Now under “normal” circumstances, where we have an asynchronous interaction between two processes, Oracle BPEL PM defaults to using WS-Addressing to ensure that messages are successfully routed between two process instances. However in these cases, it’s typical that the interaction between the two processes (say A and B) began by process A initiating process B and passing it details (using WS-Addressing) about itself so that when process B sends back a response it contains sufficient information for it to be routed to the appropriate instance of process A.

However with our Singleton process, any process calling the invokeSingleton operation will have no prior knowledge of that process, so for our purposes we can’t use WS-Addressing to correlate the message.

Fortunately for situations where WS-Addressing isn’t appropriate or available BPEL provides the concept of correlation sets. Essentially correlation sets allow you to use a unique value present in the message body of all exchanged messages (e.g. orderId) to link that exchange of related messages together.

You do this in BPEL by first defining a property which corresponds to the unique value, and then defining a property alias for each message in the exchange, that defines where that property is located within the message. Next you define a correlation set which can consist of one or more properties (see the product documentation or the Oracle SOA Suite Developer's Guide for more details on how to create Correlation Sets).

For our purpose, I’ve defined a simple element called singletonId which is contained in both the initiateSingleton and invokeSingleton operations.

Next we have defined a correlation set SingletonCS which is used in the InitiateSingleton and InvokeSingelton operations.

On the initiateSingleton operation I’ve defined this to initiate the correlation set (see screenshot below), this means whatever value is contained with the singletonId in the start operation must be present in the invokeSingleton operation for it to be routed through to this process instance.


The next part of the problem is to return the response back to the appropriate caller of the process. This at first may seem strange, but recall we may have several processes calling the singleton at the same time, and thus all waiting for a response. We need to ensure that each response is returned to the appropriate instance.

Now you’ve probably already guessed that we need to use correlation sets again (as we have disabled WS-Addressing for this Partner Link). This time the calling process will need to generate a unique key that it passes in the request (requestId) to the singleton. The singleton will then return this in the response to the requester.

If we look at the SingeltonAccessor process, we use the XPath operator generateGUID to generate this value.

Specifying the Reply to Address

So now everything looks fine, so we can go ahead and run the process; well not quite! If you do you will notice that request from the SingeltonAccessor successfully reaches our Singleton process. But for some reason the response from the Singleton never reaches the SingeltonAccessor, in fact if you take a closer examination of the Singleton Audit trail for the callback activity you will see that it actually skipped the callback!

Now it turns out that this is quite a rational behaviour for the simple reason that the Singleton doesn’t actually know where to send its callback to. This is because at design time the caller of an asynchronous process typically isn’t known, and thus the callback address needs to be specified at runt time. In fact if you log into the BPEL Console and examine the WSDL for the call back operation, you will see that the soap:address location for the endpoint is defined as:

http://set.by.caller

Now by default Oracle BPEL PM uses WS-Addressing to handle this, thus when I invoke an asynchronous process, part of the SOAP payload contains a return address for the asynchronous process, however we’ve just disabled WS-Addressing for this message exchange as we are using correlation sets.

So how do we provide the reply to address? Well one way would be to pass this in as part of the request message and then use dynamic partner links to set the replyToAddress.

However a simpler way is to define the replyToAddress property on the partnerlink to point to the reply address. This takes the format:

<wsdl endpoint>/partnerLinkTypeName/[rollName]

So in our case:

http://server:port/orabpel/default/SingletonAccessor/1.0/Singleton/SingletonRequester

Now if we deploy this it should finally work. To run the example, first go into the BPEL Console and initiate the Singleton process (set Singleton Id to "Singleton/1.0").

From the console initiate the SingletonAccessor entering whatever value you like for the name. You could kick off a number of these to see what happens.

Loose Ends

So that all works, however there are two basic problems with this approach:
  • Firstly to invoke the Singleton you need to know the value of the SingletonId. This in our case is fixed within the SingltonAccessor process.
  • Secondly, the Singleton process will only ever return a result to the SingletonAccessor process (as this is fixed by specifying the replyToAddress). What if we want to call it from another process or web service client?
Actually the solution to this is very straight forward; you use the SingletonAccessor as the entry point for accessing the Singleton. Thus the real client will always call the SingletonAccessor.

Thus the client never needs to know the SingletonId, and secondly as WS-Addressing remains turned on between the “real” client and the SingletonAccesor the client doesn’t need to worry about correlation sets or specifying a replyToAddress.

One final question you may have is how do I prevent multiple instances of the Singleton process being kicked-off by mistake? Well why not try it? You will see that as long as you specify the same value for the SingletonId, BPEL PM will throw a “Conflicting Receive” fault as an equivalent receive activity has already been enabled by the original Singleton.

Conclusion


As we can see, implementing a Singleton within BPEL is relatively straight forward. Whilst this is not a totally clean solution, it does achieve the desired effect.

Click here to download working example.