The topic-du-jour in BPM circles is the handling of human and automated decision making in processes. Two major areas that intersect here are the management of business rules (such as “customers with more than 100,000 frequent flier miles receive priority treatment when flights are oversold“) and business processes (such as “rebook voluntary denied boarding customer“). There has been plenty of work done in both domains, but until very recently they did not talk to each other very much.

That has changed quite rapidly, as the business rules community realized that it needed some ways to represent the structured order of long-running decision-making activities (as typically found in workflows), and as the process community realized that modeling decisions and rules using activity networks, BPMN, or Petri Nets results in rather bloated and complex diagrams.

On the research side, my colleague Marta Indulska from the University of Queensland and I have studied the expressiveness of process and rule modeling languages using representational analysis (i.e. we used a formal ontology as a benchmark) and found that the combination of process modeling and rule modeling languages generally offers higher expressive power than either of these languages by itself. We found the combination of BPMN and SRML particularly useful, but since SRML is an abandoned effort we would recommend the combination of BPMN and SBVR. Our paper on this topic was presented at the VORTE’07 workshop and can be downloaded here.

In practice, you can do some process management with a rules engine. Natalie Glance and colleagues have written some intriguing papers on Generalized Process Structure Grammars in the mid-1990s that essentially allow the modeling of processes using a constraint language (saying things such as “the start of activity B must occur after the start of activity A”, which are difficult to express using languages such as BPMN).

In the same vein, you can handle quite a bit of decision making using graphical process modeling techniques, by building gateways into your processes. This way, your process diagram becomes (partially) a decision tree, with alternative pathways for different cases.

Coming from the process side of things, I found the formal logic and languages used in rules management standards such as RuleML rather intimidating. So when I was offered the opportunity to speak on the integration of rules and processes at the IIR BPM conference a few weeks ago, I tried to approach this topic from a pragmatic perspective:

Say you are a process modeler: How do you approach the topic of rules?

Most process models I’ve dealt with contain at least some aspects of decision making, typically found in splits (decision gateways). In particular two general types of decisions can be distinguished:

• Decisions that affect the activities to be performed (Control Flow Decision). These types of decisions determine which process steps are appropriate for a given case (workflow instance). For instance, if you are dealing with a new customer and a large order you may want to perform a credit check, whereas you would skip this step if the customer is known to you. The decision in this case has an impact on the routing of the workflow instance.

We can distinguish some sub-cases in this scenario:

• Single-criteria decisions where we only need to review one parameter
• Multi-criteria decisions where we use decision tables or similar mechanisms to determine the case type

Similarly we can distinguish between:

• Manual decisions, made by a human (e.g. when judgement is required)
• Automated decisions that can be formalized

• Decisions that affect the assignment of activities to performers (Assignment Decisions). These types of decisions determine who gets to perform a particular activity. For instance, a customer service representative may review an order up to $5,000, but above that value we want a manager to review the order. The review activity in both cases is identical, the difference lies in who gets to perform the work. The decision in this case has an impact on the assignment of the workflow instance.

So what is my point here? Whether you model these types of rules in your process modeling environment or not depends on your context, the availability of a separate rules management environment, and most critically, the frequency with which these rules change. There is no universally right or wrong way to manage the intersection of rules and processes. If your decision rules are as stable as your process, great, leave them in your BPM development environment. But if your business users want to manipulate the parameters, separate them from the process and handle them in a separate rules management environment. You’ll be glad you did.

My presentation from the IIR conference is available on slideshare.net (see embedded presentation below). And for some well-informed outside opinion you can refer to Sandy Kemsley’s timely blog post on the presentation.

Business Process Management has the potential to transform organizations into more nimble, agile entities, that leverage both human and technology capital effectively. But very often BPM efforts are marred by an emphasis on technology, drawing diagrams, or creating the 57th architectural framework within the enterprise. I was given the opportunity to talk about some issues that Enterprise Architects should keep in mind when approaching the topic of Business Process Management.

Some of the key points are:

• BPM is not primarily about technology, modeling, or architecture, its purpose is to improve business. If you cannot demonstrate the business value of a BPM effort, go back to the drawing board.
• Processes are a perspective on organizations, but by virtue of representing a particular view of organizations they abstract from reality and do not cover all aspects of a complex socio-technical system. Don’t confuse the map with the territory.
• BPM efforts need structure and methodology. You need a structure to guide efforts at different levels of abstraction (separating the what from the how), i.e. a level framework. You need a structure to navigate among the processes of your organization, i.e. a process architecture. And you need a methodology to retain and leverage what you have learned about managing and conducting BPM projects (which is different from just saying “we use BPMN”).
• To mature BPM efforts you can focus on Governance structure, Culture, or Tools & Methodology. Don’t try to improve everything at once, but review the most important aspect for your organization before you branch out.

I have given several variations of this talk in the past, and it has evolved into a fairly comprehensive overview of the organizational, technical, and governance issues surrounding BPM. The presentation below was given at the Forum des Acteurs du Numérique in Paris, France. I’d love to hear your feedback.

Tags: enterprise architecture, frameworks, Maturity, process architecture

How to fail at Process Design

Having an unusual name comes with its own set of challenges. For example, airlines never know whether I’m filed under Muehlen, Zur, or zur Muehlen. My phonebook entry in Germany was listed under M, in the US it is Z. Banks think that I have a middle initial (I don’t). The New Jersey MVS is unable to issue a drivers license that has a space in the last name, and so on. So, when my daughter was born and I went to fill out the birth certificate request form I knew I might be in for trouble. What I didn’t know was how right I would be…

A few weeks later my wife and I received the birth certificate, and alas, my daughter’s last name was spelled Zur Muehlen instead of zur Muehlen. Not a big deal, but being German I wanted it corrected, of course. So I went to the website of the New York Department of Health and Mental Hygiene (NYHMH) to figure out the change process. There was a form (8 pages) and instructions, thankfully as a form-enabled PDF. No electronic submission, though, because additional documentation was required. Fine. We filled out the form, copied and notarized the additional documentation, found a legal sized return envelope, added $30 for additional certificates and mailed it off.

Four weeks later we received a thick letter from NYHMH – our application was returned because the ID for the mother was insufficient. No problem – we added a copy of her passport and green card, found another legal size return envelope, and mailed it back.

Four weeks later we received another thick letter from NYHMH – our application was returned because the ID with the signature for the mother did not show an expiration date. Hmmmm – her passport signature page (no expiration date) and green card (expiration date) were copied on the same page. You would expect a reasonable human being to mentally combine these to meet the requirements. But no. No problem – we added a copy of her passport and green card, found another legal size return envelope, and mailed it back.

Three weeks later (you guessed it) we received yet another thick letter from NYHMH – our application was returned because the marriage certificate we had added was “only” a notarized copy – they want the original (which was issued by the City of New York itself…). And, of course, the legal size return envelope (with postage) was missing from the returned application again (why? I don’t know. The return mail did not use the provided envelope…).

Lessons for Business Process Design

Involving the customer in your processes (in particular during early data capture stages) makes sense for many organizations. After all, you avoid re-keying of data, and can make sure that all necessary information has been provided before you initiate potentially expensive back-office operations. New York City is one of the better municipalities in terms of its e-government initiatives, much information and many forms is available online. But simply providing forms and instructions may not be enough to ensure a smooth integration of outside parties in your processes.

NYHMH’s change process fails at the first step – the application is received, and checked for completeness. If the application is incomplete, it is marked as such and sent back to the applicant. Now, that makes sense, because you want to make sure that you can actually finish processing the new case. But the crucial element is, how you check, and how you notify.

Being ignorant of the process details, I can only assume that the receiving clerk at NYHMH goes through a checklist of items that must be sent with an application. If one item is missing, or insufficient, the application is returned. But the effort required for finishing the list and marking ALL insufficiencies on the incoming case is much smaller than the cumulative effort for re-opening and re-processing the same case multiple times, as happened in our case. If you tell a client that a process cannot start, give ALL reasons why, not just the initial failure. Our application is in the fourth round of review, and I’m mentally preparing for another missing item or insufficient document notice.

The back-and-forth is aggravated by the time it takes to get the documents to NYHMH and back. Processing probably takes 5 minutes, but postal mail takes 2 days each way, add to that the wait time in the inbox of NYHMH and you get to 3 weeks cycle time for each iteration. The net processing time for this process is only a small fraction of the overall transaction time. We started this endeavor on July 1st, now in October we still don’t have a corrected birth certificate. I estimate the overall processing time that went into the application to be between 2 and 3 hours (time it took us to fill out the form, gather and notarize the documents, and for NYHMH to send them back). The overall process is 100 days old today – that’s 2400 hours of cycle time, with 0.1% actual work time – probably the worst productivity ratio I have ever encountered. To put this in perspective: It took only two weeks to receive a passport.

Bottom Line

When involving Customers in your processes:

• Give them the information and resources necessary to perform their activities.
• Design your interface activities to identify all reasons why a customer request will fail
• Enable your customers to fix possible transaction errors within a reasonable processing time to transaction time ratio
• Provide your customers with visibility into your process, so they understand how their input will be processed

Final Words

The best process design is not worth anything if the people involved in the process execution don’t care about the outcome of the process. You need to align incentives (both financial and other) with the objectives of the process

Your customers care about your processes. So should you.

Postscript (October 14th, 2007)

As you may have guessed, the third attempt was returned as well. The reason? I’ve requested a wholesale change of all last names, and I didn’t realize that my wife’s maiden name is on the birth certificate, which doesn’t need changing. So – the whole stack of documents was returned (this time including the prepaid return envelope) with a request to fill out a blank form and omit the request to change the mother’s last name (a simple cross-out on the original form would have done the same). Welcome to Absurdistan…

During the recent BPM Conference I took part in a panel on Business Process Intelligence, organized by Jan Mendling of QUT Brisbane. During the panel, Roger Tregear of Leonardo Consulting brought up the issue that BPM in its current incarnation really puts an emphasis on the wrong thing: The process as a manged entity. We want to improve our organizations, and improving our processes is one way to achieve that.

I have yet to see an organization whose processes cannot be improved by an order of magnitude. But the term Business Process Management sounds as if we wanted to spend time and effort to bureaucratize the artificial concept that processes represent. After all, we are already managing customer relations, human resources, finances, and supply chains. Managing processes sounds like one more chore that keeps us from actually doing stuff.

Roger suggested a better term: Process-based Management. Processes are the engine of value creation in organizations. Using them as a centerpiece of our management strategy allows us to visualize operations, collect performance metrics that link these operations to goals and outcomes, and have an entry point for improvement efforts. And, come to think of it, process-based management can lead to process-based improvement initiatives, and process-based leadership. It is a shift of perspective. We should no longer focus on the process as the artifact that has to be managed. Rather, the process is the foundation for the management of the organization. Now, what would be a good acronym for that?

Tonia de Bruin at QUT Brisbane is tracking the pathways organizations are taking during the Business Process Improvement efforts. Do you start with technology first? Or do you instill a process culture in your organization right away? As part of her doctoral work, Tonia is conducting a survey of organizations that are engaged in (or have already completed) Business Process Management projects. It takes 10 minutes to fill out and the results will be made available to participants. You can find the survey here: http://www.bpm.fit.qut.edu.au/students/toniadebruin/survey/

I had the opportunity to spend a week in sunny Brisbane, Australia, to attend the 5th International Conference on Business Process Management, the largest academic conference in the BPM space. Hajo A. Reijers and I presented a paper on the use of biological mechanisms for task allocation, which are particularly useful in the emergency response domain (you can download the paper in the publications section here).Today I gave a two hour tutorial on the state of BPM standards, which was very well attended. The emerging landscape of integration, choreography, notation, and interchange standards in the BPM field are both confusing for practitioners and a fertile research field for researchers. Thankfully there are some efforts from standards groups to clarify the scope of their efforts and the impact of their specifications. I had a chance to talk to Karsten Ploesser of SAP, one of the co-authors of the BPEL4People draft specification, which is emerging as a very focused document, which will probably make it easier to understand and implement in practice – one key aspect that affects the adoption and diffusion of standards.Since I received a large number of requests for the presentation slides I made them available on slideshare. You can download the PDF of the presentation here and an audio recording is available here. 

| View | Upload your own

Update: All slides from the BPM 2007 conference are available for download here: http://bpm07.fit.qut.edu.au/program/index.jsp 

I came across a call for papers for a special issue of the Cutter IT Journal on BPM – broken promises or building blocks of modern EA, edited by Bartek Kiepuszewski, which I re-posted here. What piqued my interest was how the call was phrased. There are two items that I don’t particularly agree with:

• The WfMC never attempted to standardize the architecture of a BPMS or workflow management system. The purpose of the group was to work on standards that allow for the interoperability of these systems. To make systems interoperable you need interface specifications, but you do not need to know about the internal architecture of the other system (rule #1 of loose coupling, or as Maier and Rechtin put it: The greatest leverage in systems architecting is at the interfaces). The WfMC reference model was never meant as an architecture model, it is simply a collection of those interfaces that the WfMC thinks should be standardized.
• While there are overlapping standards in the BPM area, we are far from a standards war. Compared to battles such as Blue-Ray vs. HD DVD the BPM community gets along pretty well. Most importantly, the shakeout of standards affects end users to a very limited extent, since the most visible component of the standards stack – BPMN – is universally agreed upon. Everything underneath is in flux, but that is something that vendors have to decide on, not users. The tug-of-war between BPDM, XPDL and BPEL is a bit like deciding on a file format for word processing documents. I don’t need to be able to read a .docx file in its native format, nor should it be of concern to me. It should be of concern to me whether people I collaborate with can read and write documents in a format we can all process. In 95% of the cases we either agree on a platform, or we use converters that work just fine. Caveat: If you have ever worked with people who love LaTeX, that’s a whole different issue…

John Evdemon said at the last BPM Think Tank that a lot of BPM standardization is Management by Magazine. Come to think of it, it’s a bit absurd, since the very group that makes the standards is the same group that has to implement them. In absence of any opposing views, much of what is written in magazines is being perceived as the ultimate truth, which is a bit sad. Academia has a system of peer reviews to make sure that there are checks and balances in place. But who in industry reads academic journals?

The Association for Information Systems has a system of special interest groups that focus on different research areas within the Information Systems domain. One of these groups focuses on BPM and is called sig|pam (Special Interest Group for Process Automation and Management). After a long hiatus the group has been revived at the recent AMCIS conference, where Amit Deokar, Jay Nunamaker and I chaired a minitrack on BPM.

sig|pam is maintained by the illuminati of academic BPM research, so point your browsers to www.sigpam.org and find out what is going on in the BPM research scene…

I am attending the BPM Think Tank in Burlingame this week, and there are many insightful presentations around emerging standards in the BPM space, such as BPDM, BPMM, BMM, BPMN 2.0 and OSRM. But one thing makes me wonder – with every revision, every iteration, the standard specifications grow in size. The new BPMM specification has a whopping 505 pages in draft version. A participant asked what the effect would be if the BPMN 2.0 specification, which combines BPMN and BPDM, would be a 1,000 page document. Nobody knows… I had a look at some older and newer specifications, and this is what I came up with:

There are some interesting observations to make:

• Standard specifications seem to double between versions. The only exception is UML, which actually shrank 300 pages between versions 1.3 and 2.0
• Some organizations produce shorter specifications than others. For example, IETF specifications seem to be rather concise, compared to OMG or OASIS specifications.

Now, counting pages is not a very exact metric to gauge the complexity of a specification, but it is safe to assume that a 300 page specification is significantly more complex than a 60 page specification. I brought this up at the Think Tank, and it was suggested that specs grow because the working groups add clarifications and explanations. But it is also possible that as the standard specs grow, the effort to implement them and to prove conformance with all aspects of a specification increases significantly. If that is the case, do bigger standards keep the industry from advancing?

My research group and groups at seven other universities form the Innovation & Education Network in BPM, sponsored by IDS Scheer. This group was launched in mid-June, and our local Cablevision channel stopped by and conducted a short interview, which I promptly YouTubed. What’s next? The Daily Show?