by Michael zur Muehlen (mzurmuehlen@stevens.edu) and Jan Recker (j.recker@qut.edu.au)
BPMN is the de facto standard for graphical process modeling. While there are other graphical languages to represent processes (EPCs, IDEF, Flowcharts, Petri Nets, among others), no other notation has seen such an uptake in such a short time as BPMN has. It is widely supported by both free and commercial process modeling tools, the WfMC has made XPDL 2.0 and 2.1 a de-facto persistency format for BPMN diagrams, and a large number of courses on modeling processes with BPMN are being offered.
Now, BPMN is a complex language. The current incarnation (BPMN 1.1) consists of 52 distinct graphical elements: 41 flow objects, 6 connecting objects, 2 grouping objects, and 3 artifacts. That’s a lot of vocabulary to learn, given that each graphical elements has meaning and rules associated with it. So what is the minimum subset of BPMN that a process modeler should know?The answer: Less than you think.
To answer this question we collected a large number of BPMN 1.0 diagrams (126 in total), from consultants, seminar participants, and online sources. We analyzed which BPMN symbols were actually used in these diagrams. The full version of our research, which we will present at the Conference on Advanced Information Systems Engineering in June, can be found here. But since this is an academic paper, here are the practical highlights of our study.
None of the diagrams we looked at used more than 15 different BPMN constructs, and none used less than 3. The models themselves contained considerably more elements, but a model with, e.g., 5 tasks connected by sequence flow was recorded as using the task symbol and the sequence flow symbol. The average subset of BPMN used in these models consisted of just 9 different symbols. That means that the average BPMN model uses less than 20% of the available vocabulary.
Figure 1 shows which construct we found across which percentage of the diagrams we collected.
Figure 1: Frequency distribution of BPMN construct usage
The results of our study are:
Only five elements (normal flow, task, end event, start event, and pool) were used in more than 50% of the models we analyzed. These, plus the data-based XOR gateway form what we call the common core of BPMN (marked in yellow in fig. 1).
Six additional elements were found in at least 25% of the models - gateways (parallel and unmarked XOR), lanes, text annotations, message flow, and start messages, we call these the extended core of BPMN (marked in green in fig. 1).
17 elements were used in less than 3 models - seven elements occurred in just two models, five in just one, and five elements were not used in any of the models we studied.
We then looked at the co-occurrence of BPMN symbols - i.e., are certain constructs used in combination, and how frequently? The combination of certain elements is mandated by the BPMN specification - you cannot use lanes without pools, or data objects without associations. But if there is a common subset used by many models, this would constitute a true “common core”. A detailed analysis revealed that BPMN elements fall into several well-defined groups. Figure 2 shows these groups as frames around the respective BPMN elements. The numbers within each frame represent the number of models (out of 126) that contain all elements within the frame.
Figure 2: Grouping of BPMN elements
Our findings are:
The common core of BPMN is very small. The subset of BPMN across the different models varied considerably. While nearly all models contain tasks and sequence flow, adding symbols to this set leads to a near exponential drop in models that share the (bigger) set of symbols. For example, while 65 models contain tasks, sequence flow, start and end events, only 25 also contain parallel gateways, and just 10 contain parallel gateways and data-based XOR gateways.
There are two types of BPMN modelers. While our sample is too small to explore this proposition in detail, we found anecdotal evidence that two groups of modelers use BPMN: Those who use pools and lanes to represent organizational responsibility for tasks, and those who use gateways to represent the control-flow rules of the process in detail. In other words, one group uses BPMN to specify inter-organizational settings (process choreography). Mostly, these users will be consultants or process analysts working on organizational (re-) engineering and process improvement. The other BPMN user group is leaning more towards workflow engineering (process orchestration). These users will likely be designers and analysts seeking to articulate precise flow conditions, for instance, in the context of workflow engineering or process simulation.
Implications
Our findings have implications for practitioners, software vendors, and standards makers alike.
Practitioners can begin studying the use of BPMN by focusing on the most commonly used symbols first, leaving more specialized and lesser-used constructs for those who need more specialized BPMN training (e.g. systems analysts).
Software vendors that are not supporting the entire BPMN vocabulary can assess what percentage of BPMN diagrams can be represented in their tool, and where enhancements should be made.
Finally, Standards-makers should review whether a more complete, but also more complex language is a desirable result of the standardization process. Creating BPMN took six years. How much time was spent on defining those seventeen symbols that we found are hardly used? And will the extensions of BPMN 1.1 entice users to expand their commonly used vocabulary, or will they go unused?
If you would like to learn more about this research, we encourage you to read the full version of our paper:
Michael zur Muehlen, Jan Recker. (Jun 16, 2008). “How Much Language is Enough? Theoretical and Practical Use of the Business Process Modeling Notation”, 20th International Conference on Advanced Information Systems Engineering (CAiSE 2008), Montpellier, France, June 16-20, 2008., Springer LNCS. Download (657 kb PDF)
So, you’re charged with improving business operations. Maybe even deploy a BPM system. Defining Key Performance Indicators and assess the operational risk of your processes. What are the skills you need - and who needs them? For the past year and a half, I have been working on re-tooling Stevens Institute of Technology’s educational offerings around BPM, and I’m happy to report that we have just launched our Business Process Management & Service Innovation Program online (howe.stevens.edu/BPM).
This Advanced Graduate Certificate consists of four courses that cover the strategic, tactical, and operational aspects of managing and improving business processes. Our brand new BPM & Workflow Implementation course (MIS 712 WS) will start online January 28th with an information session, and online enrollment is available this week at www.stevens.edu/registrar and gradschool.stevens.edu, the Call Number is 11624. The course focuses on the documentation, improvement, and implementation of processes using state-of-the-art BPM technology. Topics include managing Business Processes and Business Rules, documenting and managing operational process risk, simulating processes, and more. We are partnering with TIBCO, IDS Scheer, Lombardi and SunGard to provide students with hands-on experience using the latest BPM tools. Classes are delivered using WebCampus, Stevens’ award winning distance learning program, iTunes University, live instructor sessions using InterWise, and our eLearn platform WebCT.
If you are charged with analyzing, designing, or improving business operation this program will provide you with the state-of-the-art skills necessary to understand, communicate, and align your business processes and the technology that supports them. If you have any questions or comments I look forward to discussing this with you in more detail. For more musings on the educational aspects of BPM be sure to check out my column on bptrends.com.
The topic-du-jour in BPM circles is the handling of human and automated decision making in processes. Two major areas that intersect here are the management of business rules (such as “customers with more than 100,000 frequent flier miles receive priority treatment when flights are oversold“) and business processes (such as “rebook voluntary denied boarding customer“). There has been plenty of work done in both domains, but until very recently they did not talk to each other very much.
That has changed quite rapidly, as the business rules community realized that it needed some ways to represent the structured order of long-running decision-making activities (as typically found in workflows), and as the process community realized that modeling decisions and rules using activity networks, BPMN, or Petri Nets results in rather bloated and complex diagrams.
On the research side, my colleague Marta Indulska from the University of Queensland and I have studied the expressiveness of process and rule modeling languages using representational analysis (i.e. we used a formal ontology as a benchmark) and found that the combination of process modeling and rule modeling languages generally offers higher expressive power than either of these languages by itself. We found the combination of BPMN and SRML particularly useful, but since SRML is an abandoned effort we would recommend the combination of BPMN and SBVR. Our paper on this topic was presented at the VORTE’07 workshop and can be downloaded here.
In practice, you can do some process management with a rules engine. Natalie Glance and colleagues have written some intriguing papers on Generalized Process Structure Grammars in the mid-1990s that essentially allow the modeling of processes using a constraint language (saying things such as “the start of activity B must occur after the start of activity A”, which are difficult to express using languages such as BPMN).
In the same vein, you can handle quite a bit of decision making using graphical process modeling techniques, by building gateways into your processes. This way, your process diagram becomes (partially) a decision tree, with alternative pathways for different cases.
Coming from the process side of things, I found the formal logic and languages used in rules management standards such as RuleML rather intimidating. So when I was offered the opportunity to speak on the integration of rules and processes at the IIR BPM conference a few weeks ago, I tried to approach this topic from a pragmatic perspective:
Say you are a process modeler: How do you approach the topic of rules?
Most process models I’ve dealt with contain at least some aspects of decision making, typically found in splits (decision gateways). In particular two general types of decisions can be distinguished:
Decisions that affect the activities to be performed (Control Flow Decision). These types of decisions determine which process steps are appropriate for a given case (workflow instance). For instance, if you are dealing with a new customer and a large order you may want to perform a credit check, whereas you would skip this step if the customer is known to you. The decision in this case has an impact on the routing of the workflow instance.
We can distinguish some sub-cases in this scenario:
Single-criteria decisions where we only need to review one parameter
Multi-criteria decisions where we use decision tables or similar mechanisms to determine the case type
Similarly we can distinguish between:
Manual decisions, made by a human (e.g. when judgement is required)
Automated decisions that can be formalized
Decisions that affect the assignment of activities to performers (Assignment Decisions). These types of decisions determine who gets to perform a particular activity. For instance, a customer service representative may review an order up to $5,000, but above that value we want a manager to review the order. The review activity in both cases is identical, the difference lies in who gets to perform the work. The decision in this case has an impact on the assignment of the workflow instance.
So what is my point here? Whether you model these types of rules in your process modeling environment or not depends on your context, the availability of a separate rules management environment, and most critically, the frequency with which these rules change. There is no universally right or wrong way to manage the intersection of rules and processes. If your decision rules are as stable as your process, great, leave them in your BPM development environment. But if your business users want to manipulate the parameters, separate them from the process and handle them in a separate rules management environment. You’ll be glad you did.
My presentation from the IIR conference is available on slideshare.net (see embedded presentation below). And for some well-informed outside opinion you can refer to Sandy Kemsley’s timely blog post on the presentation.
Business Process Management has the potential to transform organizations into more nimble, agile entities, that leverage both human and technology capital effectively. But very often BPM efforts are marred by an emphasis on technology, drawing diagrams, or creating the 57th architectural framework within the enterprise. I was given the opportunity to talk about some issues that Enterprise Architects should keep in mind when approaching the topic of Business Process Management.
Some of the key points are:
BPM is not primarily about technology, modeling, or architecture, its purpose is to improve business. If you cannot demonstrate the business value of a BPM effort, go back to the drawing board.
Processes are a perspective on organizations, but by virtue of representing a particular view of organizations they abstract from reality and do not cover all aspects of a complex socio-technical system. Don’t confuse the map with the territory.
BPM efforts need structure and methodology. You need a structure to guide efforts at different levels of abstraction (separating the what from the how), i.e. a level framework. You need a structure to navigate among the processes of your organization, i.e. a process architecture. And you need a methodology to retain and leverage what you have learned about managing and conducting BPM projects (which is different from just saying “we use BPMN”).
To mature BPM efforts you can focus on Governance structure, Culture, or Tools & Methodology. Don’t try to improve everything at once, but review the most important aspect for your organization before you branch out.
I have given several variations of this talk in the past, and it has evolved into a fairly comprehensive overview of the organizational, technical, and governance issues surrounding BPM. The presentation below was given at the Forum des Acteurs du Numérique in Paris, France. I’d love to hear your feedback.
Having an unusual name comes with its own set of challenges. For example, airlines never know whether I’m filed under Muehlen, Zur, or zur Muehlen. My phonebook entry in Germany was listed under M, in the US it is Z. Banks think that I have a middle initial (I don’t). The New Jersey MVS is unable to issue a drivers license that has a space in the last name, and so on. So, when my daughter was born and I went to fill out the birth certificate request form I knew I might be in for trouble. What I didn’t know was how right I would be…
A few weeks later my wife and I received the birth certificate, and alas, my daughter’s last name was spelled Zur Muehlen instead of zur Muehlen. Not a big deal, but being German I wanted it corrected, of course. So I went to the website of the New York Department of Health and Mental Hygiene (NYHMH) to figure out the change process. There was a form (8 pages) and instructions, thankfully as a form-enabled PDF. No electronic submission, though, because additional documentation was required. Fine. We filled out the form, copied and notarized the additional documentation, found a legal sized return envelope, added $30 for additional certificates and mailed it off.
Four weeks later we received a thick letter from NYHMH - our application was returned because the ID for the mother was insufficient. No problem - we added a copy of her passport and green card, found another legal size return envelope, and mailed it back.
Four weeks later we received another thick letter from NYHMH - our application was returned because the ID with the signature for the mother did not show an expiration date. Hmmmm - her passport signature page (no expiration date) and green card (expiration date) were copied on the same page. You would expect a reasonable human being to mentally combine these to meet the requirements. But no. No problem - we added a copy of her passport and green card, found another legal size return envelope, and mailed it back.
Three weeks later (you guessed it) we received yet another thick letter from NYHMH - our application was returned because the marriage certificate we had added was “only” a notarized copy - they want the original (which was issued by the City of New York itself…). And, of course, the legal size return envelope (with postage) was missing from the returned application again (why? I don’t know. The return mail did not use the provided envelope…).
Lessons for Business Process Design
Involving the customer in your processes (in particular during early data capture stages) makes sense for many organizations. After all, you avoid re-keying of data, and can make sure that all necessary information has been provided before you initiate potentially expensive back-office operations. New York City is one of the better municipalities in terms of its e-government initiatives, much information and many forms is available online. But simply providing forms and instructions may not be enough to ensure a smooth integration of outside parties in your processes.
NYHMH’s change process fails at the first step - the application is received, and checked for completeness. If the application is incomplete, it is marked as such and sent back to the applicant. Now, that makes sense, because you want to make sure that you can actually finish processing the new case. But the crucial element is, how you check, and how you notify.
Being ignorant of the process details, I can only assume that the receiving clerk at NYHMH goes through a checklist of items that must be sent with an application. If one item is missing, or insufficient, the application is returned. But the effort required for finishing the list and marking ALL insufficiencies on the incoming case is much smaller than the cumulative effort for re-opening and re-processing the same case multiple times, as happened in our case. If you tell a client that a process cannot start, give ALL reasons why, not just the initial failure. Our application is in the fourth round of review, and I’m mentally preparing for another missing item or insufficient document notice.
The back-and-forth is aggravated by the time it takes to get the documents to NYHMH and back. Processing probably takes 5 minutes, but postal mail takes 2 days each way, add to that the wait time in the inbox of NYHMH and you get to 3 weeks cycle time for each iteration. The net processing time for this process is only a small fraction of the overall transaction time. We started this endeavor on July 1st, now in October we still don’t have a corrected birth certificate. I estimate the overall processing time that went into the application to be between 2 and 3 hours (time it took us to fill out the form, gather and notarize the documents, and for NYHMH to send them back). The overall process is 100 days old today - that’s 2400 hours of cycle time, with 0.1% actual work time - probably the worst productivity ratio I have ever encountered. To put this in perspective: It took only two weeks to receive a passport.
Bottom Line
When involving Customers in your processes:
Give them the information and resources necessary to perform their activities.
Design your interface activities to identify all reasons why a customer request will fail
Enable your customers to fix possible transaction errors within a reasonable processing time to transaction time ratio
Provide your customers with visibility into your process, so they understand how their input will be processed
Final Words
The best process design is not worth anything if the people involved in the process execution don’t care about the outcome of the process. You need to align incentives (both financial and other) with the objectives of the process
Your customers care about your processes. So should you.
Postscript (October 14th, 2007)
As you may have guessed, the third attempt was returned as well. The reason? I’ve requested a wholesale change of all last names, and I didn’t realize that my wife’s maiden name is on the birth certificate, which doesn’t need changing. So - the whole stack of documents was returned (this time including the prepaid return envelope) with a request to fill out a blank form and omit the request to change the mother’s last name (a simple cross-out on the original form would have done the same). Welcome to Absurdistan…
During the recent BPM Conference I took part in a panel on Business Process Intelligence, organized by Jan Mendling of QUT Brisbane. During the panel, Roger Tregear of Leonardo Consulting brought up the issue that BPM in its current incarnation really puts an emphasis on the wrong thing: The process as a manged entity. We want to improve our organizations, and improving our processes is one way to achieve that.
I have yet to see an organization whose processes cannot be improved by an order of magnitude. But the term Business Process Management sounds as if we wanted to spend time and effort to bureaucratize the artificial concept that processes represent. After all, we are already managing customer relations, human resources, finances, and supply chains. Managing processes sounds like one more chore that keeps us from actually doing stuff.
Roger suggested a better term: Process-based Management. Processes are the engine of value creation in organizations. Using them as a centerpiece of our management strategy allows us to visualize operations, collect performance metrics that link these operations to goals and outcomes, and have an entry point for improvement efforts. And, come to think of it, process-based management can lead to process-based improvement initiatives, and process-based leadership. It is a shift of perspective. We should no longer focus on the process as the artifact that has to be managed. Rather, the process is the foundation for the management of the organization. Now, what would be a good acronym for that?
Tonia de Bruin at QUT Brisbane is tracking the pathways organizations are taking during the Business Process Improvement efforts. Do you start with technology first? Or do you instill a process culture in your organization right away? As part of her doctoral work, Tonia is conducting a survey of organizations that are engaged in (or have already completed) Business Process Management projects. It takes 10 minutes to fill out and the results will be made available to participants. You can find the survey here: http://www.bpm.fit.qut.edu.au/students/toniadebruin/survey/
I had the opportunity to spend a week in sunny Brisbane, Australia, to attend the 5th International Conference on Business Process Management, the largest academic conference in the BPM space. Hajo A. Reijers and I presented a paper on the use of biological mechanisms for task allocation, which are particularly useful in the emergency response domain (you can download the paper in the publications section here).Today I gave a two hour tutorial on the state of BPM standards, which was very well attended. The emerging landscape of integration, choreography, notation, and interchange standards in the BPM field are both confusing for practitioners and a fertile research field for researchers. Thankfully there are some efforts from standards groups to clarify the scope of their efforts and the impact of their specifications. I had a chance to talk to Karsten Ploesser of SAP, one of the co-authors of the BPEL4People draft specification, which is emerging as a very focused document, which will probably make it easier to understand and implement in practice - one key aspect that affects the adoption and diffusion of standards.Since I received a large number of requests for the presentation slides I made them available on slideshare. You can download the PDF of the presentation here and an audio recording is available here.
I came across a call for papers for a special issue of the Cutter IT Journal on BPM - broken promises or building blocks of modern EA, edited by Bartek Kiepuszewski, which I re-posted here. What piqued my interest was how the call was phrased. There are two items that I don’t particularly agree with:
The WfMC never attempted to standardize the architecture of a BPMS or workflow management system. The purpose of the group was to work on standards that allow for the interoperability of these systems. To make systems interoperable you need interface specifications, but you do not need to know about the internal architecture of the other system (rule #1 of loose coupling, or as Maier and Rechtin put it: The greatest leverage in systems architecting is at the interfaces). The WfMC reference model was never meant as an architecture model, it is simply a collection of those interfaces that the WfMC thinks should be standardized.
While there are overlapping standards in the BPM area, we are far from a standards war. Compared to battles such as Blue-Ray vs. HD DVD the BPM community gets along pretty well. Most importantly, the shakeout of standards affects end users to a very limited extent, since the most visible component of the standards stack - BPMN - is universally agreed upon. Everything underneath is in flux, but that is something that vendors have to decide on, not users. The tug-of-war between BPDM, XPDL and BPEL is a bit like deciding on a file format for word processing documents. I don’t need to be able to read a .docx file in its native format, nor should it be of concern to me. It should be of concern to me whether people I collaborate with can read and write documents in a format we can all process. In 95% of the cases we either agree on a platform, or we use converters that work just fine. Caveat: If you have ever worked with people who love LaTeX, that’s a whole different issue…
John Evdemon said at the last BPM Think Tank that a lot of BPM standardization is Management by Magazine. Come to think of it, it’s a bit absurd, since the very group that makes the standards is the same group that has to implement them. In absence of any opposing views, much of what is written in magazines is being perceived as the ultimate truth, which is a bit sad. Academia has a system of peer reviews to make sure that there are checks and balances in place. But who in industry reads academic journals?
The Association for Information Systems has a system of special interest groups that focus on different research areas within the Information Systems domain. One of these groups focuses on BPM and is called sig|pam (Special Interest Group for Process Automation and Management). After a long hiatus the group has been revived at the recent AMCIS conference, where Amit Deokar, Jay Nunamaker and I chaired a minitrack on BPM.
sig|pam is maintained by the illuminati of academic BPM research, so point your browsers to www.sigpam.org and find out what is going on in the BPM research scene…
The Association for Information Systems has a system of special interest groups that focus on different research areas within the Information Systems domain. One of these groups focuses on BPM and is called sig|pam (Special Interest Group for Process Automation and Management). After a long hiatus the group has been revived at the recent AMCIS conference, where Amit Deokar, Jay Nunamaker and I chaired a minitrack on BPM.
Entries (RSS)