A retrospective look at process change with an eye to the future

Citation

Grover, V. (2003), "A retrospective look at process change with an eye to the future", Business Process Management Journal, Vol. 9 No. 2. https://doi.org/10.1108/bpmj.2003.15709baa.001

A retrospective look at process change with an eye to the future

A retrospective look at process change with an eye to the future

Find a combination of words like business, process, innovation, redesign, and re-engineering, and you would probably be referring to the phenomena that took corporations by storm in the 1990s. While much has changed since the initial wave of these initiatives, they left an indelible mark on business thinking, by bringing process concepts to the forefront of managerial interventions to cut costs, enhance service, and improve performance. Much of the hype over re-engineering was exactly that, a fad that resonated with firms feeling the tight economy, pressured to reduce costs, and wanting to see better returns on their major information technology (IT) investments. Consultants repackaged old methodologies and printed glossy brochures and charged thousands for their "proprietary" solutions to "fundamental" business problems. So in some way, the ideas of Hammer, Champy, Davenport, among others came at the right time, when there was a hunger for such solutions. However, despite the failure of a large number of initial process change projects, the concepts and ideas have survived. From their beginnings in radical IT-driven re-engineering of the late 1980s, to holistic change management efforts of the mid-1990s, to business process-based electronic commerce and ERP of the late 1990s into the 2000s, business process change has held the spotlight. So what are the sustainable ideas of this phenomenon – that are endearing enough to stand the test of time? What are the lessons learned from failure that we can imbibe into the next generations of process initiatives? Where are the major challenges – the points of inflexion that could take us down new paths for business conduct and organization?

The good

Initial proponents of process change voiced the "clean slate" approach, advocating discontinuous thinking. The mantra was – "Do not let the baggage of structures and rules that were put in place at a time and place where they made sense be an impediment to change". If you automate a mess, all you get is an automated mess. "Redesign cross-functional business processes using the power of modern IT in order to achieve dramatic improvements in business performance." The thesis was sound and went like this. The hierarchy of the 1950s is a machine, where the brain power of individuals does not matter because the structure, the jobs, and the evaluation systems designed into the machine are the basis of how the individual works. The machine is an appropriate analogy because it is a mechanistic system that keeps people under control and allows organizations to grow fast without going broke. The large cadre of uneducated people who worked in these organizations did not demand high levels of responsibility. The few that were educated were moved to the upper end of the hierarchy. Consistent with classical management thinking – people working in specialized areas (functions) could do so more efficiently. Therefore such organizations need people at the top who can get access to information and make decisions that allow these specialized functions to be integrated. The environment is stable enough to limit the number of contingencies that need to be incorporated into the machine and, if needed, more people and rules can continue to be added.

So the structure matched well with the environment. Today, however, while the need for efficiency has not disappeared, there is a corresponding need for responsiveness, quality, service, and flexibility. The environment is not stable, but hypercompetitive. The workforce is far more educated and is willing to undertake responsibility. IT now makes broad access to information possible. Unfortunately, the machines that grew and now pervade the corporate landscape – also grew clunky, with more bureaucracy that came in response to environmental contingencies, higher spans of control, and difficulty in responding rapidly through translation of strategic plans to new structures, roles, etc. – creating a mismatch between structure and environment.

Within this context, bringing "process" to the forefront makes sense. This is how work actually gets done in organizations. Customers, who are pivotal in highly competitive environments, do not care about how good a marketing or finance department is. They care about products and services, which are accomplished through processes. Therefore, rather than having fragmented tasks or functions do their "specialized" work and "hand-off" responsibility – someone should be responsible for the process. It is the process that facilitates holistic thinking about outcomes and customers. It is the process that puts more educated and responsible people in charge of teams responsible for outcomes. Such thinking emphasizes the customer over the boss, process over function, leadership over scorekeeping, outcome over task, and general integrative skills over specialization. Principles of such thinking involve moving decision making to the work point, thereby reducing information bureaucracy and flattening the hierarchy.

IT is instrumental in providing broad access to information, so that "empowered" individuals can actually act on information they can access. IT also facilitates "soft" process change, in the form of better communication between various tasks or functions involved in a process. This reduces reliance on standards (i.e. a standardized blueprint between R&D and production) and increases the importance of interaction to manage uncertainty. More important, IT can facilitate "hard" process change, where sequentially dependent tasks and functions can be changed to parallel tasks through shared repositories of information. By allowing process participants access to integrated information, all participants have a broader understanding of the process, rather than their parochial function. Such integrated repositories increase visibility of processes and reduce functional ownership of information. With the scalability of client-server and network architectures, the issue of whether to centralize or decentralize information is irrelevant – since the architectures are modular and scalable enough to provide local access to information and global access if necessary. The onus of decision-making structure rests on an organizational imperative – with the choices the organization makes.

So the logic of process change is as sound today as it was a decade ago. Organizations burdened with legacy structures need to apply process change principles to transform their organizations in order to compete in contemporary environments that demand responsiveness, innovation and customer service. The IT infrastructure, workforce changes, and progressive thinking on the part of executive management are all facilitators of these changes.

The bad

There are some who say that, despite the sound logic of process thinking, 70 percent of the initiatives through the 1990s failed. As a result of studying numerous process change projects through the 1990s, we have tried to assess the lessons that can be gathered from these initiatives.

First, the idea of "radical" change dominated the earlier years. This resulted in a number of "slash and burn" initiatives that were placed under the banner of process change, but really involved downsizing and restructuring. However, many initiatives in the spirit of the "clean slate" approach tried to optimize how work is done at the cost of people. These initiatives resulted in processes that were theoretically more streamlined, but not widely accepted. In other words imposition of radical change on cultures that were not ready for it created more problems than it solved. The more tempered view that has emerged from this is that a contingent view of process change is more appropriate. Radical change may be appropriate for some cultures and/or processes, while incremental might be better for others. Typically, if the process is sound, incremental process change can result in improvements in performance. Process change management involves the institutionalization of these changes and most important, continuous assessment of processes and their performance.

Second, many companies stressed process change, without commensurate consideration for other organizational elements. The breadth of process change was often under-emphasized. For instance, questions of incentive systems, strategy, structure, people and roles, IT, etc. have to be simultaneously considered and consciously aligned for process change success. Otherwise the result is a process that is not reinforced by other organizational elements, creating a tension that would ultimately lead to its breakdown.

Third, change management was under-emphasized. Often process change initiatives followed a consultant who brought in a proprietary methodology, worked with a hand-picked team that did not communicate effectively with the rest of the organization. Failure to get "buy-in" created strife, rumors and fears of job losses in an "us vs. them" environment.

Fourth, overemphasis on existing processes often led organizations down a path of excessive documentation of processes that did not work! However, since process documentation methods were prominent, this became the focus of the process change effort. Doing so created the "forest and trees" problem, slowing down the process and resulting in significant loss of momentum. Ultimately, project scope creep due to too many vested interests deliberating on existing processes (and existing turf) results in the inevitable project expansion and ultimate failure.

Fifth, IT was the driver of some process change endeavors. This often resulted in IT-centric solutions, without consideration for organizational adaptations (i.e. process changes). From their own functional perspective, it was a challenge to do much more than an IT-centric implementation (e.g. ERP) when what was really needed was process change and buy-in, in order to maximize the potential of the technology. In fact in one study of more than 100 process change projects, IT issues were considered most important, but had the lowest correlation with success. On the other hand, people and change management issues were considered the least important and had the highest correlation with success.

In sum, while the logic is sound, the reality of process change endeavors has facilitated new understanding. We now recognize that the nature of process change must be contingent on the context, careful alignment of process change with other organizational elements must be done, change and people management is critical, overemphasis on existing processes could be counter-productive, and top management (not IT) should lead any major change initiative.

The challenges

Creating a process structure. While there have been numerous process initiatives, most of these have been conducted as an "overlay" to existing structures. The true challenge is for organizations to implement process centric structures in an effective manner. Hierarchical structures are very good for stable environments and the functional demarcations enable these structures to handle high levels of complexity. In other words, the hierarchy is an efficient control structure and, while not very flexible, it provides the mechanism for designing work and roles based on goals and strategy. In contrast, the entrepreneurial structure that is typical for the start-up company is simpler, but can handle rapid environmental changes. This can be done because participants usually work in teams and job roles are flexible, enabling an ability to change direction rapidly. The dilemma is that today's environment is both complex and dynamic and requires both efficiency and rapid responsiveness. While we are often quick to pronounce the death of the hierarchy, the true challenge for organizations is to overlay entrepreneurial mechanisms that enable flexibility on traditional structures (or for growing entrepreneurial firms to install structures that can efficiently manage the growing complexity of operations).

Process and the other complementary initiatives (empowerment, teams, etc.) provide the ability to bridge the two types of structures. However, companies in the next decade will struggle with trying to reconcile their functional structures with new process initiatives. Some progressive firms like Duke Energy, IBM, and Texas Instruments have created true process structures in their divisions. By identifying key processes, providing real ownership to each process to senior managers with real budgets, and changing the evaluation systems to focus more on outcomes (e.g. customer satisfaction) – these firms are creating that new structure that is both efficient and flexible. To come to that point is not easy, as political battles have to be fought between functional managers and process managers who have control of the same employees. Demarcation of work and decisions between these two factions needs planning and reconciliation. In some cases, the process structure may be the primary one and the functions act as consultants. In others, the hierarchy might be the main structure and process overlays might be the best way to go. Regardless of the route chosen, the true "information age" organization will need to be efficient, flexible, and align information resources and incentives, in order to achieve performance in hypercompetitive environments.

Integrating knowledge processes. A second challenge for organizations in the next decade is to effectively generate, codify and exploit knowledge, which is argued to be the only basis for sustainable competitive advantage. Today, any discussion of knowledge quickly leads to the issue of how knowledge is defined. A pragmatic definition defines the topic as the most valuable form of content in a continuum starting at data, encompassing information, and ending at knowledge. Typically, data are classified, summarized, transferred or corrected in order to add value, and become information within a certain context. This conversion is relatively mechanical, and has long been facilitated by storage, processing, and communication technologies. These technologies add place, time, and form utility to the data. In doing so, the information serves to "inform" or reduce uncertainty within the problem domain. Therefore, information is united with the context, i.e. it only has utility within the context. Knowledge has the highest value, the most human contribution, the greatest relevance to decisions and actions, and the greatest dependence on a specific situation or context. It is also the most difficult of content types to manage, because it originates and is applied in the minds of human beings. People who are knowledgeable not only have information, but have the ability to integrate and frame the information within the context of their experience, expertise, and judgment. In doing so they can create new information that expands the state of possibilities, and in turn allows for further interaction with experience, expertise, and judgment. Therefore, in an organizational context, all new knowledge stems from people. Some knowledge is incorporated in organizational artifacts like processes, structures, and technology. However, institutionalized knowledge often inhibits competition in a dynamic context, unless adaptability of people and processes (higher order learning) is built into the institutional mechanisms themselves.

Much of the work undertaken on knowledge management by firms has been accomplished without substantial change in how the organization does business. What companies actually manage under the banner of knowledge management is a mix of knowledge, information, and unrefined data – in short, whatever anyone finds that is useful and easy to store in an electronic repository. In the case of data and information, however, there are often attempts to add more value and create knowledge. This transformation might involve the addition of insight, experience, context, interpretation, or the myriad other activities in which human brains specialize.

But firms that have reached the initial plateau of knowledge management now realize that long-term, complete success at using knowledge for business advantage requires change in many core aspects of the business. In the first phase, the emphasis was on the knowledge management project. Projects are a good way to get started with knowledge management, but they are by definition peripheral to the rest of the business. Projects "bottle up" knowledge and treat it as something separate. What firms must do in the second phase of knowledge management is to integrate it with familiar aspects of the business: strategy, process, culture, and behavior. Therefore, topics on which practitioners need direct help are in the integration of knowledge management with business strategy (how can the more effective use of knowledge support or enhance business strategy?), work processes (how can a deliberate knowledge management process be "baked" into key knowledge work processes?), culture (how does one create a culture that values the creation, sharing, and use of knowledge?), and behavior (how do individuals and firms reconcile the need to balance learning and doing and realize knowledge-based benefits?).

Inter-organizational processes. The third major challenge is the application of process concepts to the inter-organizational context. While we use downstream terms like CRM to discuss customer-oriented processes, and upstream terms like SCM to discuss supplier-oriented processes, firms are just scratching the surface of inter-organizational process integration. These terms, while useful for companies in getting their internal systems in order, may be inadequate to represent the network-oriented business collaborations involving strategic relationships that will be prominent over the next few decades. Much of the focus thus far has had a technology imperative, with IT-based enterprise systems "adding" modules to exchange operational data. The true challenge is in integrating processes that involve different companies. This includes not only customers and suppliers, but perhaps competitors and co-suppliers (suppliers to the same customer) where mutually beneficial arrangements (in say distribution processes) can yield better performance for both firms. The Internet is playing a major role in this, but the next generation of efficiency and effectiveness is going to require companies to tear down their boundaries and open up all but the most proprietary of processes. This results not only in the challenge of streamlining cross-organizational processes, but also is the challenge of outsourcing – determining which processes should be "open" and which ones should remain "closed". Ultimately, it is not the increasingly standardized, open and ubiquitous IT that is going to provide competitive advantage, but the ability to create uniqueness with the interaction of this technology with processes and other complementary assets.

In sum we can see that process change has emerged through the hyperbole of the 1990s as a sustainable foundation for organizations in the next decade. Probably the most important lesson learned to date is the recognition that any process redesign must be viewed as another form of organization change which takes place in the context of people and organization. Any attempt at process redesign and implementation without appropriate plans for organizational change would greatly increase the risk of failure. Once researchers and practitioners recognized this perspective, and moved way from the quick fix mentality, well established principles concerning orchestrating effective change from such fields as strategy, organizational behavior, social psychology, industrial engineering and quality management can be applied. We are also learning that, while information technologies may not be the driver of such changes, they are instrumental in creating flexible processes and structures. Therefore investment in information infrastructure is a prerequisite for corporate agility. Such investments must be looked at as having "latency effects" in the form of digital options. At the appropriate time, companies that have built the capabilities to exploit the process and knowledge options that they have created will be the ones that can take advantage of opportunities when they present themselves. However, the challenges of imbibing these changes into process-oriented structures, knowledge processes and the inter-organizational context will be profound and real as organizations seek to compete in an evolving age of global competition and the Internet.

Varun GroverEditorial Advisory Group Member andWilliam S. Lee Distinguished Professor of Information Systems,Clemson University, USA