Evaluation and Improvement of a Transition Business Process: A Case Study Guided by a Semantic Quality-Based Approach

ABSTRACT A transition in an outsourced IT project is devoted to the transfer of the project from an outgoing project team to an incoming one. It is a complex, risky, and challenging building block of importance, identified as being a critical factor in the success of an outsourced project. Ensuring the good quality of a transition is then fundamental. We present here our experience on quality evaluation and improvement of a real transition in a public institution.


Introduction
In the current context of increasing competition, organizations are forced to look for new solutions to generate value.Outsourcing is one of the principles adopted by companies concentrating on core business.Outsourcing is a management strategy by which an organization delegates non-core activities to an external and specialized third party.It has been widely adopted by both public and private companies.During the past decade, scientific and technological institutions from the French public sector have followed this general trend, focusing on the research activities at the core of their business.They mainly outsourced support services such as Human Resources management, Finances, or Information Systems (IS).In this context, the IS Department actors had to change their activities, which initially consisted in the design, development and maintenance of applications, into contractors' activities monitoring and communication with service providers.In French public organizations, such as a French Public Scientific and Technological Institution (PSTI), the government established a procedure for long-term outsourcing.It imposes the selection of a service provider through a process coping with the rules on public contracts.This procedure is renewed periodically, and may lead to the change of the service provider.This change is usually complex to manage, as it should transfer the project activities from an outgoing project team to an incoming one.This transfer is managed in a project phase called transition, which is a complex, risky and challenging building block of strategic importance in any outsourced project (Olzmann & Wynn, 2012).Ensuring a good quality of transition is fundamental.The literature proposes many recommendations and best practices, which help to fulfill a "good" or "successful" transition (Olzmann et al., 2012).The question of what is a good transition is still an open problem.Indeed, there is no consensus on the definition of the quality of a transition.We consider the quality problem from both the evaluation and the improvement dimensions.Evaluating quality usually means measuring a set of indicators.From a practical point of view, the question is "What should be measured to assess the transition quality?"Considering the improvement of quality, this may be achieved by implementing some recommendations from the rich literature related to the transition process improvement.However, the variety and the multiplicity of such recommendations make their practical adoption difficult.In particular, one may wonder "Which recommendation(s) should be implemented in order to improve a low-quality transition?" In this article, we present our experience on quality evaluation and improvement of a real transition in a PSTI.Guided by the above open problems, we propose to consider the transition as a business process that is modeled, analyzed and improved, using existing methodological tools for quality management.The chosen approach allows defining the quality of a transition process, as well as assessing it, using a paradigm called Goal Question Metric (GQM) (Basili, Caldiera, & Rombach, 1994).Modeling the transition process helps in identifying low-quality tasks and activities.Thus, in addition to providing a clear and precise definition of the quality of a transition, it is also useful for the selection and the prioritization of targeted improvement actions.
To the best of our knowledge such an approach has never been carried out in the setting of the quality management of transition processes.Our study on a real case is a proof of adequacy and we discuss its generalizability.
The paper is organized as follows.In the first section, we introduce the transition process of an outsourced IT project and its related quality issues.We then describe the real transition process case study.The second section is dedicated to the review of approaches from the literature on business process quality management, since we propose to see the transition process as a business process that is modeled, analyzed and improved.We then detail the quality evaluation of our use case in the third section.We conclude the paper by drawing conclusions from this evaluation and by giving some research perspectives.

1
The transition in an outsourced project Willcocks and Kern (1998) define IS outsourcing as delegating to a third party the management of IT/IS assets, resources, and/or activities for required results.Different categorizations of outsourcing were proposed (see Dibbern, Goles, Hirschheim, & Jayatilaka (2004) for a survey).In our use case, the IS Department outsources the development of a new software.
The IS Department still manages the project and keeps being the main interlocutor of the business entity.The outsourcing contract rules for a French public organization, like a PSTI, impose that each outsourced project must undergo a new tendering procedure every three years.Such a procedure may lead to change the service provider, during the project in progress.This change then implies to perform a transition, which consists in transferring the outsourced project (documentations, applications, programming codes and knowledge) from the outgoing project team to the incoming one.The outgoing team is composed of the IS Department and the outgoing service provider, while the incoming team is composed of the IS Department and the incoming service provider.In our case of an outsourcing in project management mode (Lacity & Hirschheim, 1993), the service provider holds most of the knowledge about the project.In a transition phase, the goal of the IS Department is to manage the project and the knowledge transfer from the outgoing service provider to the incoming one.The transfer concerns the material and the documentation, and also the crucial explicit and tacit knowledge.However, the transfer of knowledge is a difficult and complex task (Argote & Ingram, 2000).It follows that ensuring a good quality of the transition is both fundamental and complicated.

Managing the quality of a transition
Most contributions from the literature, related to the transition of an IT outsourced project, focus on the decision to switch vendors (Alaranta & Jarvenpaa, 2010).However, only a few works concern the management of the transition and its quality.Olzmann and Wynn (2012) even note that not much is known about methods, processes and strategies for switching providers.They propose an interesting survey of recommendations and best practices to help achieving a "successful" transition.Their analysis focuses on identifying the critical success factors, for which several and diverse improvement actions are presented.These proposals concern the phases of the outsourcing process, the involved actors, and address several issues (planning, strategic, operational and financial).Some improvement actions are general, i.e., no practical guidelines are given for their implementation, as for example "Ensure that senior managers from all parties are actively involved in the process".
The existing state of the art, on the quality management of the transition, leads us to argue that open problems remain: • (Problem 1) There is no consensus on the definition of the quality of a transition.From a practical point of view the question is "Which indicators should be measured in order to assess the quality of a transition?" • (Problem 2) Implementing all the recommendations from literature is not realistic; there are no guidelines for selecting these that fit specific use and needs.From a practical viewpoint the question is "Which recommendation(s) should be implemented in order to improve a low-quality transition?"

Use case: the transition of a PSTI
In the considered Public Scientific and Technological Institution (PSTI), the transition process consists of six activities:

Activity 1
The Initialization, which corresponds to the official beginning of the transition.

Activity 2
The Third Party Maintenance (TPM) ending during which, among other more contractual tasks, an inventory of internal and external documents, applications and programming codes is performed (called Inventory task, for short, in the following sections of the paper).Activity 3 The Transfer planning during which the content of the following activities is defined.Activity 4 The Project transfer that essentially consists in transmitting documentations, applications and programming codes to the incoming project team.The outgoing service provider writes documents and codes.The IS Department is almost not involved in this activity.Activity 5 The Maintenance in cooperation during which the outgoing and incoming service providers assume together a time-limited maintenance of the application.This activity is optional according to the procedure.In practice, for cost or time saving reasons, this activity is often skipped or cut back to the bare minimum.Activity 6 The Transmission of responsibility, which corresponds to the official ending of the outgoing service provider's job.
Each of these activities includes an organized set of tasks.For legal reasons, a PSTI has to perform a transition in twenty (or less) working days (constraint defined according to the legislation on outsourcing contract in French public organizations, article 31 of law n°75-1334).The project manager, a member of the IS Department, is responsible for managing this transition.The transition process has been modeled before the quality evaluation study presented in Section 3. Figure 1 proposes an excerpt from this model, focusing on activities 2 and 3 that are considered central in the remainder of our study.

Fig. 1. Activities 2 and 3 of the transition process
After performing several transitions, the project manager observed that: (Observation 1) The formal procedure in the PSTI indicates that, within the IS Department, only the project manager is involved in the transition.However the transition seems to informally involve more human resources.For instance, the project manager is responsible of establishing (alone) the inventory of internal and external documents, applications and programming codes of the project.This is the Inventory task, which is a task of the TPM ending activity (Activity 2).No additional contributor is listed in the formal procedure for this task.However, at his own initiative, the project manager usually delegates, through an informal process, the validation or the completion of the inventory to: − the expert of the applicative architecture, − the expert of the software architecture, who himself informally delegates parts of the work to a database administrator, a member of the front office or a programmer, − the expert of the hardware architecture, who usually informally requests some help from two system and network engineers, − a business contact, who himself informally requests some help from three other business experts (human resources, application and scientific).
To summarize, there are approximately thirteen persons involved in this task.The fact that the implication and the role of these actors are not formally taken into account hampers the suc-cess of the process.A clear identification of the involved actors and the precise definition of their roles are important to ensure the quality of deliverables (in our case, the inventory in the form of a document recording the elements to be transferred).Moreover, the absence of these persons in the formal procedure distorts the analysis of the transition process quality.Thus, while the Inventory task seems rather simple according to the formal procedure, it appears to be much more complex and riskier in practice, where the risk here depends on the availability of the knowledge needed for the task.
(Observation 2) The incoming service provider usually lacks of autonomy after the transition, although all the documents, applications and programming codes have been correctly transmitted.This observation indicates that a part of the necessary knowledge is still missing to the incoming service provider after the transition.
The knowledge and the knowledge transfer concepts require a discussion here, as they refer to non-trivial notions.The knowledge is defined as "justified true belief" (Nonaka & Takeuchi, 1995) and the capacity to analyze a situation and to act appropriately (Stehr, 1992).Usually, one distinguishes two kinds of knowledge: the tacit knowledge and the explicit one (Polanyi, 1967;Nonaka, 1994).Explicit knowledge can be codified (e.g.written and/or drawn) and articulated, since it can be expressed formally and systematically.Tacit knowledge corresponds to knowledge that cannot be explicitly expressed like, e.g., skills, senses, intuition, physical experiences, job secrets, environmental knowledge.
Let us now consider the notion of knowledge transfer.Knowledge transfer is the process by which one unit of an organization, such as a group or department, is affected by the experience of another (Argote et al., 2000).Explicit knowledge can be formalized, for instance, in a document, which can be transmitted.However, this is not sufficient for transferring it.We emphasize here the distinction made between transmitting and transferring.Indeed, knowledge that is transmitted by a sender but not absorbed by its receiver (Cohen & Levinthal, 1990) is not transferred.Davenport and Prusak (2000), in their equation Trans-fer=Transmission+Absorption (and Use), go beyond by distinguishing "knowing" and "being able to do", underlying that some knowledge is really absorbed only when it can be put into practice.Several studies showed that both the explicit and the tacit knowledge transfers play an important role in the transition success (Alaranta et al., 2010;Beulen, Tiwari, & van Heck, 2011;Olzmann et al., 2012).In addition, the tacit knowledge is needed for understanding explicit knowledge (Polanyi, 1975).The tacit knowledge transfer requires human interactions, holding for instance in a Ba, a shared physical, virtual or mental space for emerging relationships (Nonaka & Konno, 1998), such as a teleconference or a meeting.One can see that the transition presented above focuses on the transmission of the materials and the documentation, while ignoring human interactions.According to the previous discussion, this could explain Observation 2.
Notice that Observations 1 and 2 only refer to the specific use case, so no generalization can be provided at this point.
The project manager highlights some quality issues through these observations, but does not clearly define the quality of a transition.In particular, these observations do not answer the foundamantal questions: "How to define and to measure the quality of the transition?" and "How to improve the transition if needed?"The literature does not answer the first question, as there is no definition of the quality of a transition.Regarding the second question, some proposals from the literature provide a lot of possible actions to improve a transition, though some guidelines are still missing for choosing the most appropriate ones, given a specific use case (see discussion of Section 1.1).In section 3, we will present a quality evaluation (and improvement) of the previously defined transition.In our evaluation approach, we propose to consider the transition as a business process.The next section is thus devoted to a presentation of the approaches for a business processes quality evaluation.

Business process quality evaluation
Thomas H. Davenport and James E. Short (Davenport & Short, 1990) defined a business process as a set of logically related tasks performed in order to achieve a business outcome.Usual business processes are software development processes or customer order management processes.It is now well understood that the good management of a company goes through the alignment of the business processes onto the company business goals.So, Business Process Management (BPM), which is a set of techniques for the continuous improvement of all the processes involved in a business, has become a part of the corporate culture.Focusing on the quality management, standards like ISO 9001:2008 recommend to continuously control and improve the quality of processes (International Organization for Standardization (ISO), 2008;Persse, 2006).

Approaches for business processes quality evaluation
The management community initially defined high-level methods for the quality management of production processes (Shewhart, 1980;Deming, 2000).These methods have been naturally applied for the quality management of other processes, in particular business processes.One of the prevalent approaches is the Deming cycle (Deming, 2000) also known as Shewhart cycle (Shewhart, 1980) or Plan-Do-Check-Act (PDCA), which proposes to continuously control and improve the quality by iteratively executing the four steps: Plan, Do, Check and Act.The first step (Plan) consists in defining the quality requirements.The second step (Do) amounts to executing the process.The third step (Check) consists in the evaluation of the quality after this execution, i.e., checking if the quality is satisfactory (w.r.t. the quality requirements defined in the Plan step).In the last step (Act), new improvement actions are planed (if needed) and implemented in the next iteration of the cycle.The Six Sigma program proposed an adaptation of PDCA called DMAIC (De Feo & Barnard, 2005).DMAIC stands for the five steps of the method: Define, Measure, Analyse, Improve and Control.In the first step Define, the quality requirements are defined.In the second step Measure, the quality is measured.In the third step Analyze, quality results are analyzed; this possibly leads to implement improvement actions in the fourth step Improve.Finally, the effects of the improvement actions are measured in the last step Control.On the basis of the DMAIC method, we describe, in more detail, each step for the evaluation of a single business process.Quality definition (Define) step is delicate and crucial.During this step, a group of business users, with the help of other experts if needed, define the quality requirements also called quality goals associated to the business process.Each quality goal is expressed in terms of a set of questions called quality questions.Each such question is then expressed in terms of a set of quality metrics with possible associated thresholds (expected values).Measuring the quality metrics (in the Measure step that we detail after) enables to answer -or partly answerthe quality questions, and consequently enables to decide whether the business process satisfies the quality requirements.This approach for quality conceptualization is called the Goal Question Metric (GQM) paradigm (Basili, Caldiera, & Rombach, 1994).The quality metrics concern the business process, its execution, its impacts on the environment, or its model.They can be organized according to the GQM hierarchy or according to quality dimensions as it is usually done for data quality (Batini & Scannapieco, 2006).It is well known, within the quality communities (e.g.data quality, software quality, conceptual schemas quality, processes quality), that the quality definition depends on a specific underlying operational context.A quality evaluation then focuses on several quality metrics highlighted by business users having specific goals, specific background, and also having a specific point of view of the business process.
The second step (Measure) consists in measuring the quality, i.e., evaluating the quality metrics.This means collecting the information needed for the evaluation, and then calculating the quality metrics.Observe that collecting information can be an expensive task, like in our use case described next, for which we modeled a social network underlying the business process by interviewing every of its actors (see Section 3.2).
In the Analyze step, a group of business users analyze the results of the evaluation.They explain the difference between the measured results and expected values, and draw some improvement actions when necessary.Improvement actions can alter procedures, organization, data, processes, and, more generally, the information system.Decision-makers choose to apply some of these actions to reach the desired tradeoff between the improvement of the quality and the associated costs (English, 1999;Redman, 1997).
If the improvement actions are planed, then they are implemented in the Improve step.Their efficiency and effects are assessed in the last step Control, by evaluating the quality metrics a second time.The business process quality can be evaluated periodically (even without improvement) for monitoring.

Metrics for quality evaluation of business processes
Recently, the literature about modeling and improving business processes has been of increasing interest.Based on their experience, companies are aware of the undeniable impact of a better tuning of the business processes w.r.t. the effectiveness, consistency and transparency of their business operations.This tuning requires a better understanding and an effective management of business processes.Many research efforts have been devoted to the development of suitable methods and notations for business process modeling.However, to achieve the expected benefits on quality, it is necessary to revisit the approach of designing these processes by integrating the quality objectives within their design.We have classified the proposals from the literature into three categories: 1) the approaches focused on improving the business process methods of analysis and design, 2) the measurement of the process quality, and 3) the measurement of the process model quality.
In the first category, the approaches provide some recommendations and best practices to improve the quality of the models.The hypothesis is that improving the process development impacts positively the quality of the products.As an illustration, we can mention (Becker, Rosemann, & von Uthmann, 2000) where the authors propose a set of guidelines to improve various characteristics of a process model, such as clarity, comprehensibility, or accuracy (correctness).Other authors focus on improving the comprehensibility of the models by providing some naming conventions, documentation, and the use of icons or symbols (Mendling, Recker, & Reijers, 2010).Other approaches, such as the one introduced by van der Aalst, ter Hofstede, Kiepuszewski, & Barros (2003), propose a set of best practices encapsulated in reusable and applicable patterns depending on the context.
The second category focuses on the quality of the business processes and their execution.We categorize the simulation and control of processes as in (Jansen-Vullers & Netjes, 2006), where the authors present a set of simulation tools for business process evaluation.Other authors focus on the verification of some characteristics when executing the process.For instance, van der Aalst (2007) presents and discusses several techniques for the analysis of pro-cesses during their execution, such as verification, or for the discovery of a process (called process mining).
We focus on the third category that addresses the quality from the viewpoint of its evaluation and improvement.The quality of the process model has been investigated in different research fields.Consequently, a variety of standards have been introduced to define, manage, monitor, and improve this quality.Vanderfeesten, Cardoso, Reijers, and van der Aalst (2007) present a typology and an overall view of the business process model metrics.They mention five important measures: coupling, cohesion, complexity, modularity, and finally the size.One of the aspects that has been the subject of several proposals is the complexity (Gruhn & Laue, 2006;Cardoso, 2007;Ghani, Muketha, & Wen, 2008).However, these studies are based primarily on structural characteristics of the processes and their models.
The model complexity is directly related to comprehensibility and maintainability.Aguilar, Ruiz, García, and Piattini ( 2006) have adapted the complexity metric of software engineering process models to represent the complexity of general business process models.They conducted an experiment to measure the impact on maintainability of the process model complexity.Mendling, Reijers, and Cardoso (2007) conducted an interesting experiment aiming at finding out the parameters that impact the understandability of the process models.
In conclusion, our analysis of the state of the art leads us to argue that the quality of the business process models is mainly addressed in terms of their structure and syntax, but rarely in terms of their semantics.In the remainder of this paper, we present our approach, which aims to go a step further, to a semantic based quality approach of the business process models.

3
Quality evaluation and improvement of the transition: a case study Following the observations of the project manager (see Section 1.2), we decided to evaluate (and improve if needed) the quality of the transition.In the light of the open Problems 1 and 2, presented in Section 1.1, we propose to see the transition as a business process that is modeled, analyzed and improved, using classical methodological tools for the management of quality.The chosen approach is characterized by: • The quality definition and assessment of the modelled transition process, which relies on the GQM paradigm and • The identification of low-quality tasks and activities in the transition process, where tasks and activities are made explicit from the modeling of the transition process.The underlying idea is that a clear definition of the quality and the identification low-quality tasks provide a clear and precise definition of quality of the transition and some guidelines for the choice of priority and targeted improvement actions.This methodology partly lead to answers Problems 1 and 2.
In the following, we present the quality evaluation process, step by step, and we provide the methods, difficulties, and results we have used, as well as the lessons we have learned.

The definition of quality (Define step)
For the definition of quality, which is the first step of the DMAIC approach, we followed the GQM paradigm (see Section 2.1 for details).
In the operational context of our use case, the working group in charge of defining the quality (made of business actors, a quality expert and a knowledge manager) outlined four unranked quality goals.The first two goals are (QG 1 ) identifying complex activities and (QG 2 ) identifying tasks and activities that are sensitive to the risk of losing needed knowledge.Remark that the knowledge considered in QG 2 is the knowledge needed for each activity/task in order to ensure its good quality execution (see discussion related to Observation 1).This knowledge is not the project knowledge, which has to be transferred during the transition.The third quality goal (QG 3 ) is ensuring a good knowledge transfer of the explicit and tacit knowledge during the transition (the knowledge here is the project knowledge).Finally, the time constraint of executing the transition phase in twenty or less working days is expressed in a fourth quality goal (QG 4 ).
Each of these goals is refined into a set of quality questions (see Table 1).Each quality question is in its turn refined into a set of quality metrics (see Table 2), whose measurement provides an answer to the quality question.Quality or business experts possibly set thresholds for quality metrics, corresponding to expected values.

Quality goal
Quality question QG 1 (QQ

Kexplic-it_underst(p)
Level of understandability of the transition documents (containing explicit knowledge).The average for all documents and all readers, with the following scale: 1-good understanding, 2-neutral et 3-poor understanding.This subjective measure has been defined by interviewing the incoming team.

QQ3.2 Ktacit_underst(p)
Level of understandability of the tacit knowledge (e.g., organizational routines).The average for all members of the incoming team, with the following scale: 1-good understanding, 2-neutral et 3-poor understanding.This subjective measure has been defined by interviewing the incoming team.

QQ3.3 autonomy(p)
Degree of autonomy of the incoming service provider after the transition phase, with the following scale: excellent autonomy, good autonomy, neutral or poor autonomy.This subjective measure has been defined by interviewing the project manager.

GQ4
QQ4.1 global_runtime(p) Duration of the transition.≤ 20 (+) Each quality metric has a name and an associate type of parameter: t for a task, a for an activity, or p for the whole business process.One has to read that each quality metric with an activity (resp.a task) in parameter has to be evaluated for every activity (resp.task) of the business process, restricted to the activities under the responsibility of the IS Dept.
(*) The social network here is the social network of informal relations created between persons in order to achieve the activities and tasks of the business process (Grim-Yefsah, Rosenthal-Sabroux, & Thion-Goasdoué, 2011b).(**) A = no a priori fixed threshold.

Table 2. Quality metrics
The quality metrics (Table 2) mainly focus on the quality of the knowledge transfer and on the complexity of the business process model.This is not surprising as knowledge transfer and transition governance (whose complexity is a facet) are identified as key factors for transition success, the knowledge transfer being the most important one according to Beulen et al. (2011).We now analyze the assumptions underlying the identified metrics.Some metrics focus on the complexity and the robustness (w.r.t. the risk notion discussed in Observation 1) dimensions of the transition process.The metrics size and complex measure, respectively, the size of each transition process model activity in terms of number of tasks (independently of the structure) and in terms of the activity structure.These metrics are commonly used for measuring the complexity of business process models.The metrics runtime and glob-al_runtime respectively measure, for an execution of the process, the duration of each activity and the duration of the entire transition.The metric tasks_per_day counts the average number of tasks per day; in our case, it allows estimating the complexity of the transition management, from the project manager point of view.Grim-Yefsah, Rosenthal-Sabroux, and Thion-Goasdoué (2011b) proposed metrics for measuring the complexity of the social network of informal help (resource taken into account for relationships) between persons (individuals), underlying the execution of each task.The hypothesis is that the more complex (in size for SN_size and in structure for SN_max_depth) is the underlying social network, the more sensitive is the network to the absence of persons and so the more risky is the task (related to Observation 1).Some other metrics focus on the quality of the knowledge transfer.The metric Kexplicit_transm concerns the quality of the explicit knowledge transmission: it (partly) measures completeness and currency of the explicit knowledge.The metrics Kexplic-it_underst and Ktacit_underst measure (a part of) the quality of respectively the explicit knowledge transfer (here the knowledge codified in documents) and the tacit knowledge transfer (here the knowledge of PSTI processes, routines and practices).Finally, the metric autonomy partly measures an efficiency facet of the transition.Some quality features were not measurable.Here, it is assumed that the motivation of the outgoing service provider for transferring the project to the incoming team plays a major role in the success of the transition.This motivation is not always obvious, since the outgoing service provider is not renewed and is about to definitively leave the project (Chua, Lim, Soh, & Sia, 2012).Evaluating this motivation is very difficult.Furthermore, we limited the evaluation to tasks managed by IS Department for not to interfere with tasks that the outgoing service provider independently carries out.Therefore some tasks are intentionally not evaluated.This shows us that the evaluation of a business process is usually partial, for intentional and unintentional reasons.This must not be forgotten when analyzing the results.
We can note the wide range of quality metrics that concern: − the business process model (independently of its execution), for size and complex metrics; − for a specific execution: − the effects of the process on the environment, for autonomy; − the quality of data and information "produced" during the business process, for Kexplicit_underst and Ktacit_underst; − the environmental parameters like § duration for runtime, tasks_per_day, Kexplicit_transm and glob-al_runtime; § the underlying relationships between persons involved in the execution for SN_size and SN_max_depth.The quality metrics also concern the measurements for various aggregation levels: tasks, activities or even the whole process.Moreover, the measurement can be objective (e.g., the execution time for runtime) or subjective (e.g., the project manager perception for autonomy).
Concerning the generalizability of the approach, it is obvious that a different set of quality metrics could have been defined in another operational context.Nevertheless, the proposed metrics are reusable for the evaluation of another transition.Moreover the methodology, which consists in seeing the transition as a modeled business process and then defining quality metrics guided by the GQM paradigm, is easily adaptable to any other operational context.

The quality metrics measurement (Measure step)
The quality metrics measurement consists, after the definition of the quality, in calculating the defined metrics.Some of them could statically be evaluated on the model, while others require dynamic evaluation during process execution.Data collection.The metrics complex and size are measured on the business process model, independently of its execution.All the other metrics depend on an execution of the transition business process.The values for the metrics runtime and global_runtime trivially result from the execution.tasks_per_day is a calculated measure.The metrics Kexplicit_underst, Ktac-it_underst and autonomy are measured by interviewing the project manager and the incoming service provider.The measurement of the metrics SN_size and SN_max_depth is more complex as these metrics are based on the social network of informal relations underlying the execution of the business process.Some sociologists helped us for the methodological aspects.The (simplified) approach consists in discovering the social network by interviewing the actors (executors and contributors) of the business process, modeling it as a graph and then analyzing its structure (Wasserman & Faust, 1994).Such data collection is expensive in time and human resources.
Results.Table 3 shows some interesting results of the measurement.We analyze results in the following step (Section 3.3).

Maintenance in cooperation
Transmission of responsibility Metric size Kexplicit_transm: 0 (no delay in the delivery of transition documents) Kexplicit_underst: 2.2 (closer level in the scale: neutral) Ktacit_underst: 3 (poor tacit knowledge) autonomy of the incoming service provider: neutral global_runtime: 20 days (satisfied constraint).
Shaded values are values of interest and are further discussed.
(*) indicates a non-measurable metric (here the Project transfer activity does not contain any task under the responsibility of the IS Department).
Table 3.Some results after the measurement step

Analysis and improvement (Analyze and Improve steps)
The project manager, with the help of a knowledge manager and a quality expert, has analyzed the results (see Table 3).The goal of the analysis was to explain the low quality measures in order to identify suitable improvement actions.We outline below the conclusions of the analysis, leading to select improvement actions that we discuss and position w.r.t.those proposed in literature.
Values for SN_size and SN_max_depth are higher for the activity TPM ending than for the other activities.This identifies the TPM ending activity as being the most sensitive activity to the risk of losing some knowledge needed for executing the activity.
The results of the quality metrics evaluation, at the task level (the detailed results are not given here), exhibited two very sensitive tasks in the TPM ending activity: the Inventory and the Conception of the TPM ending planning (see Fig. 1).This means that these tasks are much more complex than the official procedure suggested, as the executors of these tasks need additional help (the official procedure does not reflect this).
These observations leaded to the following improvement action.
(Improvement 1) As a simple improvement action, the project manager will take a closer attention in the future to the quality of the deliverables produced during riskiest tasks (Inventory and Conception of the TPM ending planning of the TPM ending activity), especially when these tasks are performed during a period favorable to absence of persons (holidays, seasonal flu corporate reorganization, etc.) Another observation concerns the measure for Kexplicit_underst, which shows that some documents written by the outgoing team are difficult to understand for the incoming service provider.This denotes a poor absorption of knowledge by the incoming service provider.A reason for this is that the outgoing team and the incoming one rarely meet during the transition process, since the transition focuses on writing and transmitting documents.The transmission occurs during the Project transfer activity.The two following improvement actions, focusing on the improvement of knowledge transfer, were then proposed.
(Improvement 2) The Project transfer activity will focus on the explicit and tacit knowledge transfer.The project manager now orchestrates the Project transfer activity.He not only ensures the transmission of the documents, he also organizes working face-to-face sessions during which the outgoing and the incoming teams, including senior engineers of all parties, share explicit and tacit knowledge.This agrees with the recommendations proposed in by Beulen et al. (2011) and by Olzmann et al. (2012).
(Improvement 3) According to Nonaka et al. (1998), a Ba is a shared space favorable to individual and collective knowledge advance.The Maintenance in cooperation activity will now become an exercising Ba.
During the Maintenance in cooperation activity, the outgoing and incoming service providers assume together the maintenance of the application (see Section 1.2).This activity is optional according to the procedure.In practice, this activity is often either restricted to the bare minimum, i.e., a short observation phase of the project by the incoming service provider, or simply skipped for cost or time saving reasons.The project manager now strongly encourages the incoming service provider to use the knowledge learnt in previous activities of the transition in order to solve real incidents during the Maintenance in cooperation.This is done under the supervision of the outgoing team, which can help the incoming service provider (if needed) by transferring additional or misunderstood knowledge.This allows ensuring a good knowledge assimilation by the incoming service provider and solving the missing knowledge problems before the definitive departure of the outgoing service provider.This good practice agrees with the recommendations proposed by Grim-Yefsah, Rosenthal-Sabroux, and Thion-Goasdoué (2011a).
(Improvement 4) Another improvement option would be to enrich the business process procedure by a decomposition of sensitive tasks into subtasks where each subtask corresponds to an official request to a contributor (instead of an informal help request).
Improvement 4 would have complicated the business process procedure, making its management very complex.In terms of quality metrics, this would have lead to lower sensitivity measures (for SN_size and SN_max_depth metrics) and would have sensibly increased the complexity measures (complex and size).This situation underlines the problem of the quality metrics interdependencies: some updates made to improve a measure may lead to degrade another.This impact could be assumed or not by the business actors.In our use case the impact on the complexity has been judged unacceptable and the Improvement 4 has consequently been rejected.
We can notice that improving the transition business process implies involving more human actors, therefore generating higher costs.Indeed, the project manager spends more time on managing the transition.We also added more face-to-face meetings with all parties.This implies much more investment from the service providers, more specifically for their senior engineers and managers.This is along the lines of recommendations proposed by Sia, Lim and Periasamy (2010) and by Beulen et al. (2011) that emphasize the importance of providing sufficient resource from the client to manage the transition, and also by Olzmann et al. (2012) that recommend involving senior management from all parties in the process.
We conclude the Improve step by saying that the literature dealing with the improvement of the transition proposes a lot of possible improvement actions (Olzmann et al., 2012).In practice, it is not realistic to implement all of them.The choice of the "good" or priority improvement actions is an open problem.The approach chosen here provides a preliminary answer this problem, since some improvement actions are chosen based on results of the evaluation taking the specific operational context into account.Analyzing the business process model also allows implementing targeted actions improving low quality parts (activities, tasks) of the process, even if improvement actions are not necessarily applied locally to these parts.This is what we could call a "tailored" quality management of the transition.
Concerning the generalization of the approach, note that the chosen improvement actions could be different in another operational context.Nevertheless, the underlying methodology that is illustrated here could be adapted and even enriched to be applied in any other operational context.

Quality control (Control step)
We performed a second evaluation on the next transition in the project, in order to assess the effects of the implemented improvement actions.Table 4 shows the results of this evaluation.From the project manager point of view, the improvement actions allowed a much better communication between the outgoing and the incoming service providers.He considered that the autonomy of the incoming service provider (autonomy metric) is really improved, achieving now a good autonomy (which is obviously an indication of good knowledge transfer).
We can also see that the measures for Kexplicit_underst and Ktacit_underst were improved.We believe this is a consequence of the face-to-face meetings organized by the project manager for explaining and discussing the documentations.These meetings are (a Ba) favorable to the explicit and tacit knowledge transfer.
The time constraint (QG 4 ) is still satisfied (global_runtime metric), since the project manager decided to use the allowed time differently.More time is assigned to the Project transfer activity (runtime metric), which also becomes more complex (complex metric).This had an impact on almost all the other activities that had to be executed in less time.
The value for size(Project transfer) shows that the project manager now manages two additional tasks in the process, on eight working days (value for runtime(Project transfer)), while he was almost not involved in this activity before.This means that the project manager spends more time on the management of this activity and consequently on the transition business process (as discussed in the previous subsection).
Note that the context of the first evaluation (before improvement) and the one of the second evaluation (after improvement) are different, as service providers are different.However, we believe that these evaluations have enough similarities to be fairly compared: the same company, the same outsourced development project, with the same project manager and the same team for the IS Department.

Conclusion and perspectives
Managing an outsourced IT project is a complex and difficult task.Among all the challenges of an outsourced IT development project is the management of a service provider switch; we focused on the transition, which is its major building block (Olzmann et al., 2012).The transition is devoted to the transfer of the outsourced project (documentations, applications, programming codes and knowledge) from the outgoing project team to the incoming one.It is a delicate phase, favorable to losing knowledge (Alaranta et al., 2010).The literature proposes many recommendations and best practices helping to fulfill a "successful" transition (see Olzmann et al. (2012) for a survey).Still, open problems remain: (Problem 1) There is no consensus definition of the quality of a transition, and (Problem 2) Implementing all recommendations from the literature for transition improvement cannot be considered, moreover implementing such recommendations requires specific adaptations.So, open questions remain in practice: "How to define and measure the quality of a transition?" and if the quality is unsatisfactory, "How to choose and implement the adequate improvement actions?" In this paper, we presented our experience on the evaluation and improvement of the quality of a real transition.We developed the evaluation step by step and progressively detailed the used methods, limits, difficulties, results and lessons learned.
The underlying methodology chosen for the evaluation has the following properties: • The transition is seen as a business process that is modeled, analyzed and improved.
• We followed the classical DMAIC approach for managing the quality, implementing a complete iteration of the DMAIC cycle (Define-Measure-Analyze-Improve-Control), using the GQM methodological tool for the definition of the quality.This approach allows the definition and assessment the quality of the transition process in a specific operational context, and also the identification of the low-quality tasks and activities of the process, guiding the choice of the priority and targeted improvement actions.We believe that the successful results of our use case are a proof of concept of the chosen underlying approach adequacy for the management of the quality of a transition.
Let us now discuss the generalizability of this work.
• The methodology described above is naturally adaptable to the quality assessment of a transition in any other operational context.• Concerning the quality metrics, notice that the chosen set of quality metrics is context-dependent.Nevertheless, the chosen quality metrics constitute some potential quality metrics that can be reused for the quality assessment of a(nother) transition.• Similarly, the choice of the improvement actions is context-dependent.Nevertheless, they can also be reused for the improvement of another transition.
Concerning the quality evaluation of the transition in our use case, we identified the Inventory task (inventory of internal and external documents, applications and programming codes of the project) as being the riskiest task in the transition process (according to the definition of quality, provided by the business users in their operational context).We believe that the Inventory task is also a risky task in any operational context.A generalization of this work to several transitions (in different contexts) would certainly reveal interesting other general feedbacks.Another interest of such a generalization would be to exhibit some consensual quality metrics, which would be a first step towards a general definition of the quality of a transition.
For our study, the quality metrics expressed only the IS Department point of view.Other interesting metrics could complete this point of view e.g., a metric measuring the time spent by the members of the service providers.Let us note that we could consider a larger set of metrics, but a comprehensive quality evaluation would be too expensive.One will always have to find a compromise between the cost and the completeness of the evaluation.
As discussed in this paper, even if the Inventory is a risky task, a proposed improvement action (Improvement 4) was rejected, as its implementation would have degraded the complexity of the business process.This observation refers to the problem of identifying, anticipating and managing inter-dependencies of some quality factors, which is a difficult and still open problem for quality management communities.
We stressed the fact that this quality evaluation was expensive as it is human resource consuming.This is however, according to us, a necessary price to pay.We also note that improving the transition business process implies involving more human resources, so makes the transition more expensive.This usual question remains: "Is the improvement more expensive than the non-quality costs?".It refers to open research problems.
Another perspective is the development of software (or a plug-in in existing software) for helping in defining the quality of a transition business process (for instance proposing a set of pre-defined quality metrics according to pre-defined quality questions), automating part of the evaluation by the means of tools.A previous work of Grim-Yefsah and Diaz (2012) has shown the feasibility of automating the evaluation of quality metrics based on the social network analysis, the ability of providing a preliminary quality diagnosis, and suggestions on the improvement actions to perform.This opens an interesting perspective, since it requires establishing a correlation between the quality metrics, success factors and improvement actions.Such a tool could also help in capitalizing on quality studies, which is an important aspect for the efficiency of future quality studies and, at a higher level, for corporate memory.
From a methodological point of view, the existing approaches for the management of quality give a high-level description of their methods and associated underlying concepts, but instantiating these approaches to a specific application requires a high effort.Moreover, these methods usually lack precise guidelines to help their adoption.

Table 1 .
Quality goals and quality questions

Table 4 .
Some results after the Improve and Control steps