The Critical Success Factors Of The ERP System Project: A Meta-Analysis Methodology

Many articles have been written on enterprise resource planning success in the last two decades in both the public and private sector. An important number of empirical studies attempt to delineate the steps of ERP project integration and their specificities. These research works can be divided into two principal phases: the implementation and the post-implementation. The complex nature of the ERP system and its implementation stages lead us to investigate about the critical success factors mentioned in both phases of integration. Recently, several studies have tried to assess the success of ERP system and highlight the CSF’s based on some theoretical models. This study uses a meta-analysis methodology to highlight the principal factors leading to ERP success, and it evaluates the weight of CSF’s in the process of implementing the ERP project. Based on the best studies published in the last years about ERP success, we conduct this research to determine the most important factors highly correlated with the ERP success. Eleven CSF’s are identified in our meta-analysis and classified according to their significant importance based on the correlation coefficients finding in 32 articles focus only on the ERP system. Some criteria were selected to choose studies such as: Sample size, the availability of correlation coefficient finding (quantitative empirical data), the availability of reliable constructs (Cronbach’s alpha), and the measurement scale of each factor .


INTRODUCTION
ore than 20 years ago, one of the most important information systems software appeared; the beginning of the 90's was characterized by the emergence of ERP system, 1 considered as a strategic weapon for many companies around the world. The investment in this kind of project is under increasing scrutiny and pressure to justify their value and contribution to the performance, quality, and competitiveness of organizations (Gable et al., 2003). Currently, and after approximately two decades, all the largest business companies are now equipped with the ERP system in order to follow the environment change and business development. The integration of this project is considered as one of the most important challenges for the top management, project manager, ERP consultant, and vendor at different levels of the organization, to be able to maximize its appropriation. The ERP integration requires large investment, and it is associated with many problems in the implementation phase (Markus & Tanis, 2000). Despite the substantial investments made by organizations, its success had been minor (Davenport, 1998;Davis, 1989b;Gable et al., 2003;Sedera & Gable, 2010). In the literature review, many theoretical researches attempt to develop models to evaluate the information systems success. However, these models are not entirely appropriate for measuring ERP system success (Gable et al., 2003) for many reasons such as its specificities, characteristics, and complexity of implementation. The organization must support and manage the change introduced by the ERP system, because its integration needs an important reorganization and transformation in the business process, at both strategic and technical level. In many cases, this resistance is We will expose our conceptual model and highlight the principal success factors identified in both theoretical models and empirical studies. After that, we will explain how these CSF's are classified taking into account the theoretical background in order to justify our conceptual perception. Then, we will explain the goal of our study and why a meta-analysis methodology was adopted. The methodology that we chose is a quantitative meta-analysis to identify the most significant factors that contribute to the ERP success in the different phases, depending on one side on Hunter et al.'s (1982) meta-analysis technique, and on the other side Lipsey and Wilson's (2004) practical meta-analysis. In this part, will we justify the use of this methodology and its contribution as the first study is interested in CSF's of the ERP system success. Later, we will examine the selection procedure of empirical studies and why our intention focuses only on studies published in the information system journals. Last, we will show both the process studies selection and treatment process of the effect size of every factor and its coefficient of correlation and its weight of success contribution.
Despite, meta-analysis is often criticized as a method mixing apples and oranges, but in this case we will choose only one choice to increase the reliability of the result. The meta-analysis method gives us a quantitative output, and it could compute the effect of variables and their contribution in the organizational performance. This study will give us the opportunity to design a new categorization of the CSF's and the value of each one in the ERP process based on the best published papers in the last decade. The literature review about the success of the ERP system has shown an important number of CSF's. Some factors are present in the majority of the empirical studies but some others are not. Those present are considered as the most critical in the ERP system project success. Some researchers were interested in the following factors: WGI: Myers et al. (1997) claimed that any system success model should include workgroup impact (WGI) and consider this factor as the contributions made by the teams/groups toward organizational productivity. The workgroup impact is an important stage between the individual and the organizational impacts, many organizations place a greater emphasis on the role of teams in the workplace (Ifinedo et al., 2010;Myers et al., 1997).
II: Individual impact is one of the most important factors determining the acceptance and the adoption of the technology by users. Thus, this factor is the most difficult to define in a non-ambiguous fashion (DeLone & McLean, 1992). Many acceptance technology models are developed to investigate the question about the individual impact on the information system (Davis, 1989;Venkatesh & Davis, 2000;Venkatesh et al., 2003). In the ERP system, this factor is determinant to perform the ERP project integration. Users' involvement, competence of users, training of users, users' resistance of change, user competency, and user knowledge of the system (Gorla & Lin, 2010) are considered as individual factors in the ERP system success that must be taken into account by top management in the ERP system project.
OI: Organizational impact could be considered as a reciprocal impact between the organization and the technology. The first impact focuses on the organizational resistance, readiness of change, and organizational fit with the ERP system. The second impact could be explained by the contribution of the technology on the organizational performance improvement in operational tactical and strategic levels.
SQ: System quality plays an important role in the information system success. This quality could be defined in terms of many attributes such as, data accuracy, database contents, ease of use, ease of learning, convenience of access, usefulness of use, realization of users' requirements, system accuracy, system flexibility, system reliability, system integration, system sophistication, system features and functions, integration of system, system efficiency, resources utilization, response time, and turnaround time (DeLone & McLean, 1992). Software quality could be defined in terms of two main types of product characteristics: external quality such as, usability reliability, ease of use and usefulness, and internal quality, such as, software structure and complexity (Gorla & Lin, 2010).

IQ:
The growth of data warehouse and the direct access of information by managers and information system users increase the need for, high quality information in organizations. To assess the quality of the information there are four main dimensions: Intrinsic IQ, Contextual IQ, Representational IQ, and Accessibility IQ . This information quality is considered by DeLone and McLean (1992) as a product of the information system that could be named IS output. They propose many criteria to measure the IQ such as, accuracy, output timeliness, reliability, completeness, relevance, precision, currency, accessibility, and adaptability.

TMS:
Top Management Support is one of the most widely cited CSF's (Finney & Corbett, 2007). This factor refers to the commitment of leadership to the diffusion of innovations (Somers & Nelson, 2001) and is considered as the most critical factors in organizations embarking on ERP implementation (Al-Mashari et al., 2003). The decision to implement an ERP system is taken at the high level of organizational hierarchy, the senior management team input is crucial in the process of selecting of suitable vendor. The role of the leadership support and commitment does not end in this step of ERP system selecting, but must extend to the full implementation project (Al-Mashari et al., 2003). This factor is critical because management make fast and effective decisions, resolve conflicts, bring everyone to the same thinking to promote company-wide acceptance of the project, and build cooperation among the diverse groups in the organization (Sternad & Bobek, 2012). & Bobek, 2012); the communication and cooperation between the different stakeholders: client, consultant, and vendor (Finney & Corbett, 2007); vendor transfers of knowledge, documents, services, and support. Both vendor and consultant quality were grouped together because they present an external source of expertise to the organization regarding ERP system implementation (Ifinedo & Nahar, 2006). This factor could be considered as an external knowledge factor (Sedera & Gable, 2010), that contributes to perform the ERP integration project throughout the product life cycle (implementation, upgrade, new version integration…). The VCQ factor can substitute the SQ (Service Quality) proposed by DeLone and McLean (2003) that must be added to the IS success model. The SERVQUAL concept is defined by Tsai et al. (2012) as the overall support delivered by the service supplier; they claimed that a poor user support can translate into lost costumers and lost sales. This concept is developed in the marketing to measure the service quality to evaluate the degree of users' satisfaction using five main dimensions (Tsai et al., 2012): (1) Tangibles: Suppliers and consultants provide up-to-date hardware and software. (2) Reliability: Suppliers and consultants are reliable.
Responsiveness: Suppliers and consultants provide prompt service to users. (4) Assurance: Suppliers and consultants have the knowledge to do their job better. (5) Empathy: Suppliers and consultants have users' best interests at heart.
The main constructs used to measure the service quality in this study are: TED: A particular challenge in the ERP implementation is to select an appropriate plan for end-user training and education (Al-Mashari et al., 2003). Everyone who uses ERP systems needs to be trained on how they work and how they relate to the business process early on the implementation phase (Somers & Nelson, 2001). Many ERP system adopters need the help of the consultants during the implementation project; this transfer of consultant's knowledge to internal employees is crucial to success and facilitates the ERP system adoption (Davenport, 1998). However, inadequate training and education could be considered as a significant reason for BPR: Business process reengineering, to benefit from all the advantages and best practices provided by the ERP system, business process redesign and change is required, because ERP systems are essentially developed to improve business processes such as manufacturing, purchasing, or distribution (Al-Mashari et al., 2003). To achieve the greatest benefits provided by the ERP system, it is imperative that the business processes are aligned with the new system (Somers & Nelson, 2001). The new organizational business model after the business process reengineering is an enabling factor that contributes to the ERP success. The main appellations used to indicate the BPR:  (Bradford & Florin, 2003), and ERP Fit and alignment (Bernroider et al., 2014;Wang & Chen, 2006). The ERP system integration requires some modifications that should take into consideration both ERP system features and organization specificities; this fit between the organization and the ERP system is a critical factor of success. The integrative design of the ERP systems increases the complexity involved in source code modification (Sternad & Bobek, 2012). Organizations seek competitive advantage by aligning with other organizations, usually for their non-core business activities, and thus may intend to integrate ERP with partner systems (Ram et al., 2013a). System integration/configuration is considered as a CSF's at the deployment stage of ERP (Al- Mashari et al., 2003). This result reinforces the importance of ensuring that all the ERP modules are interfaced for the seamless operation of ERP system and allow successful implementation (Ram et al., 2013a).
In the following table we expose all the critical success factors identified in 32 articles selected for our mete-analysis. In the columns we find the eleven CSF's most cited by the researchers in the last ten years and in the lines, researchers who have studied quantitatively these factors. We can see that in one study many factors may be treated.

The Journal of Applied Business Research -September/October 2014 Volume 30, Number 5
Copyright by author(s); CC-BY 1413 The Clute Institute

RESEARCH METHODOLOGY
The literature review related to the success of ERP system provides an important number of empirical studies. Throughout the past two decades, researchers have investigated the question about the determinants of the ERP system success in order to understand better their impacts both on organizational and individual performance. The divergent results of studies about the CSF's lead us to investigate the quantitative review to determine the effect size magnitude (Lipsey & Wilson, 2004) of each factor in the success process of the ERP System and its benefit in both practice and research. The most appropriate methodology to answer this question is not to add a new study with a new result but to combine all the extant studies in a meta-analysis cumulating all research findings (Hunter et al., 1982). The choice of this methodology is motivated by both the nature of the extant results studies and the feasibility of quantifying the findings results in one study.
Meta-analysis is one of many ways to summarize, integrate, and interpret selected sets of scholarly works in various fields (Lipsey & Wilson, 2004) like, psychology, medicine, and recently in management (Chau, 1999 The studies that we can use in meta-analysis are those using a quantitative measurement of variables and reporting descriptive or inferential statistics that summarize the resulting data. The meta-analysis is a technique for encoding and analyzing the statistics that summarize research findings as they are typically presented in research reports (Lipsey & Wilson, 2004). The meta-analysis should use an effect size statistic that provides appropriate standardization for the particular research form design of quantitative finding, variables, and operationalization presented in the set of studies under investigations (Lipsey & Wilson, 2004).
The process of meta-analysis is defined by Glass as a composite process: Starting by cumulating descriptive statistics across the studies, coding study characteristics, and finally the regression of study outcome onto the coded study characteristics (Hunter et al., 1982, p. 32). In the course of doing meta-analysis, the steps would be: (1) search for and gather studies, (2) extract information from the studies, and (3) cumulate the information extracted; these are the main three steps of the meta-analysis process as defined by Hunter et al. (1982).

Strengths of Meta-Analysis
Using a meta-analysis to summarize and analyze results of research studies is more reliable than conventional research reviewing techniques (Lipsey & Wilson, 2004); there are four main reasons that constitute the advantages of meta-analysis: Firstly, meta-analysis methodology imposes a useful discipline on the process of cumulating research results; secondly, the meta-analysis summarizes key study results in a way that they are more differentiated and sophisticated than the traditional review procedures; thirdly, meta-analysis is able to find effects or relationships that are obscured in other approaches to summarize research; fourthly, meta-analysis provides an organized manner of handling information from a huge number of study findings under review.

Identifying and Selecting Studies
The meta-analysis process starts by identifying studies. To identify these studies, we use both online databases accessible via our university and Google scholar as a supplementary tool for optionally adding other

Copyright by author(s); CC-BY 1414
The Clute Institute studies that go unnoticed. Online databases used in our research are: Science Direct, JSTOR, Taylor & Francis, Wiley, and Springer link, but the majority of articles were found in Science Direct. The main subject of this step of meta-analysis process is to include as many published studies as possible to enhance the reliability of our result. In the process of identifying studies we fixed many criteria of research such as: keywords used in the article title and abstract, including only the studies published between 2002 and 2014, using the following keywords: ERP system success, CSF's, ERP system, performance, effectiveness, ERP benefits. These keywords are used in the advanced research option in the online databases using all the combinations possible. We also sought the studies that referred to IS Success model such as the DeLone & McLean model, TAM Model. We examined the references of each article for probably finding new studies.
After the first step of identifying studies using all possibilities to enhance the number of articles published in the period between 2002 and 2014, we selected only the articles that used a quantitative approach. Our metaanalysis includes studies according to two criteria: Firstly, the studies that reported the correlation coefficient; secondly we retained only the articles that treated the success of ERP System taking on consideration three levels of performance: individual, work group, and organizational performance. Taking into account these three levels of performance as a measurement criterion of the ERP System success that have been justified both theoretically and empirically by an important number of researchers. In the individual level, all studies that used TAM model assess the ERP Success in the individual impact using the following items: User satisfaction, individual performance, computer self-efficacy and user satisfaction, ERP efficacy usage and perceived usefulness.  Tables (1, 2, 3) presented above summarize the articles used in the meta-analysis and classify them using three main criteria: topic, year of publication, and journal. Table 2 shows all the articles found in the first step of collecting studies and presents the result of our research using the following key words: ERP system, ERP system success, ERP system performance, and ERP CSF's. Table 3 presents only the articles including a quantitative methodology (correlation coefficient) needed to compute the effect size.

Coding Studies
This step is focused on the extraction of the information from each study selected. To accomplish this task we use an excel table with the coded study criteria (Hunter et al., 1982) such as the date, journal, authors, and sample size. We included as many criteria as possible in our database to lead our meta-analysis. We also carefully selected the measurement items of each construct. This step was critical because the constructs are not always measured by the same items, for example IQand SQ could be measured using different items; for this reason we used Cronbach's alpha or inter-item reliability scores for each construct (Chau, 1999;Petter & McLean, 2009).
Cronbach's alpha (1) K indicates the number of items; is the sum of the variances of the items, and is the variance of total score. In our case Cronbach's alpha is indicated in each study used in our meta-analysis.
When coding our finding results, we classified them into different categories, for example the geographic context: America, Asia, Europe, and Australia. The respondents function: ERP Users, ERP project Manager, Senior ERP Manager, CIO's, and some studies focus on various hierarchical levels of respondents. Two studies used in our research have taken into account students as respondents to assess the ERP success system; these two studies measure the individual performance level using the technology acceptance model (Davis, 1989b). Therefore, we classified the selected studies into two principal categories: public and private sector, and the two studies mentioned above are classified in the education sector because they were applied on students in two different universities.
Then 32 studies with 124 effect sizes have been encoded according to the criteria that have been fixed after the step of identifying and selecting study. Our initial research produced more than 80 papers, 48 of them were eliminated because they were not empirical studies with an effect size. We saved only the empirical studies that have provided a correlation coefficient, sample size, p-value, and reliability of constructs (using Cronbach's alpha).

Copyright by author(s); CC-BY 1417
The Clute Institute

Data Analysis
In the meta-analysis there are two main families of effects size: the r family and the d family; the two most important members of the former are Persons' product-moment correlations (r) and (Zr), Fisher's r to-z transformation. The three most important members of the d family are Cohen's d, Hedge's g, and Glass's (Rosenthal, 1995). To analyze our data, we had the possibility to follow one of the three meta-analysis techniques (Hedges-Olkin, Rosenthal-Rubin, and Hunter-Schmidt methods).
Recently many sophisticated meta-analysis software were developed to help researchers in their data analysis, like (Revman, CMA etc). The advantages of using meta-analysis software are: The possibility to create and customize graphics for interpretation and the facility for data entry.
However, in our meta-analysis we will use both conventional technique using Excel as a support for computing the effect size, and CMA software (Comprehensive Meta-Analysis software) to analyze our data and to confirm results that were found before. Based on articles published in IS (e.g., Petter & McLean, 2009) using a meta-analysis, the most appropriate technique selected to analyze our data was the Hunter and Schmidt technique. After gathering the correlation coefficient from each study, this technique appears to be the most appropriate because it is based on the correlation coefficient to compute the effect size.

Cumulating Correlations Across Studies (Hunter & Schmidt Technique)
This technique considers the correlation coefficient as a subject to two principal sources of error that can be eliminated at the level of meta-analysis: sampling error and error of measurement. In our meta-analysis we will eliminate both the sampling error and the error of measurement for each construct based on the criterion reliability (as reported using Cronbach's alpha).

Eliminating the Sampling Error
Hunter & Schmidt report that if the population correlation is assumed to be constant over studies, the best estimate of that correlation is not the sample mean across studies but a weighted average in which each correlation is weighted by the number of persons in that study. The formula to estimate the population correlation is: (2) where is the correlation in study I and is the number of people in study i. The corresponding variance across studies is the frequency weighted average squared error: To compute the error variable (sampling error in the sample correlation in study i.) we use the following formula: (4) and its variance: The error variance across studies is computed using the following formula:

Copyright by author(s); CC-BY 1418
The Clute Institute where K is the number of studies and is the total sample size. And the error variance of the population correlation is computed with following formula: (7) After computing the variance, we then calculated 95% confidence interval of the correlation corrected only for the sampling error using the standard deviation (computing from the variance). Confidence intervals that don't include zero consider that the relationship is significant.
Confidence interval: 1.96 (8) Eliminating the Measurement Error Variables in management science are not perfectly measured (Hunter et al., 1982). There is always an effect of error of measurement on the correlation between variables, the error of measurement automatically lowers the correlation coefficient, and this effect must be eliminated in the meta-analysis. To eliminate this effect the true measurement scores of both dependent and independent variables must be computed: (9) where, is the correlation corrected for the sampling error, is the reliability of the independent variable and is the reliability of the dependent variable.
Based on Petter and McLean's (2009) meta-analysis, another test must be computed; this test is a homogeneity test which determines if there is heterogeneity in the correlations in a meta-analysis, this heterogeneity, often suggests that an additional variable is creating the variance and affecting the effect size. To know if the studies are homogeneous, credibility interval must be determined using the corrected standard deviation using the following formula: (10) Lower Endpoint = and Upper Endpoint = Each study contains all the information to correct correlation for attenuation due to both sampling error and measurement error. At first we corrected the correlation for the sampling error, and then we computed the 95% confidence interval of the corrected correlation from the sampling error. Secondly we based ourselves on the reliability of both independent and dependent variable to compute the average of each one over studies using Cronbach's alpha as reported. After that, we compute the correlation corrected for measurement error and its credibility interval using the formula mentioned above (11).

Meta-Analysis Result
The study presented in this work summarizes quantitatively the relationship between the CSF's and the ERP system success. After identifying the CSF's selected from empirically studies we classified them into three main taxonomy of significant magnitude, strong, moderate and weak.

Copyright by author(s); CC-BY 1419
The Clute Institute One of the benefits of the meta-analysis methodology is its ability to classify the strength of the relationships between the constructs ( , Fz = 0.310; , Fz = 0.304). We found that there is a significant relationship between CSF's and the success of the ERP system; most of these relationships are strong or moderate.

Copyright by author(s); CC-BY 1420
The Clute Institute

Failsafe N and the Funnel Plots
To investigate the publication bias, and heterogeneity, we used the funnel plots and failsafe N (See Appendix). The funnel plots are defined by Wu and Lederer (2009) as a simple scatterplots of the Fisher's Z (horizontal axis) against the standard error (vertical axis). In the absence of bias, the funnel plot shows a symmetric inverted funnel shape with effect sizes and the inverse of the standard deviation. The publication bias may not be only identified by the funnel plots, other tests must be taken into account such as failsafe N, that represents the number of additional studies (with null results) needed to render the results for that relationship non-significant at a pre-specified level (p ≤ 0.05) (Wu & Lederer, 2009). Both tests were computed for each factor to determine the publications bias. The Failsafe N shows the robustness of our results, it varies from 186 to 9509 with an average of 2112. They provide a confidence in the robustness of our meta-analysis (For example, we need 9509 additional studies with non-significant results to make the positive correlation between the system quality and the ERP success non-significant). In this study both funnel plots test and failsafe N show that the publication bias is not a problem.

Heterogeneity Test
The heterogeneity or homogeneity test (Hedges, 1982;Rosenthal, 1995) could be used as a tool in deciding whether observed effect sizes are more variable than would be expected from sampling error alone (Wu & Lederer, 2009). If there is a strong heterogeneity we have to search the moderator variables that could be considered as an explanation of the effect sizes variation. In our case we identified five main variables that could be considered as moderators: the geographic area and culture, the nature of the sector, the type of ERP software, the implementation phase, and the respondent function. (Table 4), this index describes the percentage of variation across study that is due to heterogeneity rather than the chance, on the other hand it measures the variation that is not due to chance, it can take values from 0% to 100%.
The Fisher's Z is computed by normalizing the correlation coefficients using the following formula: Fisher's z ; then to calculate the weighted average z: ; where is the sample size in study i.

Managerial Implications
This research work provides a new tool to practitioners enabling them a better understanding of the ERP system success project. Information system managers, top management, and ERP users need to understand the implication of their actions in the success process and how they contribute in the performance improvement. Thus, this work seeks to highlight the vendor and consultants contributions to perform the ERP project. To face more than three quarters of unsuccessful ERP project, organizations need to be able to evaluate the contribution of each CSF in the ERP life cycle. This need leads us to investigate this question by developing a new model that explain the relationships between the ERP partners on the one hand and propose the main factors to succeed the ERP project on the other. Eleven significant CSF were identified in this study and classified by the magnitude of effect sizes (Lipsey & Wilson, 2004) given by every one based on a robustness methodology tests. This study offers a practical understanding for organization to determining the CFS's of complex information system such as the ERP system.
While we classify the effect sizes of the eleven CSF's by the strength of relationships using three main categories of taxonomy, strong, moderate, and weak. This taxonomy by the relationship importance between the constructs gives a clear vision to IS managers.

Conclusion, Limitation, and Perspectives
The meta-analysis methodology is a recent tool considered as a subject of many critics (Laroche & Soulez, 2012). This methodology has shown its robustness in some fields like psychology and medicine. However, in management science there are a few numbers of researchers that use this methodology despite its importance and the reliability of its results, but this methodology requires a good knowledge of statistical tools and a detailed explanation of each step in the process of collecting, selecting, and coding studies.
We stress that this work is considered as the first one that tries to meta-analyze quantitatively the CSF's of ERP Success and attempts to summarize all the existing empirical researches in one study using a meta-analysis methodology. We have empirically validated our theoretical model that we proposed; and we found that our entire hypotheses were supported. Factors that we identified contribute to the ERP success. However, this contribution changes from one factor to another, but all the relationships are significant between the factors that we selected and the ERP Success. The Critical Factors of ERP system success in both pre-implementation and post-implementation phases are significant. However, some factors were found more significant than others, the factors with a high significant magnitude explain a very important role in the ERP system success and they need an important investment by organization to improve their benefits from the ERP System implementation.
This work provides important findings regarding the ERP system success; it also proposes and explains the contribution of the CSF's in the performance improvement. However, there are some limitations of this study that should be emphasized: Firstly, the methodology used in our study: The meta-analysis methodology as a new technique using only a quantitative resultants stays a subject of many critics in information system field. It can only be performed on research that performs quantitative analysis; qualitative or conceptual works must be excluded (Petter & McLean, 2009). Additionally, the number of studies found and used in this work is not to large, it could be considered as a limit of work results.