Comparative Analysis of Credit Risk Models in Relation to SME Segment

The importance of credit risk management is well known and was deeply investigated by the banking industry. There is a pressure on financial institutions to still improve their credit risk management systems, so the credit risk of a bank is an unflagging object of discussion. The aim of this article to compare the predicting abilities of several bankruptcy models to the SME segment in the Czech Republic and its subsegments medium sized, small and micro enterprises. We have focused on small and medium sized enterprises (SMEs) considering their fundamental role played in the Czech economy and the considerable attention placed on SMEs. We have chosen popular bankruptcy models that are often applied, namely the Altman Z-score, Altman model developed especially for SMEs in 2007, the Ohlson O-score, the Zmijewski’s model, the Taffler’s model, and the IN05 model. The basic form of the models was used as proposed by their authors. The results were compared using the contingency table and ROC curve. We have found that the best prediction models are Zmijewski ́s and Ohlson ́s models which use probit and logit methodologies and according to our analysis, their prediction ability is better than that of models based on discriminant analysis. Surprisingly, model IN05 designed for Czech companies provides average results only. One of the worst performing models is Altman 2007, which was created specifically for SMEs, but according to our analysis it only provides subordinates results.


Introduction
The prediction of bankruptcy and the quantification of credit risk is a subject of interest of many studies, scientific articles and publications. The ability to predict bankruptcy is a factor that eliminates credit risk of a bank. Academics and practitioners have focused their researches to improve the performance of existing bankruptcy models to achieve credit risk elimination as it is one of the biggest financial risks in the banking industry and because of the former financial crisis when bankruptcy models failed to adequately set the credit risk (Diakomihalis, 2012).
According to the introduction of the Third Basel Capital Accord (Basel III), financial crisis starting in 2008 caused that the institutions are facing the problem of bad debts (outstanding loans), therefore, they must make effort to improve their credit risk management system and find more effective ways to face credit risk.
The aim of this contribution is to compare the predicting abilities of several bankruptcy models in relation to small and medium sized enterprises in the Czech Republic. We have compared results within the SME segment as a whole and each sub-segment of SMEs, medium sized, small and micro enterprises. We have chosen the bankruptcy models which are used very often in many scientific papers. We have compared models of the Altman Z-score, Altman model developed especially for SMEs in 2007, Ohlson O-score, the Zmijewski's model, the Taffler's model, and the IN05 model.
We focus on SMEs because they are reasonably considered the most important segment of economy in many countries. For OECD members, the percentage of SMEs out of the total number of firms is higher than 97%. SMEs employ approximately two thirds of employees and create more than half of added value in EU-28 (Eurostat, 2017). Over the past decade, we have witnessed intensity in the studies on their financial health, particularly after the introduction of Basel III. Recent studies show that SMEs demonstrate capacity to drive economic development at domestic and international levels (Gupta et al., 2014). Thanks to their simple structure, they can quickly respond to changing economic conditions and meet local customers´ needs, growing sometimes into large and powerful corporations or failing within a short time of the firm´s inception. From a credit risk point of view, SMEs are different form large corporates for many reasons. For example, Dietsch and Petey analysed a set of German and French SMEs in 2004 and concluded that they are riskier but have a lower asset correlation with each other than large businesses (Dietsch and Petey, 2004).
We hypothesize that applying a default prediction model developed on large corporate data to SMEs will result in lower prediction power and likely a poorer performance of the entire corporate portfolio than with separate models for SMEs and large corporates. The motivation is to show the significant importance of modelling credit risk for SMEs separately from large corporates.
Most of the commonly used models are developed for foreign markets and because the Czech market is quite specific we suppose that it leads to lower prediction power than the application of a prediction model developed especially for the Czech market.
We have analysed 127ths non-bankrupt enterprises and around 1.3ths bankrupt enterprises during the years 2008 to 2014. We have compared quality assessment of the models by ROC curve and the classification table. We acknowledge that our analysis will be improved by developing our own default prediction model, which is a further part of our research.

Literature Review
There is extensive empirical literature on default prediction methodologies. Many authors during the last fifty years have examined several possibilities of predicting default or business failure. The seminal works in this field were Beaver (1967) and Altman (1968). The researcher William Beaver was the first to apply a number of ratios which could discriminate between failed and non-failed companies up to five years prior to bankruptcy. Altman improved Beaver´s method and assessed a more complete financial profile of firms. Altman examined 22 potentially helpful financial ratios and selected five that provide, when combined, the best overall prediction of corporate bankruptcy. The variables were classified into five standard ratio categories -liquidity, profitability, financial leverage, activity, and solvency. Being the first person to successfully develop multiple discriminate analysis (MDA) prediction model with a degree of 95.0% rate of accuracy, he is considered the pioneer of insolvency predictors. Altman´s model has been applied successfully in many studies worldwide concerning the subjects of capital structure and strategic management, investment decisions, asset and credit risk estimation and financial failure of publicly traded companies (Lifschutz and Jacobi, 2010).
For many years thereafter, MDA was the prevalent method applied to the default prediction models. It was used by many authors, for example very often cited in research literature is Taffler model developed in Great Britain in 1977(Taffler, Tishaw, 1977. However, in most of these studies, authors pointed out that two basic assumptions of MDA are often violated -the independent variables included in the model are multivariate normally distributed; the group of variancecovariance matrices are equal across the failing and non-failing groups (Barnes, 1982 andKarels andPrakash, 1987).
Another MDA model was developed by Inka and Ivan Neumaier in 1995 and is known as IN95. This model was constructed especially for the Czech market and was updated in next years. We use the last version -IN05 model which was developed in 2005 (Inka and Ivan Neumaier, 2005).
Considering these MDAs´ problems, Ohlson (1980), for the first time, applied the conditional logit model to the default prediction study. The practical benefits of logit methodology are that they do not require the restrictive assumptions of MDA and allow working with disproportional samples. After Ohlson, most of the academic literature used logit models to predict default. Another, very often cited model, which uses conditional probability, is the model by Mark E. Zmijewski (1984). He was the pioneer in applying probit analysis to predict default but, until now, logit analysis has given better results in this field. A probit approach is the same as the logit approach, only the distribution of random variables is different.
From a statistical point of view, logit regression seems to fit well with the characteristics of the default prediction problem, where the dependent variable is binary and where the groups are discrete, non-overlapping and identifiable. The logit model yields a score between 0 and 1, which conveniently gives the client´s probability of default. Despite the theoretical differences between MDA and logit analysis, academic studies show that empirical results are quite similar in terms of classification accuracy (Altman et al., 2010).
We witness a substantial increase in the number and complexity of default prediction studies due to the rapid advancement in technology and methodology. Above all, we can mention artificial neuron networks used by Angelini et al. (2007), decision trees method used by Gulnur and Fikret (2011) and hazard models used by Shumway (2001).
Recent empirical literature also gains momentum in understanding the credit risk behaviour of small firms. Altman and Sabato (2007) studied a panel of 2010 U.S. SMEs including 120 defaults in period 1994 to 2002. They chose five accounting ratio categories describing the main aspects of a company´s financial profile: liquidity, profitability, leverage, coverage, and activity. For each of these categories they created a number of financial ratios identified in the literature as being the most successful in predicting firms´ bankruptcy. Finally, five variables (one from each category) were selected with the best prediction power of SME default and a distress prediction model for SMEs was developed using logistic regression technique. However, they acknowledge the need to employ qualitative information to improve the predictive performance of their model.
Empirical literature also highlights the significance of qualitative information such as business type, industrial sector, location, age, etc. in understanding of firms´ credit risk behaviour (Grunert et al., 2005). Altman et al. (2010) took account of this issue and studied about 5.8 million SMEs, of which 66 000 failed during the observed period 2000-2007. They reported that the prediction performance of Altman and Sabato (2007) model improved by about 13% when qualitative information was added. They found that data relating to legal action by creditors to recover unpaid debts, company filing histories, comprehensive audit report/opinion data and firm-specific characteristics make a significant contribution to increasing the default prediction power of risk models built specifically for SMEs (Altman et al., 2010).

Methodology
We choose the most popular bankruptcy models, specifically the Altman´s Z-score, the Altman's model for SMEs, the Ohlson´s O-score, the Zmijewski's model, the Taffler's model, and the IN05 model. In tested models we used the original estimated coefficients used by their authors. We did not use our data for estimating coefficients in the models; therefore, we are able to use the whole dataset as a validation sample for the verification of these models.
Quality evaluation of bankruptcy models is also dependent on the determination of the so-called 'cut-off' points. This is the value above (or below) which the firm will be regarded as bankrupt. The optimal cut-off point is the value that minimizes errors of type I and II. However, everything depends on the purpose for which the model will be used. Therefore, for example, we may choose a higher cut-off limit if the request is to better characterize the companies that are going bankrupt at the expense that there will be a higher number of healthy ones wrongly ranked.
ROC (Receiver Operating Characteristic) curve is a graphical method, which is based on a square showing the relationship between true positive rate (TPR -also called sensitivity) and false positive rate (FPR -also called fall-out). TPR measures the proportion of positives that are correctly identified as such. FPR is also known as probability of false alarm, and it is calculated as the ratio between the numbers of negatives that are wrongly identified as positives. TPR is applied to the y-axis and FPR to the x-axis. ROC curve combines the values of TPR and FPR.
There are two possible extreme cases. The first case occurs when the predicted values are absolutely similar to real values. In this case, the curve copies the border of the graph beginning in the bottom left corner, going through the upper left corner and ending in the upper right corner. The second case is the exact opposite and describes the model with no predictive power. The curve in this case is a diagonal from the bottom left corner to the upper right corner of the graph. Thus, the closer the curve is to the upper left corner, the better predictive power of the model. ROC curve is closely related to AUC (Area Under Curve) indicator that numerically represents the graph and helps with comparison of two or more models. This indicator quantifies the area under the curve and is useful for comparing two or more ROC curves, because they are transformed into one measure and easily comparable. AUC ranges from 0.5 to 1, where a higher value indicates a better prediction model.
Another method is to use the classification table. The classification table is a very simple and intuitive method of assessing binary prediction models. As the name suggests, its principle is to assess the correct and incorrect classification of the individual observations and consequently the whole model. A prediction model is assessed by the proportion of correctly classified observations to the total number of observations. As discussed above, the total percentage is dependent on the determination of the cut-off value. From the classification table we are also able to identify type I and type II errors for given cut-off boundaries.
We create classification table with the most common performance metrics for each model. To measure the overall performance of the models we use the area under the receiver operating characteristic (ROC) curve (AUC). This measure allows us to compare individual models without the need of calibrating their cut-off boundaries.

Compared Models
There are many different versions of the compared models. We used the original forms of the models as proposed by their authors (a detailed description of each model is provided in Appendix 1).
Each model has different variables with different weights. The authors measure most of all earning, liabilities and assets in various concepts with different indicators. All authors give the biggest attention to assets volume and measure it especially with earnings and liabilities. Altman´s Z-score measures the assets

Data
The comparative analysis contains a sample of financial data for 113,717 SMEs gathered from Bisnode database over the period 2008-2014. To create this sample, we first assess the number of defaulted firms contained in the Bisnode database during the selected period and we found 729 companies (with no-missing data) that failed.
When analysing business failure, it is extremely important to distinguish between failure and closure. Watson and Everett (1996) mentioned that closing firms could have been financially successful but closed for other reasons: the sale of the firm or the owner's personal decision to accept employment with another firm, to retire or the like. To define failure, they created five categories: ceasing to exist (discontinuance for any reason); closing, or a change in ownership; filing for bankruptcy; closing to limit losses; and failing to reach financial goals. Headd (2003) found that only one third of new businesses closed under circumstances that owners considered unsuccessful. We think that it is essential to carry out this kind of analysis before starting with the comparative analysis. Separating cases of closure from the ones of failure improves the quality of the available information and helps exclude possible outliers from the sample and avoid biases. In this paper we have taken into account only SMEs that entered into liquidation.
We perform our analysis on the whole SME sector and we also subset SMEs into three smaller segments, namely small, medium, and micro enterprises. Figure 1 shows a pie chart representing shares of individual segments in the whole dataset. We can see that almost two thirds of the companies are characterized as micro enterprises, 27% are small enterprises and there are only 9% of medium sized enterprises in our sample.

Figure 1 Individual segments in SMEs
Source: Author's calculations The first step was to calculate some important descriptive statistics that characterize our data in the SME sector and in each individual segment separately. Table 1 contains this information for SMEs as a whole, and medium, small, and micro enterprises separately. Because Ohlson's model requires longer investigated period, it significantly decreases the number of usable observations to 30,922. To handle this issue, we use the full dataset to compare all models excluding Ohlson's model. Then the restricted dataset is created, and all models including Ohlson's model are analyzed using this data. Results are evaluated separately for both datasets. This helps us to use as many observations as possible and also avoid bias that could occur when comparing models applied on different dataset. Basic characteristics for the restricted dataset are presented in Table 2. Interesting fact present in both datasets is that the bankruptcy rate in each segment steadily decreases as we move to smaller companies. Therefore, smaller companies are less likely to go bankrupt. In other words, the bigger the company, the higher probability of bankruptcy. We hypothesized the opposite trend -the bigger the company is, the lower probability of bankruptcy. The results reject this hypothesis.

Results
At first we created classification tables for all market segments. The results are presented in Tables 3, 4, 5, and 6. We calculated the most common measures, specifically accuracy (ACC), false positive (FP, type I error), false negative (FN, type II error), true positive rate (TPR), and false positive rate (FPR).
Accuracy measures the percentage of observations that were classified properly. Because we only have a low percentage of bankrupt companies, the result is highly dependent on the selected cut-off boundary. Each model has different recommended cut-off boundaries which determine if the company is considered bankrupt or healthy.    These results in classification tables do not tell us which model is better. But they evaluate sensitivity of their cut-off boundaries. Setting the right cut-off boundary depends on our goals and needs. In general, it is more preferred to mark a healthy company as bankrupt than vice versa. Selecting the right cut-off boundaries is essential for successful predictions and it should be closely considered.
To compare individual models with each other we use AUC measure. AUC stands for "area under curve". It represents the area under ROC (receiver operating characteristic). AUC is a number between 0.5 and 1, where a higher AUC means better performing model. Table 7 shows results for the full dataset and Table 8 for the restricted dataset.

Conclusions
This study analysed the most popular bankruptcy models and their performance for Czech SMEs. We have investigated the SME segment as a whole and its three subsegments, namely medium sized, small, and micro enterprises. The aim of the comparative analysis was to evaluate the original forms of the chosen models and recommend a preferable approach for Czech SMEs.
Our findings confirm our hypothesis and several suggestions arise from them. The segment of small and middle enterprises should be evaluated separately from large corporate; moreover, a new bankruptcy model especially for Czech SMEs should be developed. Our results suggest using probit and logit methodologies rather than discriminant analysis, as according to AUC measure, the best performing models were Ohlson or Zmijewski model, which use the probit and logit methodology.
The second part of our analysis is highly dependent on the cut-off boundaries which determine if the company is considered healthy or bankrupt. The results in classification tables are fairly similar for each segment. Models like Taffler, Altman and Altman 2007 use too low boundaries, so they can predict healthy companies precisely. Ohlson´s model, IN05 and Zmijewski´s model can predict bankrupt companies precisely. This is a very important point to have in mind when using these models as using different cut-off boundaries greatly affects the results.
An interesting fact present in both datasets is that the bankruptcy rate in each segment steadily decreases as we move to smaller companies. Therefore, smaller companies are less likely to go bankrupt. In other words, the bigger the company, the higher probability of bankruptcy. We hypothesized the opposite trend -the bigger the company is, the lower probability of bankruptcy, but the results reject this hypothesis.
In this study we did not find any significant differences between individual segments of Czech SMEs. For anyone interested in this topic, we recommend using Ohlson or Zmijewski model and set cut-off boundaries based on the aim of the study. The next step in our research will be to re-estimate coefficients for analysed companies and develop a new model which should provide the best possible results.