This paper was originally published in Liiketaloudellinen aikakausikirja (1995).
Mathematical programming has been suggested by several authors as a method for discriminant analysis. Most of the studies published to date focus on the technical development of the method, whereas empirical testing has received very little attention. The present study aims at testing one of the basic model formulations on a large sample of firms, the task being to predict business failure. Our findings suggest that the mathematical programming model performs at least as well as logistic regression.
Articles proposing mathematical programming (MP) as a method for discriminant analysis have appeared at irregular intervals in scientific journals for more than twenty years. Grinold [6], who in the early 1970s formulated several model variations for pattern classification and cluster analysis, is one of the pioneers. Freed and Glover [2, 3] revived interest in this field in the early 1980s, and judging from the list of references in, e.g., Ragsdale and Stam [8] a strong interest among many researchers survived through the 1980s into the early 1990s.
What interests most researchers is the fact that under certain circumstances some model formulations are afflicted by a propensity for degeneracy and instability. Most of these problems have to do with how the weights of the estimated function are normalized. Several remedies have been suggested. In this study we choose a simple normalization procedure in the form of a constraint that is added to the basic Freed-Glover model. The procedure was employed as early as 1974 by, among others, Pekelman and Sen [7] in a study of brand choice. This was proposed much later later by (Freed and Glover [4]). Several other suggestions have been made later in the literature (among others, Glover [5]; Ragsdale and Stam [8]); Rubin [9], but since these complicate computations substantially, and since we feel that the earlier suggestions have not been sufficiently tested, we focus on the one mentioned above. What this entails in detail will be described in the next section.
Apart from problems of a technical nature, there may also be a problem of inferior performance. One of the few empirical tests that have been carried out suggests that the performance of the original Freed-Glover model does not seem to be as good as that of statistical techniques such as logistic regression (logit) and sequential partitioning (Srinivasan and Kim [10]). Several ways to improve the performance of the model have been suggested in recent years (among others, Glover [5]; Ragsdale and Stam [8]). The problem with these suggestions is that they also substantially increase the number of computations, which reduces the practical usefulness of the method. The rationale for developing MP models for the discriminant problem was initally that the models were very simple. Since there already exist complicated statistical techniques that function well, one could seriously question the need for developing alternative models that do not have the advantage of being simple and easy to use. This is the main reason why we choose to focus on an augmented version of the basic Freed-Glover model. An additional reason is that we feel that the original version of this model has not been adequately tested. As Eisenbeis [1] has pointed out, the sample Srinivasan and Kim used was rather small, which makes it premature to draw any firm conclusions about the empirical performance of the method.
The present study aims at cross-validating the estimated discriminant function on a large sample of Finnish firms and comparing it with a logit function in a bankruptcy prediction setting, the hypothesis being that the predictive ability of the MP model is at least as good as that of the logit model estimated with the aid of the default version of the BMDP program LR.
The MP model to be tested in this study aims at classifying firms into two categories, failed and non-failed ones, by finding a breakpoint for the weighted sum of a set of financial ratios. The weights are also unknown and will be determined by the model. In the case of binary classification, the ideal situation is that the weighted sums of financial ratios representing non-failed firms are higher than the breakpoint, whereas the sums representing failed firms are lower than this. In practice, however, the situation is not this clear-cut: there may be some failed firms that perform better in terms of the selected ratios than some non-failed firms. This means that the weighted sums of the ratios can be higher for some failed firms than the breakpoint, and vice versa for some non-failed firms. The objective is then to determine the breakpoint and the weights so that these violations of the ideal situation will be minimized.
The last constraint, which sets the sum of the weights to 1, is used to normalize the raw scores of the ratios and to prevent a degenerate solution in which all weights are equal to zero. This constraint was used by Pekelman and Sen [7] in a model of brand preference expressed in pairwise comparisons. This is also one of the suggestions Freed and Glover make in [4]. In the original Freed-Glover model the breakpoint B was treated as a constant which could result in a degenerate solution.
Since financial ratios representing profitability, liquidity and leverage have often turned out to be good predictors of bankruptcy, we decided to limit the selection of ratios to one per category. The selected ratios are: (1) net income plus depreciation/total assets (NIPDTA), (2) quick assets less current liabilities/total assets (QALCLTA), and (3) total debts/total assets (TDTA). Since the third ratio can be expected to be higher for failed firms than for non-failed firms, the weight of this ratio can be expected to be negative. The logit model can handle this automatically, but not the MP model; all unknown variables in the latter type of model must be non-negative. Instead of TDTA we therefore used its inverse, named TATD.
The Central Statistical Office of Finland (CSOF) supplied us with data for the study. The ratios were calculated based on unadjusted financial statement data and stored on a floppy disc. Generally, it may be desirable to adjust raw data, but since we are interested in a comparison of two competing methods, this should not be a problem.
Focusing our attention solely on the manufacturing industry, we used a sample of matched pairs to estimate the discriminant functions, a procedure that is common and economical. Since there were very few failed firms in the CSOF database for a particular year, we decided to include firms that went bankrupt in 1986, 1987, 1988 and 1989. In this manner we obtained 55 failed firms. Adding 55 randomly selected non-failed firms for fiscal 1986 resulted in an estimation sample comprising 110 firms. The estimated weigths and breakpoints are shown in Table 1.
Table 1. Estimated weights and breakpoints.
MP Logit
Constant -5.228
NIPDTA 0.86275 17.460
QALCLTA 0.06180
TATD 0.07545 3.646
Breakpoint 0.11815 0.458
As can be seen, the logit model includes two ratios only: the profitability ratio, NIPDTA, and the leverage ratio, TATD, whereas the MP model includes all of the three ratios used as input variables: NIPDTA, QALCLTA and TATD. The breakpoint for the logit model was set to 0.458 because this value minimized both the number of misclassified failed firms and misclassified non-failed firms in the estimation sample.
The estimated discriminant functions were cross-validated on data for fiscal 1987. We wanted the holdout sample to be large and the proportion between failed and non-failed firms to be an approximation of the proportion in the population. This resulted in a holdout sample comprising 1,000 firms, 78 of which failed in 1987, 1988, 1989 and 1990. Thus, the task was to predict bankruptcy within three years or less. This means that the models were subjected to a severe test, especially since 1990 marks the onset of a deep recession in Finland.
The classification rule used for the purpose of cross-validation was the following: If the weighted sum (or the logit probability of success) was greater than the breakpoint, then the firm was classified as a non-failed firm, otherwise it was classified as a failed firm. The results of the comparison of the estimated discriminant functions are shown in Table 2.
Table 2. Results of the predictive test.
Failed firms Number
classified by both
methods
as failed 51
as non-failed 20
classified
differently
MP: failed; 6
logit: non-failed
MP: non-failed; 1
logit: failed
Total 78
Non-failed firms
classified by both
methods
as non-failed 673
as failed 214
classified
differently
MP:non-failed; 22
logit: failed
MP:failed; 13
logit: non-failed
Total 922
As can be seen, application of the MP discriminant function resulted in fewer misclassified failed firms (6-1=5) as well as fewer misclassified non-failed firms (22-13=9). Translated into an ordinary classification table, the results are as shown in Table 3. A comparison is also made with classification results from the estimation sample.
Table 3. Percentages of misclassified firms.
Estimation Holdout
sample sample
MP Logit MP Logit
Failed firms 23.6% 21.8% 26.9% 33.3%
Non-failed firms 21.8% 16.4% 24.6% 25.6%
Total 22.7% 19.1% 24.8% 26.2%
The holdout sample results indicate that the MP function had a "hit rate" for failed firms equal to 73.1%, whereas this rate for the logit function was equal to 66.7%. For non-failed firms, the "hit rates" were 75.4% and 74.4%, respectively. Thus, this test can be said to support the hypothesis that mathematical programming performs at least as well as logistic regression. In fact, the test results showed that the former even performed better than the latter, but one test cannot provide conclusive evidence as regards the superiority of the former method. Overall, the predictive ability of both models was rather poor, but this is not surprising considering that the task was to predict bankruptcy within three years or less.
It is interesting to note that the logit model performed considerably better than the MP model when applied to the estimation sample. Obviously, the logit estimation procedure provides a "tighter" model of the data than the MP procedure, but this can lead to diminished predictive ability if, as in this case, the failure rate increases and the raw scores of key financial ratios deteriorate as a result of recession, resulting in considerable differences between the estimation and holdout samples.
The purpose of this study has been to test whether mathematical programming performs as well as logistic regression when the task is to predict bankruptcy. In the empirical test carried out here on a large holdout sample (78 failed firms, 922 non-failed firms), the mathematical programming function performed better than the logit model both for failed firms and for non-failed firms, confirming our hypothesis that mathematical programming performs at least as well as logistic regression in terms of ability to predict bankruptcy.
1. Eisenbeis, R. (1987) Discussion, Supplement to Srinivasan, V. and Kim, Y. H. (1987) Credit Granting: A Comparative Analysis of Classification Procedures. J. Fin. XLII (3), July, 665-680.
2. Freed, N. and Glover, F. (1981a) A linear programming approach to the discriminant problem. Dec. Sci. 12, 68-74.
3. Freed, N. and Glover, F. (1981b) Simple but powerful goal programming formulations for the discriminant problem. Eur. J. Opl Res. 7, 44-60.
4. Freed, N. and Glover, F. (1986) Resolving Certain Difficulties and Improving the Classification Power of LP Discriminant Analysis Formulations. Dec. Sci. 17, 589-595.
5. Glover, F. (1990) Improved Linear programming Models for Discriminant analysis. Dec. Sci. 21, 771-785.
6. Grinold, R. C. (1972) Mathematical Programming Methods of Pattern Classification. Mgmt Sci. 19 (3), November, 272-289.
7. Pekelman, D. and Sen, S.K. (1974) Mathematical programming Models for the Determination of Attribute Weights. Mgmt Sci. 20 (8), April, 1217-1229.
8. Ragsdale, C. and Stam, A. (1991) Mathematical programming Formulations for the Discriminant problem: An Old Dog Does new Tricks. Dec. Sci. 22, 296-306.
9. Rubin, P.A. (1991) Separation Failure in Linear Programming Discriminant Models. Dec. Sci. 22, 519-535.
10. Srinivasan, V. and Kim, Y. H. (1987) Credit Granting: A Comparative Analysis of Classification Procedures. J.Fin., XLII (3), July, 665-681.
Jump back to:
All feedback welcome. The authors can be contacted through E-mail:
jan.wallin@wasa.shh.fi stefan.sundgren@wasa.shh.fi