Working Paper 153

An Unpublished Paper in 1983

This paper was rejected by two major journals and has not been revised since 1983. I considered this one of my better papers over the years, so now I am providing it for the world to judge. It is somewhat critical of eigenvector scaling in Analytic Hierarchy Process (AHP) applications to decision models. This is a conclusion that some promoters of AHP did not want published.

COMPARISONS OF EIGENVECTOR, LEAST SQUARES, CHI SQUARE,
AND LOGARITHMIC LEAST SQUARES METHODS
OF SCALING A RECIPROCAL MATRIX

Bob Jensen at Trinity University

Table of Contents

Abstract

1. The Bootstrap Paradox

2. Four Matrix Scaling Methods

3. Element Preference Reversals (EPRs)

4. Strong Row Preference Reversals (RPRs)

5. Moderate Row Preference Reversals

6. Weak Row Preference Reversals

7. Magnitude (Distance) Threshold Consistency Adjustments

8. Degeneracy and Nonuniqueness of LSM Scalings

9. The Chi Square Method

10. The Logarithmic Least Squares Method

11. Summary and Conclusion

References

Exhibit 1

Exhibit 2

Exhibit 3

Abstract

This paper compares four reciprocal matrix scaling methods on five criteria as defined below:

Analytical and empirical results in this paper claim the following for moderate to high levels of r_ij response inconsistency:

* (New proofs are provided in Exhibit 2 and Exhibit 3).

No method is Good or Best on all criteria. Even when other methods exist that avoid EPRs and RPRs of moderate form, none of the four methods above will always do so in practice. It is difficult to make sweeping generalizations. For example, although X²M usually avoids EPRs and RPRs at a lower Euclidean distance than EM, it certainly cannot be relied upon to do so in all instances. Although LLSM solutions are usually very close to EM, such is not always the case. This paper makes a case for presenting and comparing different scaling methods outcomes.

1. The Bootstrap Paradox

Professors Saaty and Vargas [16] endeavor to make a case that the eigenvector method (EM) utilizing the principal right PR-eigenvector component u_i / u_j ratios is the "best" method of adjusting for inconsistency in paired comparison response judgment inconsistency. In an earlier paper [15, p. 24], they state (emphasis added): "In fact, it (EM) is the only method that should be used when the data are not entirely consistent." In my paper [10], I tried to point out that this is subject to debate. Although I have not seen any of the unpublished papers [1, 2, 3, 11, 18] cited by Saaty and Vargas in [16], I suspect that those papers also sought to make a point that there are alternatives to the EM method.

I am reminded of an extensive debate that ensued years ago concerning which metric is "best" to use in paired comparison association (distance, similarity, dissimilarity, covariation) coefficients to utilize in identifying "natural" clusters or subgroupings of entities in cluster analysis and numerical taxonomy. Sneath and Sokal [17, p. 117] note:

The intensive development of numerical taxonomy since the middle of the 1950's has resulted in the rapid elaboration of such coefficients, frequently on an empirical basis without adequate theoretical justification.

In analogous fashion, Saaty and Vargas [16] take three empirically analyzed matrix scaling metrics (from among many that might be defined) and attempt to give a theoretical justification that EM is the best of all possible alternative methods. But there is no uniformly "best" metric any more than there is a "best"metric (coefficient) for cluster analysis. The well-known "bootstrap" paradox of cluster analysis applies as well to the Saaty and Vargas [15, 16] problem. Friedman and Rubin [4, p. 1162] emphasize this in cluster analysis:

Here we begin to see the "boostrap" nature of the problem. If we knew the appropriate groups, then we could define the appropriate distance. If we knew the appropriate distance, then we would be much closer to knowing the groups.

In an analogous fashion, if we knew the appropriate response matrix scaling (e.g., priority scaling of rows) or ranking (e.g., priority ranking of rows) then we could define the appropriate consistency adjustment metric. If, on the other hand, we knew the appropriate consistency adjustment method, then we could derive the optimal matrix row scaling or ranking.

Resolving the bootstrap paradox entails somehow obtaining additional bootstrap information above and beyond that contained in a paired comparison response matrix [R] that we are attempting to scale. Both my paper [10] and the Saaty and Vargas [16] paper serve to focus upon what additional information must be sought. If, for example, the [R] matrix is obtained from paired comparison judgments of a respondent, then the respondent must supply added information concerning his or her preferences for ranking-sensitive or magnitude-sensitive adjustments for response inconsistency.

Consistency adjustment of inconsistent r_ij responses may entail confronting the respondent (or turning to the user of the results) for the following added bootstrap information beyond that contained in an inconsistent, n-dimensional response matrix containing r_ij initial paired comparison response judgements:

RLIMS Question:

(1) Are there lower r_i^/_j and upper r_i^/_j^/ threshold limits (termed here as RLIMS) such that any consistency adjusted surrogate s_ij for r_ij that lies outside (r_i^/_j < s_ij < r_i^/_j^/) is deemed inappropriate? For example, r_i^/_j and r_i^/_j^/ might be set at the extreme bounds of the original r_ij response scale imposed upon the respondent.

EPR Question:

(2) Which, if any, r_ij > 1 (or r_ij <1) element preferences may be reversed to achieve consistency? This will be termed an Element Preference Reversal (EPR).

Strong RPR Question:

(3) Which, if any, row preferences may be reversed in order to achieve consistency when r_ik > r_jk for all k=1,...,n? This will be termed a strong Row Preference Reversal (RPR).

Moderate RPR Question:

(4) Which, if any, row preferences may be reversed in order to achieve consistency when r_ik > 1 for all k=1,...,n? This will be termed a moderate Row Preference Reversal (RPR).

Weak RPR Question:

(5) Which, if any, row preferences may be reversed in order to achieve consistency when the consistency solution differs from limiting row priorities obtained from raising the [R] matrix to an infinite power? By definition, the EM solution cannot have this form of weak Row Preference Reversal (RPR).

Uniqueness Question:

(6) In the LSM method, if alternative optimal consistency adjustment surrogates for r_ij values exist, which surrogates are preferred? This is not a practical problem, however, since nonuniqueness of an optimal LSM solution will rarely, if ever, be encountered in practice.

Consistency Question:

(7) Are any feasible consistency adjusted surrogates for r_ij responses acceptable? Respondents may want to remain inconsistent.

Saaty and Vargas [16] argue that the EM scaling is better than LSM scaling because EM scaling avoids strong RPRs and LSM may result in strong RPRs. This is incorrect. Later on it will be proved that LSM, like EM, always avoids strong RPRs. Their argument in any case assumes strong RPR avoidance is more important than RLIMS violations. This type of judgment is best left to respondents or users of the analysis who must supply the added bootstrap information listed above.

2. Four Matrix Scaling Methods

Suppose we define the following in the notation of my earlier paper:

To this I will add logarithmic least squares and chi square method consistency adjustments defined below:

See Exhibit 1

In Exhibit 1, the above scaling solutions are compared for the wealth of nations data focused upon in my earlier paper. This illustrates how the scaled values of rows (countries) may differ rather greatly even when the row (country) priority rankings are identical under all methods. For reasonable levels of inconsistency, all methods often yield the same rankings. But inconsistency must be almost negligible (e.g., CR < .01) for the scaled differences between u_i, y_i *, x_i *, and w_i * to be negligible. Initially in this paper I will focus only upon EM versus LSM scalings. Discussions of X²M and LLSM will follow in due course.

3. Element Preference Reversals (EPRs)

In my viewpoint, the most important advantage of the EM scaling approach is that it has a strong propensity to avoid response element preference reversals (EPRs). In other words, EM strongly tries to keep u_ij > 1 (or u_ij < 1) whenever r_ij > 1 (or r_ij < 1), where r_ij = 1 depicts equivalence in preference. In contrast, the LSM method is more concerned with consistency adjustment distance (error) squared (r_ij - w_ij) ². Although LSM commonly avoids EPRs, it does not have as strong a propensity to do so relative to EM scaling. The above avoidance of EPRs in response matrix elements is termed by Saaty and Vargas [16, p. 4] as "preserving rank weakly." This is poor terminology since EPR avoidance is the most difficult form of rank preservation. Unfortunately, all EM, LSM, X²M, and LLSM approaches for converting [R] to unit rank will not always avoid EPRs even when a feasible rank one consistent matrix may be derived that avoids all EPRs. For example, both EM and LSM reduce r₁₂ = 2 to u₁₂ < 1 and w₁₂ * < 1 thus reversing the respondent's preference for i=1 versus j=2 in the following situation:

In this case, EPRs arise for r₁₂ = 2 and r₂₁ = 0.5. Note that EM, in particular, fails to avoid the above EPRs even though feasible consistency adjustments that do so exist. For example, the following consistency adjusted surrogates avoid all EPRs in the above r_ij elements:

The point here is that solutions that avoid EPRs can be found even when EM fails to avoid EPR type rank reversals. Such avoidances, however, usually arise at very high ESS costs. Even though EPRs may arise using EM scaling, the EM approach strongly resists them relative to LSM. This, in my judgment, is the main advantage of EM over LSM if EPR avoidance is deemed paramount.

4. Strong Row Preference Reversals (RPRs)

Avoidance of strong RPRs (when r_ik > r_jk for all k=1,...,n) is termed "preserving rank strongly" by Saaty and Vargas [16, p. 4]. They incorrectly argue that LSM will not always avoid strong RPRs, which thereby makes LSM inferior to EM. They present an illustration for a purported "counter example" in which EM avoids a strong RPR and the LSM reverses Row 1 and 2 priorities. Their LSM solution, however, is incorrect and both EM and LSM scalings turn out to satisfy the strong RPR criterion in this case where Row 1 preference to Row 2 is "intuitively obvious" [16, p. 10]:

See Exhibit 2

In Exhibit 2, I present a proof that LSM optimal solutions always avoid strong RPRs. In this Exhibit, I define DE_ij as the change in ESS that arises if the w_i and w_j elements are transposed in any ESS solution vector [w₁,...,w_n] that is not necessarily optimal. The theorem states that, if Row i has strong form dominance over Row j, then any solution in which w_i < w_j cannot be LSM optimal, because DE_ij > 0 implies that ESS can be reduced by transposing the w_i and w_j values in the solution vector.

5. Moderate Row Preference Reversals

Row i dominance in a "moderate" form was defined earlier as arising when r_ik > 1 for k=1,...n, although presumably at least one r_ik > 1 such that the Row i is preferred to at least one other row. In general, neither EM nor LSM may always avoid moderate RPR type row ranking reversals. For example, consider the following matrix:

In the above case the respondent clearly indicated that Row 1 was preferred to all other rows since r₁₂ > 1, r₁₃ > 1, r₁₄ > 1, and r₁₅ > 1. However, both the EM and LSM scaling outcomes give the highest importance to Row 2. This is a moderate RPR violation that fails in rank preservation for EM as well as LSM.

Even though both EM and LSM are prone to moderate RPR-type preference reversals, the EM method is more resistant to such reversals. For example, in the matrix below EM avoids a moderate RPR, whereas LSM fails to preserve rank priority of Row 1 over Row 2:

For lower levels of inconsistency, however, LSM also avoids moderate RPRs. LSM tends to do much better at avoiding moderate RPRs than EPRs.

6. Weak Row Preference Reversals

Lastly, there are [R] matrices where there are no "intuitively obvious" row preferences, e.g., because for any pair of Rows i and j there are some r_ik > r_jk and other r_ik < r_jk. When [R] is taken to an infinite power, however, the normalized row sum limits may indicate that Row i "dominates" Row j. This will be termed "weak form" row dominance. Since u_i components of the PR-eigenvector are those normalized row sum limits of the infinite power matrix, the u_i scaling solutions under EM always, by definition, preserve weak RPRs. To argue as Saaty and Vargas do in [16, pp. 10-11] that EM is thereby a better method entails circular reasoning. EM is better only if one accepts avoidance of weak RPRs is paramount relative to other possible criteria.

Because it is primarily concerned with rank preservation, the EM scaling approach is better suited to situations when r_ij responses are ordinal rankings rather than ratio scaled. For example, in [9] I discuss the use of ordinal responses where r_ij = 2 if i is preferred to j, r_ij = 1/2 if j is preferred to i, and r_ij = 1 if they are equally preferred. With such an arbitrary coding scheme for ordinal responses, the LSM is inappropriate whereas the EM may be more appropriate provided the u_i ranks are utilized and their scaled scores are otherwise ignored. An example from [9] is shown below:

Recall, however, that EM may not preserve rank even when other scaling methods do preserve rank, i.e., the EM is generally a good but not always optimal rank preserving scaling method that avoids EPRs and RPRs of moderate form.

7. Magnitude (Distance) Threshold Consistency Adjustments

In my earlier paper [10], I pointed out some key advantages of LSM scaling. Nowhere did I appeal to or believe that "there is something immutable (or holy) about the Euclidean metric" [16, p. 3]. That would obviously be nonsense. However, the Euclidean metric is a commonly used metric of distance and, to the extent that one can return to a respondent and compare
Ö (r_ij - w_ij *)² versus Ö (r_ij - u_ij)² Euclidean distances, it may be helpful to the respondent in judging whether u_ij surrogates derived by EM or w_ij* surrogates derived by LSM are more in line with his or her preferred consistency adjustments.

Whereas EM scaling tries to preserve EPR and RPR criteria on rank (ordinal) preservation, the LSM method tries to minimize the ESS sum of square deviations between inconsistent responses and their consistency-adjusted surrogates. Like any least squares criterion, LSM in this context is very sensitive to outliers. LSM strongly avoids enormous consistency adjustments even if it sometimes incurs avoidable EPRs or RPRs. In contrast, EM strongly avoids EPRs and RPRs even if EM sometimes must force enormous consistency adjustments upon certain matrix responses. However, even when LSM and EM both preserve EPR and RPR type element and row priorities, the EM method makes larger than necessary consistency adjustments. For example, in the following matrix, the EM method makes an unnecessarily enormous r₁₃ consistency adjustment from r₁₃ = 9.00 to u₁₃ = .778/.041 ~ 18.7:

In contrast, the LSM solution has the same row priority ranks as the EM solution, but preserves this ranking with only a consistency adjustment from r₁₃ = 9.00 to w₁₃ * = .812/.081 ~ 10.0.

The main issue between EM and LSM arises in only isolated cases where EPR or RPR type rank reversals arise with the LSM but not with EM. An EPR arises below (where LSM reverses the r₂₃ > 1 priority to w₂₃ * = .097/.124 ~ 0.781 < 1) in order to minimize ESS:

In this case an EPR arises under LSM but not EM (where u₂₃ = .207/.058 ~ 3.57 > 1 is consistent with r₂₃ > 1). However, in order to avoid this EPR, the EM makes an extreme consistency adjustment of r₁₃ from r₁₃ = 5.000 to u₁₃ = .735/.058 ~ 12.651. In contrast, the LSM requires a much smaller adjustment from r₁₃ = 5.00 to w₁₃ * = .779/.124 ~ 6.265.

Added bootstrap information must be obtained when LSM yields EPRs or RPRs that do not arise in EM scalings. This bootstrap information must also be sought when other methods yield different solutions of interest. There cannot be enough information in [R] alone to resolve these issues. Bootstrap information must be obtained concerning the relative importance of rank preservation vis-a-vis magnitudes of consistency adjustments. For example, it may be necessary to return to the respondent that supplied the original r_ij judgments and require him or her to choose the preferred u_ij, w_ij *, p_ij *, y_ij *, or other possible surrogates for inconsistent r_ij values.

Another approach is to have the respondent accompany each r_ij response with r_i^/_j and r_i^/_j^/ lower and upper bounds (termed RLIMS) such that any consistency adjustments in excess of these RLIMS are viewed as excessive. The objective of the analysis would then be to find a consistency adjustment method yielding a rank one surrogate matrix for [R] that simultaneously: (i) satisfies RLIM bounds, (ii) avoids EPRs, and (iii) avoids RPRs. If alternative u_ij, w_ij *, _ij *, y_ij *, or other surrogates for r_ij satisfy (i), (ii), and (iii) above, then additional criteria must be imposed for choosing the "best" surrogate alternatives.

As pointed out in my earlier paper, it seems somewhat reasonable to assume that RLIMs at a minimum do not fall outside the original response bounds imposed upon the respondent, e.g., [1/9 < r_ij < 9]. Such RLIMs, however, are often very restraining on consistency adjustments, especially EM, LLSM, and X²M scaling which commonly violate response scale bounds when making implicit consistency adjustments. LSM is much better at staying within those bounds.

8. Degeneracy and Nonuniqueness of LSM Scalings

There is a uniqueness problem with LSM that I was not aware of in the first draft of [10]. I subsequently discovered nonuniqueness that arises whenever the least squares objective function is nonconvex. Two referees of that paper also raised this issue. Taken to extreme forms, nonconvexities may also give rise to a type of degeneracy resulting whenever there is "no information" in [R] to justify any differential importance weightings of its rows. Degeneracy is not likely to be encountered in practice. If it is encountered, the [R] consistency ratios (CR values) and maximum eigenvalue will be absurdly high.

Response matrix [R] degeneracy exists whenever CR values are extremely large and u_i = 1/n for all components of an n-dimensional PR-eigenvector. This situation arises when the respondent is so highly inconsistent that there is no basis whatsoever to assign differential priority scores to rows of the response matrix. In this unique instance in a n-dimensional response matrix, there are n different LSM solutions that have differential least squares weightings. Use of any one of the LSM solutions should be avoided.

By way of illustration, consider the absurdly inconsistent responses below along with alternative minimum ESS values:

The respondent is clearly so highly inconsistent that there is no basis whatsoever to justify a higher scaling of one row over any other. This is correctly indicated by the EM scores being all the same. The LSM solution, however, is not unique and each LSM solution assigns misleading differential scores.

As indicated previously, degeneracy is not likely to be encountered in practice. It is easily detected and, thereby, easily avoided if it should ever arise. In fact, if it is detected it is highly unlikely that the respondent's paired comparison judgments should be taken seriously.

LSM nonuniqueness, however, may arise even when there is no degeneracy, i.e., when the PR-eigenvector components are unequal suggesting a basis for differential row importance weighting. Nonuniqueness of LSM solutions, when there is no degeneracy, does not require the absurd levels of inconsistency that are encountered with degeneracy. LSM nonuniqueness is not likely to be encountered in practice. However, nonuniqueness is possible and LSM computer programs should be adapted to detect its presence and to generate the alternative LSM optimal solutions. Consider the following matrix in which LSM yields two solutions rather than one:

The LSM in this case presents a dilemma. One means of resolving the problem would be to return to the respondent and explain that two quite different LSM consistency adjustment solutions minimize the least squared error. The respondent would then have to choose which solution, if any, is more in line with his or her revised judgment. Of course this is somewhat an academic problem since nonunique LSM solutions are much less likely to arise at reasonable levels of inconsistency (e.g., when CR < .10) than for absurd inconsistent matrices such as the above example where CR = 1.149 suggests that no mathematical consistency adjustments should take place. Nonunique LSM solutions are also less likely to arise in larger response matrices where it is quite unlikely that responses will have a symmetry required to give rise to nonunique LSM solutions.

9. The Chi Square Method

My graduate assistant, Francis Wu, suggested that we overcome the possibilities of nonuniqueness in LSM scaling by instead minimizing chi square in the form:

This not only results in unique y_i * scaling solutions that are based upon squared deviations, but X²M appears to strongly resist EPRs and RPRs of moderate and weak form. In Exhibit 3 a proof is provided to show that X²M always preserves rank in terms of strong form RPRs. In the degenerate [R] matrix, where no information for differential priority weighting exists, the X²M and EM solutions are identical, i.e., u_i = y_i = 1/n. In contrast, the LSM has multiple solutions that are inappropriate.

See Exhibit 3

For most practical problems in which reasonable levels of inconsistency exist (e.g., when CR < .10), the X²M approach resembles LSM in being sensitive to magnitudes (distances) of adjustment of r_ij responses. But it cannot be relied upon to automatically yield closer adjustments than EM and LLSM. As an illustration, the X²M outcomes are contrasted below with EM and LSM solutions for a matrix discussed earlier in this paper:

Whereas LSM reversed the Row 1 priority over Row 2, both EM and X²M preserved the rank ordering. However, the X²M did not require such large consistency adjustments as EM and is, therefore, much more in line with LSM outcomes than is EM.

In the matrix below, EM scaling makes a gigantic consistency adjustment from r₁₃ = 5.00 to u₁₃ = .735/.058 ~ 12.65 in order to avoid reversing r₂₃ = 9 > 1 to u₂₃ < 1, i.e., to avoid a r₁₃ EPR:

The LSM solution makes a much smaller r₁₃ = 5.00 to w₁₃ * = .779/.124 ~ 6.27 consistency adjustment but reverses r₂₃ > 1 priority ranking to w₂₃ * = .098/.124 < 1, i.e., an EPR that the EM avoids is not avoided by the LSM.

The X²M approach yields a worse outcome in the above matrix. Like EM, the X²M preserves the r₂₃ > 1 priority ranking since y₂₃ * = .187/.046 > 1. It does so, however, by making a consistency adjustment from r₁₃ = 5.00 to y₁₃ * = .768/.046 ~ 16.70, thereby making the most extreme adjustment shown above.

In summary, the X²M avoids the degeneracy and nonuniqueness problems of the LSM. In addition it is more resistant than LSM to EPR and RPR-type rank reversals. Thus, it has much in its favor relative to the LSM except that it sometimes yields _________________________________________it is somewhat disappointing as a compromise between EM and LSM. However, it tends to yield good compromises in many instances, i.e., EPRs and RPRs are avoided at lower ESS values than under the EM.

10. The Logarithmic Least Squares Method

Up to now, I have postponed discussing the logarithmic least squares (LLSM) scaling approach. Saaty and Vargas [16, pp. 10-11] are critical of the LLSM because it can yield weak form RPRs that are always avoided in the EM scaling. Their reasoning, however, is somewhat circular in that they compare EM and LLSM scalings according to the criterion of taking [R] to higher (infinite) powers. Quite naturally EM will win out under this criterion since u_i eigenvector components are, in fact, the limits of row sums of the infinite power matrix. By definition, EM beats LLSM scalings under the higher (infinite) power criterion. The issue is whether this criterion (i.e., the weak RPR avoidance criterion) is as sacrosanct as Saaty and Vargas [15, 16] would like us to believe. In this paper I have tried to stress that the EM has advantages and limitations on other criteria. In other words, EM scaling is not sacrosanct unless one accepts the infinite power (weak RPR avoidance) criterion ipso-facto!

The LLSM approach can be shown to always avoid strong RPRs. Proof of this entails only trivial modification of the LSM proof derived previously in Exhibit 2. If r_ij and w_ij symbols are replaced by log r_ij and log x_ij counterparts, the remainder of the proof follows exactly as shown for LSM in Exhibit 2. An alternate proof for LLSM (but not LSM) is provided in Saaty and Vargas [15].

A comparison of LLSM scaling with EM, X²M, and LSM was provided earlier in Exhibit 3. The very high ESS outcome under LLSM illustrates the potential insensitivity of LLSM to Euclidean deviations between r_ij and x_ij * = x_i */x_j * surrogates implicit in LLSM scaling. It rarely performs worse than LSM in resisting EPR and RPR-type rank preference reversals. My experience with LLSM is that, at reasonable to moderate levels of inconsistency, LLSM and EM tend to have very close solutions that may differ substantively from LSM in magnitude.

Two out of three referees of my original paper alluded to a preference for LLSM over EM and LSM scalings:

Referee A: "I'm sensitive about this because I believe the most relevant metric in this application is based on the sum of squared differences of logarithms. Since I probably have an extreme (albeit correct, at least in my mind) point of view, I'll leave it to the author to decide whether this is a real issue or a red herring."

Referee C: "Incidentally, there is a school of people who think that logarithmic least squares method is best."

They do not elaborate in their comments to me, however, about why they have this LLSM preference. I do not share this preference, because my experience is that LLSM performs no better than EM if EPR or RPR criteria are employed and no better than LSM if magnitude (distance) criteria such as RLIMS bounds are employed. If LLSM is to be preferred over EM or LSM, it must be defended using other criteria.

One reason I do not like LLSM is that it is difficult to explain its implicit consistency adjustment basis to respondents who supplied the [R] matrix or users of the ultimate analysis. The LLSM has an underlying multiplicative error term e_ij where

Taking logarithms we obtain

The LLSM approach seeks x_i * solutions that

Minimize å_i å_j (log e_ij)² = å_i å_j (log r_ij - log (x_i/x_j))²

In terms of additive consistency adjustments, the LLSM approach yields the following in contrast to LSM, X²M, and EM adjustments:

The LLSM consistency adjustment may be rewritten as:

Whereas LSM seeks to minimize the sum of squared entire adjustments, LLSM seeks to minimize the sum of squared (log e_ij) component values. The reasons for doing so are much more difficult to explain to respondents.

By way of illustration, consider the alternative matrix scalings:

The LSM changes the priority of Row 1 over Row 2. In doing so, it also has an EPR by lowering r₁₂ = 4 > 1 to w₁ */w₂ * = .1845/.2212 < 1. There are also RPRs for r₁₅, r₁₆, and r₂₅ and their reciprocals r₂₁, r₅₁, r₆₁, and r₅₂. The EM, LSM, and X²M approaches resist those EPRs and have much closer scaled outcomes relative to LSM that allows EPRs in order to get w_i */w_j * surrogates as close as possible to r_ij responses in the Euclidean sense.

The LLSM is quite close to the EM solution. This is very typical except for very high levels of response inconsistency. As mentioned earlier, I find little of advantage in LLSM relative to EM and LSM. There is some intuitive appeal in having e_ji = 1/e_ij. This reciprocal property is lost, however, when consistency adjustments are translated to r_ij - x_ij additive form.

11. Summary and Conclusion

Professors Saaty and Vargas [15, p. 24] would like the EM method to be the "only method that should be used when the data are not entirely consistent." The purpose of my earlier paper and this rejoinder is to point out that this is subject to debate. There are other reciprocal matrix scaling methods that have key advantages over EM. In particular, LSM has more degrees of freedom in deriving consistency adjustment surrogates that are "closer to" r_ij responses, LSM avoids the extreme consistency adjustments often encountered using EM, and LSM consistency adjustments are more apt to yield w_ij * surrogates that lie more within the response bounds imposed upon r_ij judgments (relative to u_ij surrogates under EM that often fall way outside response bounds and are, therefore, difficult to justify to respondents ex post).

I find not much in favor of LLSM or X²M relative to EM and LSM. The primary advantage of EM relative to LSM is stronger rank preservation in certain types of inconsistencies where LSM gives rise to EPRs or RPRs when rows have moderate or weak dominance. This does not mean EM is better than LSM or vice versa. It simply implies that both EM and LSM solutions should be compared using the following policies where possible:

(1) Compare alternative scaling solutions, such as EM, LSM, LLSM and X²M. Chances are they will not be greatly different for small CR values.

(2) Whenever u_i > u_j and w_i * < w_j * or vice versa, obtain added bootstrap information as to whether rank preservation or consistency adjustment magnitudes are more important.

(3) Whenever r_ij - u_ij, r_ij - y_ij *, and r_ij - x_ij *consistency adjustments are very large relative to r_ij - w_ij * consistency adjustments, obtain added bootstrap information as to whether avoidance of large consistency adjustments is important.

One means of assessing the "importance" mentioned above is to return to the respondents that provided r_ij inconsistent responses and ask them about the relative importance of rank reversals versus adjustment magnitudes in using consistency adjusted surrogates for their original responses.

The X²M proposed in this paper was undertaken as a compromise solution between EM and LSM extremes. Analysts might discover that in certain matrices the X₂M consistency adjusted surrogates are the most palatable compromise between EM and LSM extremes. X²M strongly resists EPRs and RPR. The LLSM often tends to be close to EM without apparent advantage over EM in terms of EPRs, RPRs, and EUCLID criteria.

In conclusion, unless added bootstrap information beyond that contained in an initial response matrix [R] is provided, there is no general way of determining whether EM, LSM, LLSM, X²M or any other consistency adjustment method is best among alternative approaches to finding explicit or implicit surrogates for inconsistent r_ij responses in [R]. When the solutions substantively differ, there is no "best" scaling method given only the information in [R]. At CR < .10 moderate levels of response inconsistency, however, these solutions often do not differ substantively, especially under EPR and RPR criteria. They are more apt to differ under a Euclidean distance criterion except when inconsistencies in response are virtually negligible such as when CR < .01.