2019

STATISTICS ROUNDTABLE

Player Rankings

Using coefficient of variation to measure MLB players’ worth

by Julia E. Seaman and I. Elaine Allen

The coefficient of variation (CV), sometimes called the relative standard deviation, is often used to assess the quality of an assay,1 the diversity in organizations2 or as a benchmark for ranking.3 It is measured as a fraction or percentage of how much variation exists in a variable in relation to the mean, so it represents a highly useful—and easy to interpret—concept in data analysis.

Alone, the mean or maximum value may not be meaningful because it does not indicate this parameter’s variability. The standard deviation also may not be particularly useful without examining its relative value with respect to the mean. Within a context that relates the standard deviation to the parameter it is measuring, however, it becomes more informative. The CV is a measure that jointly provides context for the mean and the standard deviation.

For example, knowing that the standard deviation is 1.25 doesn’t mean much. Understanding that past data consistently yielded a standard deviation of 2 gives context that the variability of this new sample is less than expected. Knowing the standard deviation has been 0.5 or lower in past data would lead you to believe, however, that the new value, 1.25, should be considered high. Without this perspective, the figure for standard deviation has little meaning.

The CV provides a reference value that combines the mean and standard deviation. By looking at the ratio of a standard deviation (s) to a mean (µ), which is then usually multiplied by 100, you have a measure of the percentage of variability surrounding the mean.

CV = (σ/µ) * 100

If the CV is large, the data have a great deal of variability relative to the mean.

σ = 50 and µ = 100, then the CV = (50/100) * 100, or 50%.

If the number is small, this reflects a small amount of variability relative to the mean.

σ = 5 and µ = 100, then the CV = (5/100) * 100, or 5%.

Widely used in the development of biological and chemical assays,4 the CV must often be below a certain value (5 to 15%) for the assay results to be considered valid. Quality specifications for assays are usually given in terms of the CV. When examining diversity, the CV is often used as an evaluation tool when comparing organizations or ethnicity within geographic areas.5 In hospital evaluations, for example, the CV is often used as a quality indicator when looking at variability of care or hospital infections over subgroups of patients.6

MLB player rankings

The CV can also be used to examine consistency of performance versus overall or peak performance. When determining the contract value of an athlete, both metrics should be considered. Using an example from Major League Baseball (MLB) to illustrate using the CV as a ranking metric along with performance, compensation received will be ranked along with the CV and overall performance to determine whether compensation is solely related to peak performance or also to consistency as measured by the CV.

For background, there is a phenomenon known as the Sports Illustrated cover jinx. Some athletes who have been featured on the cover of Sports Illustrated because of their unique peak performances invariably show a decline from that performance in the future.

Conversely, it is also a commonly held belief that some MLB players will perform best when they are in a contract year—that is, the last year of an existing contract. During a contract year, players often produce more on the field, demonstrating their apparent value. The player hopes this surge in performance will result in a new, high-dollar contract.

Simply on the basis of contract-year performance, and standardizing by the number of at bats, Figure 1 shows there is a statistically significant improvement during a contract year. But are these peak performances also consistent?

Figure 1

To examine whether performance and consistency are used to determine a new contract salary, the OPS (on-base percentage + slugging percentage) has been calculated for the top 25 players with new contracts negotiated between 2000-2006. The overall CV is calculated for the five years leading to a new contract, and the salary value of the new contract is also ranked.

Figure 2 provides these rankings by three panels: highest OPS, lowest CV and highest salary. From these rankings, it is clear there is little relationship between the CV and salary, but there is a significant relationship between OPS and salary. When the rankings are correlated using either panel of ranks, the only significant correlation is between OPS and salary (0.463, 0.466 and 0.637, respectively), showing that as OPS increases, salary also increases.

Figure 2

The CV is not significantly related to the other two variables and is either negatively correlated with salary or is almost zero (-0.173, -0.245 and 0.030, respectively). Only seven players, including five of the top 10 in salary, appear in the top 25 for OPS and CV. 

What can be learned from this example? First, the ranking of mean values as a measure of distribution of future reward can be misleading; however, with respect to salaries in MLB, it is the norm. Salary is strongly related to performance, with little attention to consistency.

Second, by incorporating consistency of performance along with peak performance, you can identify undervalued baseball players that have high OPS and low CV but are not in the top 25 for salary. For example: 

  • Travis Hafner—OPS rank 2, CV rank 22.
  • Lance Berkman—OPS rank 10, CV rank 5.
  • Victor Martinez—OPS rank 23, CV rank 1.

Similarly, there’s a group of overvalued players with high OPS but variable performance (not in top 25 for CV). The group includes:

  • Barry Bonds—OPS rank 1, salary rank 4.
  • Todd Helton—OPS rank 4, salary rank 8.
  • Chipper Jones—OPS rank 6, salary rank 6.

Finally, the analysis also identifies players that have high OPS and highly ranked consistency of performance. These include Manny Ramirez, Alex Rodriguez, Sammy Sosa and Gary Sheffield. 

It is interesting to note that all four of these players have been linked to performance-enhancing drugs, although it is unclear how this would impact performance, consistency or both.


References

  1. U.S. Food and Drug Administration, "Guidance for Industry and FDA Staff—Statistical Guidance on Reporting Results From Studies Evaluating Diagnostic Tests," www.fda.gov/downloads/medicaldevices/deviceregulationandguideline/
    guidancedocuments/ucm071287.pdf
    .
  2. I. Elaine Allen and Julia E. Seaman, "An Examination of Performance During the Contract Year in Major League Baseball," presentation at American Statistical Association annual meeting, Denver, August 2008.
  3. I. Elaine Allen, Kirill Kustov and George Recck, "Building a Better Fantasy Baseball Team," Quality Progress, April 2007, pp. 24-29.
  4. Ibid.
  5. Ibid.
  6. C. Glen Mayhall, ed., Hospital Epidemiology and Infection Control, third edition, Lippincott Williams & Wilkins, 2004.

Bibliography

  • Bedeian, Arthur G., and Kevin W. Mossholder, "On the Use of the Coefficient of Variation as a Measure of Diversity," Organizational Research Methods, Vol. 3, No. 3, 2000, pp. 285-297.

Julia E. Seaman is a researcher at Genentech in South San Francisco, CA. She earned a bachelor’s degree in chemistry and mathematics from Pomona College in Claremont, CA.

I. Elaine Allen is director of the Babson Survey Research Group and professor of statistics and entrepreneurship at Babson College in Wellesley, MA. She earned a doctorate in statistics from Cornell University in Ithaca, NY. Allen is a member of ASQ.


Average Rating

Rating

Out of 0 Ratings
Rate this article

Add Comments

View comments
Comments FAQ


Featured advertisers