You're just a number: introduction to the h-index

Measuring a single scientist's output has always been problematic. Why? First, in order for the statistics to be reliable, the scientist has to produce a considerable publication output and get cited. That takes time. Second, measures like research productivity, number of publications and citations don't always correlates. Measuring the output of journals and universities has been far more reliable than measuring that of one person.

Suggested by physicist Jorge Hirsch, h-index (2005) offers an attractive way of quantifying one's scientific output as a single number. The index is defined as:

“A scientist has index h if h of his or her N_p papers have at least h citations each and the other (N_p − h) papers have ≤ h citations each” (Hirsch, 2005).

So, if a scientist published at least ten papers, which each were cited at least ten times, her h-index is ten. A zero h-index, on the other hand, says that the scientist perhaps published papers, but is yet to have an actual impact.

The h-index is attractive because it takes into account both the number of publications and the number of citations. It isn't phased by "one hit wonders", but favors a body of work that each of its components has at least a certain impact (citations).

Problems and disadvantages

Which database to use? Different databases cover different journals, conferences, etc. Web of Science, for example, has better coverage of STEM than of the humanities, which tend to publish books rather than papers. Using Google Scholar will likely inflate the h-index.

Which field are you in? Larger fields mean a larger potential for citations, resulting in a higher h-index.

You aren't a number! (Or at least, not just *one* number). Reducing scientists to a single number ignores other factors, such as their teaching skills and ability to collaborate. Can an entire career really be described as a single number?

Source: PhD Comics

The age factor: The older the scientist gets, the longer she had to publish and get cited. Younger scientists are at disadvantage with the h-index.

Relevance: Since the h-index doesn't decrease, it can't tell whether a scientist is still active and/or where her work is still relevant for others in her field.

Since the h-index is a single number, scientists with the same h-index can have very different numbers of papers and citations. In the following table, scientist A and scientists B have the same h-index, but scientist A has far more citations in the overall raw calculation.

Source: Bornmann & Daniel, 2008

Because of the h-index many problems, offering new corrections to it, or coming up with other indices altogether is the official new sport for bibliometricians. The new indices are supposed to offer a better way to make decisions about promotions and grants, but despite all the efforts, it seems that the way to the promised tenure will continue to be paved with peer evaluation.

Bornmann, L., & Daniel, H. (2007). What do we know about the h-index? Journal of the American Society for Information Science and Technology, 58 (9), 1381-1385 DOI:10.1002/asi.20609

Bornmann, L., & Daniel, H. (2008). The state of h index research. Is the h index the ideal way to measure research performance? EMBO reports, 10 (1), 2-6 DOI: 10.1038/embor.2008.233

Hirsch, J. (2005). An index to quantify an individual's scientific research output Proceedings of the National Academy of Sciences, 102 (46), 16569-16572 DOI: 10.1073/pnas.0507655102

The Matthew Effect Strikes Again

The new Bornmann, de Moya Anegón and Leydesdorff paper, published in PLOS ONE, shows that highly cited papers tend to reference other highly cited papers more often. That is true especially for the life science and health science disciplines. Ms. Corbyn from Nature News saved me the need to summarize the paper by writing an excellent article about it.

Based on their findings, Bornmann et al. suggested to concentrate funding on already highly-cited researchers and research groups ("A concentration of resources on these elite structures seems to be practical especially for the life sciences and health sciences"). This is already happening, to some extent, but if the authors' offer is to be accepted, I suspect highly-cited authors will forever remain at the top, simply because they will get most of the funding.

Now, citations don't exist in a social vacuum; despite the tendency to see them as "objective" representations of papers which have influenced other scholarly works, they are affected by factors like the paper's publishing journal, personal connections, coverage of the paper in the mass media, and more. Bornmann et al.'s recommendation might cause funding agencies to end up giving their money to the most popular, connected researchers, who are already well-established in their discipline.

Bornmann, L., de Moya Anegón, F., & Leydesdorff, L. (2010). Do Scientific Advancements Lean on the Shoulders of Giants? A Bibliometric Investigation of the Ortega Hypothesis PLOS ONE, 5 (10) : 10.1371/journal.pone.0013327