The post-journal era

Most of the scholarly publication today goes more or less like this: a scientist writes a manuscript about research funded by her university and/or the grant fairy (usually a government agency) then submits it to a commercial peer-review journal. An editor (either working for free or for "honorarium") reads her manuscript and sends it to appropriate peer reviewers (payment? what payment?). Then, if her manuscript is accepted, her institute's library gets the privilege of buying access to the published manuscript. This state of things is very profitable for the commercial publishers' stock owners, but less so for scientists, libraries and the general public, who rarely get to read research they paid for. While many people agree this system is, shall we say, less than optimal, attempts to remedy the situation have been less than successful, and the commercial publishers might be targeting our research budgets next.

The latest attempt to renovate the system is by Priem & Hemminger (2012). At the beginning of their paper, they suggest that previous attempts in reinventing scholarly publishing have failed due to two reasons:

1.Change to peer review are just patches on a fundamentally broken scholarly journal system
2.Proposals offer no smooth transition from the present system.

Today, the journal fills four main functions: It archives scholarly material and time-stamp the researchers' contributions, if disseminates scholarly products and it certifies contributions (if it's published in a high-impact journal it must be of value). Priem and Hemminger want to make each of these functions independent from the others.

Their first suggestion is to "refactor" the system. This means locating "parts which are confusing, inefficient or redundant" and improving them without hurting the rest of the system. Their second suggestion is the "decoupled journal" (DcJ) (more about this later).

Overlay journals
These journals were suggested by Ginsparg (1997) and only provide the "stamp of approval" to an already published-archived-registered material. Despite the promise the overlay model represents, it hasn't been successful so far, and almost every journal which tried it went back to the traditional coupled model.

The PLoS One model

PLoS One is an open access journal which publishes work not according to what the editors and reviewers consider significant, but consider only the paper's methodological quality. They decoupled the significant approval from the methodological approval. PLoS One also decoupled copy-editing: they warn in advance that they don't copy-edit in details, and instead provide a list of services which do just that. This model has proven to be profitable: PLoS One published more than 5,000 papers in 2010 at 1350$ each (and the other PLoS journals charge even more). The flaws here, beyond the price, are the exclusivity: authors publish only in one journal, and the danger of a future with only a few mega-journals.

Post-publication review services
There are  a few existing post-publication peer review services, the best-known of them are Faculty of 1000 (F1000) and Mathematical Reviews. F1000 "...identifies and evaluates the most important articles in biology and medical research publication." F1000 is supposed to function as additional help for researchers in managing their reading. It has actually been shown to identify quality papers which were overlooked by leading journals (Allen et al., 2009).

Mathematical Reviews is and abstracting service, but as Priem & Hemminger say, it is "occasionally called into service as a post-publication peer review venue when the traditional journal fail in their role as certifiers. In this case, abstracters may abandon objectivity and attack papers and their reviewers directly."

These services have one major problem: they aren't brand names, and can't replace the certification of well-established journals, no matter how much their peer review is sound.

The Deconstructed Journal
Smith (1999) had three insights about the Deconstructed Journal (DJ):
1. The means (journal) and the functions are not the same.
2. Any system that will be implanted instead of the journal has to be at least as good.
3. Several cooperating agencies could successfully replace the central publisher.

Priem and Hemminger cite van de Sompei et al. (2004) and Smith (2003) as those who pointed out the advantages of a deconstructed system:

"...encourages innovation, adapts well to changing scholarly practices, and democratizes the largely monopolized scholarly communication market" 

However, van de Sompeis' and Smith' proposals are a bit outdated, because they hadn't taken into account the social media.

The functions of the decoupled journal
The Decoupled journal (DcJ, rather than DJ) is the updated version of the DJ. This is a universal, or meta, journal, where everything scholars produce and share is stored long-term, added to other projects, linked to, commented about...etc. etc.With the DcJ, publication is the first step in the process of revisions, reviewes, etc. Scholarly items will need persistent IDs, storage, and mirror backup in order to survive long-term. This can be done with persistent identifiers such as the DOI and institutional or subject-area repositories (ArXiv, Pubmed).

After the publication of a draft, it's time for preparation. Preparation is defined by the authors as "Changing the format of a work to make it more suitable for a given (human or electronic) audience". Today, many companies sell authors services (like copy-editing), but preparation is still mostly left to the journal. The DcJ will allow authors the freedom to choose the preparation they prefer (say, PDF or HTML format). PLoS One, as mentioned before, already leaves copy-editing to authors, perhaps showing the beginning of a trend.

After the preparation comes the assessment. Defined as "Attaching an assessment of quality to a scholarly object". Today's method of assessment, peer review, is usually anonymous, unpublished to the general public, and done by invited reviewers. The reviewers give their opinion in free text first, then a final assessment whether the material should be published.

In the Priem & Hemminger model, reviewers don't decide whether the material is publishable or not  (it's already published!) but certificate it. In the future, Nature could become "Nature stamping agency" and give papers its "seal of approval". It will even be able to do so by giving grades, rather than just accept or reject the paper. There will be agencies that will only review the soundness of the work (like PLoS One does today), agencies who will certify only certain parts, open peer reviews and blind peer reviews. Other forms of assessments - blog posts, number of downloads, and even tweets - will be stored as well. The authors see the DcJ as a way to allow peer-review to evolve freely, without its tight coupling with the other functions of the journal.

With libraries' budgets tighter than ever (even Harvard decided that commercial journals are just too expensive ) I expect more and more authors will choose the DcJ route. However, it could be that a certification bottle-neck will be created, with the prestigious journals of today becoming the prestigious stamping agencies of tomorrow. The number of expert peer-reviewers in each field could become a limitation as well. Will our grandchildren complain about the amount of money they have to pay for a Science certification? Only time will tell.

Allen L, Jones C, Dolby K, Lynn D, & Walport M (2009). Looking for landmarks: the role of expert review and bibliometric analysis in evaluating scientific publication outputs. PloS one, 4 (6) PMID: 19536339

Ginsparg, P. (1997). Winners and Losers in the Global Research Village The Serials Librarian, 30 (3-4), 83-95 DOI: 10.1300/J123v30n03_13

Priem, J., & Hemminger, B. (2012). Decoupling the scholarly journal Frontiers in Computational Neuroscience, 6 DOI: 10.3389/fncom.2012.00019


Smith, J. (1999). The deconstructed journal – a new model for academic publishing Learned Publishing, 12 (2), 79-91 DOI: 10.1087/09531519950145896

Smith, J. W. T. (2003). The deconstructed journal revisited: a review of developments ICCC/IFIP Conference on Electronic Publishing-ElPub03: From information to knowledge. (Minho, Portugal).

Van de Sompel, H., Payette, S., Erickson, J., Lagoze, C., & Warner, S. (2004). Rethinking Scholarly Communication D-Lib Magazine, 10 (9) DOI: 10.1045/september2004-vandesompel

Introducing new vaccines into poor African nations

The GAVI alliance (used to be called the Global Alliance for Vaccines and Immunizations) was founded in 2000 in order to help vaccinate children in poor nations. GAVI funds vaccines in any nation with a GNI per capita of less than $1,000. Glatman-Freedman et al. (published November 2010) investigated the factors involved in successful introduction of the Hib (Haemophilus influenza) and HepB (Hepatitis B) vaccines into poor nations.

The Hib and HepB vaccines are expensive. The basic battery of vaccines (Polio, Diphtheria, Pertussis, Tetanus, Measles and BCG) costs about $1 per child, but the Hib and HepB vaccines raise the cost to $7-13. Taking the new vaccines cost into account, it is very important to determine the best way to introduce them into poor nations.

The authors included in the study GAVI-eligible nations from the WHO African region (AFRO) with population bigger than half a million (35 countries overall).

The countries were included in one of three groups:

I. Countries where both Hib and HepB were introduced.

II. Countries where only HepB was introduced.

III. Countries where neither vaccine was introduced.

(Table and explanation from the original paper)

The authors looked at country-level governance indicators (political stability, government effectiveness, regulatory quality, rule of law, control of corruption and voice and accountability) and (as expected) found that mean scores for all indicators were highest at group I and lowest at group III.

Next, the authors tried to create a combined governance scoring, which consisted of the average of all the governance indicators of each country for each year. There was a significant difference.

Symbols represent means and error bars represent standard error of the mean. Grey plot background highlights the Pre-GAVI years, white plot background highlights the GAVI funding years. * p value smaller than 0.05, ** p value smaller than 0.01 (figure and explanation from the original paper).

Overall, the study indicated that the best way to predict poor African nations ability to introduce new vaccines is to determine their country-level governance. Good governance and political stability help nations attract both foreign aid and investments. Other than funding, the introduction of new vaccines requires trained personnel, cold chain capacity, the ability to reach remote locations, and safe disposal of needles and syringes. Hopefully, this study will be able to help GAVI to introduce new vaccines into poor nations more efficiently.

Glatman-Freedman, A., Cohen, M., Nichols, K., Porges, R., Saludes, I., Steffens, K., Rodwin, V., & Britt, D. (2010). Factors Affecting the Introduction of New Vaccines to Poor Nations: A Comparative Study of the Haemophilus influenzae Type B and Hepatitis B Vaccines PLoS ONE, 5 (11) DOI: 10.1371/journal.pone.0013802

How the NEJM became an advertising platform for the pharmaceutical industry

These days it's common practice for authors, peer-reviewers and even editors of medical journals to declare conflicts of interest, if those exist. However, medical journals normally don't issue the same declarations. Journals publish regularly industry-supported papers reporting large clinical trials. Reprints of those trials are regularly bought by pharmaceutics companies and distributed to clinicians. The result is an increase of the journals' income as well as an increase in their prestige, since papers read by a large number of clinicians are likely to increase those papers' citation rates and the journals' impact factor.

In order to study conflicts of interest in journals, Lundh et al. (October 2010) chose six high-impact medical journals: Annals of Internal Medicine (Annals), Archives of Internal Medicine (Archives), BMJ, JAMA, The Lancet and NEJM and studied the proportions of industry-supported randomized clinical trials (RCTs). The authors focused on citations from 1996-1997 for 1995 papers and 2007 citations for 2005-2006 RCTs. They categorized funding as industry support, mixed support, nonindustry support and no statement about support.

Then came the tricky part: getting financial data from the journals about their income from advertisements, reprints and industry-supported supplements as percentage of the journals' total income as well as the total number of reprints sold. BMJ and Lancet (British) provided the data, but the other four journals (American) refused to do so. Given that lack of cooperation, the authors had to become creative. The journals' owners are the American College of Physicians (ACP) for Annals, the American Medical Association (AMA) for JAMA and Archives, and the Massachusetts Medical Society (MMS) for NEJM. The authors obtained their publicly available tax forms, that included data on the total income from all types of publishing. The societies, however, publish each more than one journal, so the authors couldn't determine incomes for individual journals. They had to calculate the relative income from industry sources, to which they received confirmation from ACP, but not from AMA and MMS.

In 2005-2006, 32% of the RCTs published in NEJM had industry support. However, for BMJ, only 7% of the RCTs were industry-supported. Declines in proportion of industry-supported trials were statistically significant for Annals and Archives.

Citations and industry support

For 1996-1997 trials, there was a significant correlation between citations and industry support for Lancet and NEJM. The correlation was statistically significant for all journals in 2005-2006. The authors write that "Industry-supported trials published in Annals, Archives and Lancet in 2005-2006 were cited more than twice as often as nonindustry trials and one and a half times more in BMJ, JAMA and NEJM".

Impact Factor

The authors calculated the IF of each one of the journals without the industry-supported trials. The NEJM had the largest decrease in IF, followed by the Lancet. The BMJ's IF barely changed.

Implications for mass media reporting

NEJM is the most prestigious medical journal in the world. Moriarty et al. (2010) found that it was the most cited source in news stories about cancer research (see my last post). If a third of the clinical trials published in NEJM are published by the industry, which means they are more likely to have positive results for the funding company (Lexchin et al., 2003), and taking into account that the NEJM is a very popular source of health news, that means the industry doesn't just gain influence with clinicians by publishing in NEJM, but with the general public as well.

See my follow-up post: How JAMA managed to avoid becoming an advertising platform for the pharmaceutical companies

Lexchin, J. (2003). Pharmaceutical industry sponsorship and research outcome and quality: systematic review BMJ, 326 (7400), 1167-1170 DOI: 10.1136/bmj.326.7400.1167

Moriarty CM, Jensen JD, & Stryker JE (2010). Frequently cited sources in cancer news coverage: a content analysis examining the relationship between cancer news content and source citation. Cancer causes & control : CCC, 21 (1), 41-9 PMID: 19784789

Lundh, A., Barbateskovic, M., Hróbjartsson, A., & Gøtzsche, P. (2010). Conflicts of Interest at Medical Journals: The Influence of Industry-Supported Randomised Trials on Journal Impact Factors and Revenue – Cohort Study PLoS Medicine, 7 (10) DOI: 10.1371/journal.pmed.1000354

The Matthew Effect Strikes Again

The new Bornmann, de Moya Anegón and Leydesdorff paper, published in PLOS ONE, shows that highly cited papers tend to reference other highly cited papers more often. That is true especially for the life science and health science disciplines. Ms. Corbyn from Nature News saved me the need to summarize the paper by writing an excellent article about it.

Based on their findings, Bornmann et al. suggested to concentrate funding on already highly-cited researchers and research groups ("A concentration of resources on these elite structures seems to be practical especially for the life sciences and health sciences"). This is already happening, to some extent, but if the authors' offer is to be accepted, I suspect highly-cited authors will forever remain at the top, simply because they will get most of the funding.

Now, citations don't exist in a social vacuum; despite the tendency to see them as "objective" representations of papers which have influenced other scholarly works, they are affected by factors like the paper's publishing journal, personal connections, coverage of the paper in the mass media, and more. Bornmann et al.'s recommendation might cause funding agencies to end up giving their money to the most popular, connected researchers, who are already well-established in their discipline.

Bornmann, L., de Moya Anegón, F., & Leydesdorff, L. (2010). Do Scientific Advancements Lean on the Shoulders of Giants? A Bibliometric Investigation of the Ortega Hypothesis PLOS ONE, 5 (10) : 10.1371/journal.pone.0013327