Friday, December 30, 2011

Correlation between reference managers and the WoS

Even though web citations have been a part of our lives for several years now, the correlation between "traditional" citations and web resources like Mendeley, CiteULike, blog networks, etc. hasn't been thoroughly studied yet, and any new research in the field is very interesting (to me, anyway).

The new paper was published at Scientometrics by Li, Thelwall (still one of my dissertation advisors) and Giustini. They focused on the correlation between user count - the number of users who save a particular paper - and WoS and Google Scholar citations.

The researchers extracted from WoS all the Nature and Science research articles that were published in 2007 and their references. They ended up with 793 Nature and 820 Science articles, or 1,613 articles overall (not including references, of course). Then, they searched CiteULike for those articles' titles and number of citations, as well as for their user count in Mendeley. They also collected the same data from Google Scholar. It's important to note that Mendeley had 32.9 million articles indexed while CiteULike had only 3.5 at the time of the study.

Google Scholar's mean and median number of citations were higher than in WoS (not surprising; If you want better citation numbers, always use GS). They found that despite Mendeley being "younger" than CiteULike (launched in 2004 and 2008 respectively), CiteULike had only about two-thirds of the sample articles saved, while Mendeley had about 92%.

Spearman correlations between citations in GS and WoS were high in this research (0.957 for Nature and 0.931 for Science). The correlations between Mendeley's user count and the citations in GS and WoS were also rather good (0.559 and o.592 for WoS and GS respectively for Nature, 0.540 and 0.603 for Science). CiteULike had far weaker correlations: 0.366 with WoS and 0.396 with GS for Nature, 0.304 with WoS and 0.381 with GS for Science.

Limitations

The authors remind us that correlation isn't causation, saying they can't conclude a casual relationship based on correlations between two data sources. Therefore, it can't be determined for sure whether there is a connection between a high user count and a high number of citations. Only Nature and Science were studied, so it can very well be that the results aren't true for other journals. Also, group-saved and single-user saved references were given the same weight. The number of saved references in Mendeley and CiteULike is much smaller than in the WoS counts and therefore the results might be less reliable.

The authors speculate that user count may represent a more accurate scientific impact of articles, and take note that one can measure the impact of all sorts of resources in online reference managers, unlike in the limited bibliographic indexes.

I think it could be reference managers don't always reflect readership: one could save a reference and forget about it all together later (so many articles, so little time...). On the other hand, citation counts might suffer from the same problem, as many scientists use a "rolling citation" from other articles citing an earlier article, without actually having read the article themselves.

Priem et al. also presented lately a study about web citations and WoS citations, based on data from the seven PLoS journals, but I think I'll wait for the journal article to cover it in the blog.



Li, X., Thelwall, M., & Giustini, D. (2011). Validating online reference managers for scholarly impact measurement Scientometrics DOI: 10.1007/s11192-011-0580-x

28 comments:

  1. I enjoyed this article because it is not including worthless information .The author clearly describes the information.

    ReplyDelete
  2. Mahi Chopra an Independent Escorts in Mumbai. I am a high-class Mumbai Independent Escorts. I am residing in Mumbai. And offering a genuine high-class Escort Services in Mumbai and all over India to my genuine clients.

    Best Escort services in Mumbai | Escorts services in Mumbai | Independent Call Girls in Mumbai | Russian Escorts in Mumbai | Collage Call Girls in Mumbai | Celebrity Escorts in Mumbai | Call Girls Mumbai Whatsapp No | Mumbai Escorts Whatsapp Phone Number | Mumbai Escorts Phone Number | High Class Mumbai Escorts | http://www.mahichopra.com, Mumbai Escorts

    ReplyDelete
  3. Superb post, we enjoyed each and everything as per written in your post. Thank you for this informative article because it’s really helpful, I really like site. @ Packers And Movers Delhi
    Packers And Movers Delhi to Jaipur
    Packers And Movers Delhi to Gurgaon
    Packers And Movers Delhi to Kolkata
    Packers And Movers Delhi to Chennai

    ReplyDelete
  4. Call us and book the Young Female Mumbai Call Girls all sectors.This is the reason these women have always gained vast experience in this field. Therefore, availing their service will really be advantageous for you in different ways.

    * Mumbai Escorts Service |
    * russian escorts in mumbai |
    * call girls in mumbai |
    * call girls in andheri |
    * call girls in navi mumbai |
    * call girls in lokhandwala |
    * call girls in lower parel |
    * call girls in malad |
    * call girls in chembur |

    ReplyDelete
  5. Of course, Escorts In Delhi all of our escorts are vetted to the highest standard possible. We ensure that every Diva listed on our books is capable, versatile, flexible and of course, stunningly gorgeous.

    paschim vihar escorts ##
    aerocity escorts ##
    cr park escorts ##
    rk puram escorts ##
    rajouri garden escorts ##
    pashchim vihar escorts ##
    punjabi bagh escorts ##
    dwarka escorts ##
    connaught place escorts ##

    ReplyDelete
  6. Delhi Escorts are truly gorgeous in their appearance and have some of the stunning characteristics inside them to enthrall their customers. Whatever you would want from our darlings, they are each time being ready to flaunt their ideas.

    $$ vasundhara call girls
    $$ kashmiri gate call girls
    $$ kamla nagar call girls
    $$ shahdara call girls
    $$ karol bagh call girls
    $$ vasant kunj call girls
    $$ lajpat nagar call girls
    $$ munirka call girls
    $$ Russian Call Girls Dwarka

    ReplyDelete

  7. We basically deal with the high profile Noida Escorts who can fulfill all the requirements of the clients. So, all our past clients never see any other escorts agency because we know better our clients and their requirements.
    Escorts In Vasundhara ##
    Escorts In Kaushambi ##
    Escorts In Vaishali ##
    Escorts In Indirapuram ##
    Escorts In Safdarjung Enclave ##
    Escorts In Kamla Nagar ##
    Escorts In Kirti Nagar ##
    Escorts In Krishna Nagar ##
    Escorts In Inderlok ##

    ReplyDelete
  8. Those Russian Escort In Delhi can be reached directly via their website. You can hire those ladies for adult enjoyments as well for other purposes. You can hire those dashing ladies to enjoy your weekend at any nearby place where you can enjoy for a long time.

    $$ Independent Escorts Delhi
    $$ Vip Call Girls Delhi
    $$ Escorts In Delhi
    $$ Independent Escorts Chanakyapuri
    $$ Independent Escorts Connaught Place
    $$ Independent Escorts Defence Colony
    $$ Independent Escorts Dhaula Kuan
    $$ Independent Escorts Dwarka
    $$ Independent Escorts East Of Kailash

    ReplyDelete
  9. Thus, don’t think so much about the quality of young teen call girls because our Mahipalpur Escorts are also a better performer in the bedroom through which you can find and make sure the hot things in your life.
    Mahipalpur Escorts

    ReplyDelete
  10. Mumbai Escorts is the best partners for all types of travelers. With various nightlife activities and several options to fulfill their needs, they make great choices for night parties and social events. 
    Russian Goregaon Escorts ###
    Call Girls Malad ###
    Panvel Call Girls ###
    Call Girls in Dadar ###
    Premium Call Gils Mumbai ###
    Juhu Escorts ###
    Andheri Escorts ###
    South Indian Call girls in Juhu ###

    ReplyDelete
  11. Number One Movers in Abu Dhabi
    https://www.wemovers.ae/

    ReplyDelete