Altmetrics

File:Altmetrics.svg

The original logotype from the Altmetrics Manifesto.

In scholarly and scientific publishing, altmetrics are non-traditional metrics proposed as an alternative to more traditional citation impact metrics, such as impact factor and h-index. The term altmetrics was proposed in 2010, as a generalization of article level metrics, and has its roots in the #altmetrics hashtag. Although altmetrics are often thought of as metrics about articles, they can be applied to people, journals, books, data sets, presentations, videos, source code repositories, web pages, etc. They are related to Webometrics, which had similar goals but evolved before the social web. Altmetrics did not originally cover citation counts. It also covers other aspects of the impact of a work, such as how many data and knowledge bases refer to it, article views, downloads, or mentions in social media and news media.

Adoption

Projects such as ImpactStory, and various companies, including Altmetric.com, and Plum Analytics are calculating altmetrics. Several publishers have started providing such information to readers, including BioMed Central, Public Library of Science, Frontiers, Nature Publishing Group, and Elsevier. Starting in March 2009, the Public Library of Science also introduced article-level metrics for all articles. Funders have started showing interest in alternative metrics, including the UK Medical Research Council. Altmetrics have been used in applications for promotion review by researchers. Furthermore, several universities, including the University of Pittsburgh are experimenting with altmetrics at an institute level.

However, it is also observed that an article needs little attention to jump to the upper quartile of ranked papers, suggesting that not enough sources of altmetrics are currently available to give a balanced picture of impact for the majority of papers.

Important in determining the relative impact of a paper, a service that calculates altmetrics statistics needs a considerably sized knowledge base. The following table shows the number of papers covered by services:

Website	Number of papers
Altmetric.com	~ 5 Million
ImpactStory	~ 1 Million

Categories

Altmetrics are a very broad group of metrics, capturing various parts of impact a paper or work can have. A classification of altmetrics was proposed by ImpactStory in September 2012, and a very similar classification is used by the Public Library of Science:

• Viewed - HTML views and PDF downloads
• Discussed - journal comments, science blogs, Wikipedia, Twitter, Facebook and other social media
• Saved - Mendeley, CiteULike and other social bookmarks
• Cited - citations in the scholarly literature, tracked by Web of Science, Scopus, CrossRef and others
• Recommended - for example used by F1000Prime

Viewed

One of the first alternative metrics to be used was the number of views of a paper. Traditionally, an author would wish to publish in a journal with a high subscription rate, so many people would have access to the research. With the introduction of web technologies it became possible to actually count how often a single paper was looked at. Typically, publishers count the number of HTML views and PDF views. As early as 2004, the BMJ published the number of views for its articles, which was found to be somewhat correlated to citations.

Discussed

The discussion of a paper can be seen as a metric that captures the potential impact of a paper. Typical sources of data to calculate this metric include Facebook, Google+, Twitter, Science Blogs, and Wikipedia pages. The correlation between the mentions and likes and citation by primary scientific literature has been studied, and a slight correlation at best was found, e.g. for articles in PubMed. In 2008 the Journal of Medical Internet Research began publishing views and tweets. These "tweetations" proved to be a good indicator of highly cited articles, leading the author to propose a "Twimpact factor", which is the number of Tweets it receives in the first seven days of publication, as well as a Twindex, which is the rank percentile of an article's Twimpact factor.

Besides Twitter and other streams, blogging has shown to be a powerful platform to discuss literature. Various platforms exist that keep track of which papers are being blogged about. Almetric.com is one that uses this information for calculating metrics, while other tools just report where discussion is happening, such as ResearchBlogging and Chemical blogspace. Moreover, platforms may even provide a formal way of ranking papers or recommending papers otherwise, such as Faculty of 1000 does.

Saved

Even more informative is the number of people that bookmark a paper. The idea behind this metric is that someone would not bookmark a paper of little influence to their own work. Providers of such information include science specific social bookmarking services such as CiteULike and Mendeley.

Cited

Besides the traditional metrics based on citations in scientific literature, for example as obtained from Google Scholar, CrossRef, PubMed Central, and Scopus, altmetrics also adopts citations in secondary and other knowledge sources. For example, ImpactStory counts the number of times a paper has been referenced by Wikipedia.

Interpretation

While the concept of altmetrics is questioned, the interpretation of altmetrics in particular is discussed. Proponents of altmetrics make clear that many of the metrics show influence or engagement, rather than impact on the progress of science. It should be noted that even citation-based metrics do not indicate if a high score implies a positive impact on science; that is, papers are also cited in papers that disagree with the cited paper, an issue for example addressed by the Citation Typing Ontology project.

Controversy

The usefulness of metrics for estimating impact is controversial, but the community shows a clear need: funders demand measurables on the impact of their spending. Limitations that affect the usefulness due to heterogeneity, data quality and particular dependencies have been studies. Like other metrics, altmetrics are prone to self-citation, gaming, and other mechanisms to boost one's apparent impact. Additionally, it has been argued that the currently adopted metrics are suggestive of positive impact, while negative metrics are equally important.

However, it should be kept in mind that the metrics are only one of the outcomes of tracking how research is used. Even more informative than knowing how often a paper is cited, is which papers are citing it. That information allows researchers to see how their work is impacting the field (or not). Providers of metrics also typically provide access to the information from which the metrics were calculated. For example, Web of Science shows which are the citing papers, ImpactStory shows which Wikipedia pages are referencing the paper, and CitedIn shows which databases extracted data from the paper.

Altmetrics can be gamed: for example, likes and mentions can be bought. Altmetrics can be more difficult to standardize than citations. One example is the number of tweets linking to a paper where the number can vary widely depending on how the tweets are collected.

Another source of objections against altmetrics, or any metrics, is how universities are using metrics to rank their employees, and the aim should limited to measure engagement.

The score on each field does not directly tell you anything about the quality or impact of the paper. For example, a much discussed paper may merely be very controversial: papers discussed on Retraction Watch will typically get high altmetrics score, despite being retracted from the literature.

Altmetrics for more recent articles may be higher because of the increasing uptake of the social web and because articles may be mentioned mainly when they are published. As a result, it is not fair to compare the altmetric scores of articles unless they have been published in the same year and, especially for fast increasing social web sites, at similar times in the same year.

Ongoing research

The specific use cases and characteristics is an active research field in bibliometrics, providing the much needed data to measure the impact of altmetrics itself. Public Library of Science has an Altmetrics Collection and both the Information Standards Quarterly and the "Aslib Journal of Information Management" recently published special issues on altmetrics. A series of articles that extensively reviews altmetrics was published in late 2015 .

See also

• Academic journal
• Open Researcher and Contributor ID — nonproprietary unique identifiers
• San Francisco Declaration on Research Assessment
• Webometrics

External links

• Altmetrics Manifesto
• NISO Altmetrics Standards Project White Paper
• Special issue of Research Trends: Issue 37 – June 2014