What Will Happen to References?

by Matt Hudson, Publications Manager, Geological Society of America

Most scientists will tell you that there are more and more papers being published by more and more journals. As an example, The Geological Society of America (GSA) published 57% more pages of journal content in 2012 than it did in 2007. This ever-growing mass of literature has put increasing strain on library budgets and reviewers’ time, but another somewhat less obvious result is that a growing pool of literature produces a growing volume of cited references. If you spend even a small portion of your time looking at the references of earth science papers, you may have noticed that they seem to get longer every year. But are they really growing and, if so, what kind of impact is this having on researchers and publishers?

To find out, I examined the reference sections of Geology, GSA Bulletin, and GSA’s Special Papers in ten-year intervals from 1953 to present (1973 to present for Geology since it did not exist until then). I selected 20 random papers from each year and counted the number of papers cited and the number of characters for the entire reference section. For the Special Papers, I strayed into nearby years in order to get a full random sampling of content since some archive years did not contain enough edited volumes.

Figure 1. Comparison of the total number of characters appearing in the average reference sections of Geology, GSA Bulletin, and GSA’s Special Papers in ten-year intervals from 1953 to present.

Figure 1. Comparison of the total number of characters appearing in the average reference sections of Geology, GSA Bulletin, and GSA’s Special Papers in ten-year intervals from 1953 to present.

Are Reference Sections Growing?

Yes. There are several ways to look at this. The number of characters in the average GSA Bulletin paper’s reference section has grown more than 400% since 1953 (see Figure 1). Special Papers have also seen significant growth, but it has recently leveled off. I suspect that is in part because of the variable nature of book content. It is likely that if I had examined more than 20 papers per year this trend would not be as visible.

Figure 2. Comparison of the total number of papers being cited in the average reference sections of Geology, GSA Bulletin, and GSA’s Special Papers in ten-year intervals from 1953 to present.

Figure 2. Comparison of the total number of papers being cited in the average reference sections of Geology, GSA Bulletin, and GSA’s Special Papers in ten-year intervals from 1953 to present.

Geology has seen consistent growth, but not on the same scale as GSA Bulletin or Special Papers. Geology’s strict four-page limit per paper has likely been the limiting factor.

The average number of papers being cited per article has also grown over the years, although Geology and Special Papers have both shown recent drops (Figure 2). For Geology, I suspect that authors have simply run out of room. As mentioned above, the overall size of Geology’s average reference section has continued to increase, albeit slowly. Interestingly, all three journals have shown nearly identical trends in the size of individual references, with each reporting an 84% increase in the number of characters since 1953 (Figure 3). Since individual references are growing faster than the overall reference sections for Geology, the number of citations has actually dropped simply because there is not enough room to include them.

Figure 3. Comparison of the total number of characters appearing in the average individual reference for Geology, GSA Bulletin, and GSA’s Special Papers in ten-year intervals from 1953 to present.

Figure 3. Comparison of the total number of characters appearing in the average individual reference for Geology, GSA Bulletin, and GSA’s Special Papers in ten-year intervals from 1953 to present.

Why Are Reference Sections Growing?

As already discussed, one reason for the growth in references is the vast pool of available literature. Unlike in a lot of scientific fields, such as medicine, the citable value of an earth science paper is known to last a long time. Another reason may be that reference and citation tools like Endnote, Mendelay, and Zotero are making it easier for authors to track and organize their sources.

Both of these scenarios explain why more papers are being cited, but they don’t explain why the size of the average reference has steadily increased. Some of this can be explained by editorial changes. For example, GSA used to use a dash in reference lists when author names were repeated from one reference to another. Likewise, we used to abbreviate journal names and now we spell them out. Over the years we have also added new features to references, such as digital object identifier (doi) numbers and uniform resource locators.

While I did not specifically look at the number of authors appearing in each citation, my guess is that this may be another reason we are seeing steady growth in reference size. In a world of increasing specialization, authors frequently rely on other people to round out the research necessary for a complete paper. I recently heard of a Science paper that had more than 200 authors.

Publishers Respond

Nearly 40 years ago, Bennie Troxel wrote an editorial in Geology titled “What Price Reference Lists?” in which he questioned the value of reference lists and solicited ideas for how to handle what some considered a growing problem. In it, he noted that a recent GSA Bulletin issue contained about 10% references, or the equivalent of three papers. Troxel also noted that Geology was “a medium for experimentation” and that it planned to test a smaller font for references.

The editorial received a flood of comments, and while the vast majority of respondents were okay with the idea of smaller type for references, there was strong resistance to any further tampering of references. The comments highlighted the value of references for further reading lists, tracing the source of an author’s idea, and providing a glimpse of the author’s overall knowledge. As Marvin Lanphere summed up, “The thought of deleting reference lists is appalling.”

Today, Geology’s average reference section is about double the size it was when Troxel wrote that editorial. Even with its smaller font size for references, which was later adopted by GSA Bulletin, there can be little doubt that the growing references are reducing the amount of space available for an author to discuss her research.

For GSA Bulletin and Special Papers, which do not limit an article’s page length, the larger reference sections are still having an impact. In 1953, GSA Bulletin published about 76 pages of references. This year, I anticipate the journal will publish about 250 pages of references, or roughly the equivalent of 2.5 Geology issues. This means that even though a page of GSA Bulletin references today contains about 9500 characters compared to 5500 characters in 1953, the portion of the journal devoted to references has gone from ~4% in 1953 to ~12% today. Is this a problem?

Some publishers think so. At a recent meeting, a biology publisher revealed that they will not accept articles with more than ten references per paper. While this may limit the cost of producing, proofing, and printing large reference sections, is it the best decision for the science?

Other publishers have responded by shortening the amount of information in each reference. The idea here is that, now that we have (supposedly) definitive tools like doi numbers, do we still need to cite the full author list, article title, etc.? Science, for example, publishes full reference sections in its online version, but significantly abbreviated references in their print version. While this does save space, having two versions of a paper’s reference section also creates its own challenges.

Nature frequently eliminates all but the lead author’s name, does not include the article title, and abbreviates journal titles. As a result, the average reference for one of its recent papers was only 83 characters, which is well below the average length GSA was producing in the 1950s and about a third of the size of a current GSA reference. If GSA Bulletin took a similar approach to Nature, the journal would produce ~160 fewer pages per year, or roughly the equivalent of 10 articles.

Some will say that GSA should take this approach so that it can publish 10 more GSA Bulletin papers. Others might say that publishing 10 more papers would only feed this ever-increasing volume of literature and would actually exacerbate the problem. On one hand the idea of adopting a heavily abbreviated reference section is appealing, but on the other hand the future remains pretty uncertain. While it is tempting to think of things like doi numbers as permanent, we have made wrong turns before. It wasn’t until last year that we were finally able to digitize the GSA Bulletin content produced in microfiche only from 1979–1981. In the end it’s not just about what we need today, but also about what we may need in the future.

5 responses to “What Will Happen to References?

  1. Reblogged this on Orogeny and commented:
    I suggest we use name of author, year and DOI only in the reference section and use numbers for in text citation like Nature. Most of us use the reference to dive deeper into someone’s work. Getting electronic addresses via DOIs in the pdf is the way to go.

  2. “While [limiting references] may limit the cost of producing, proofing, and printing large reference sections, is it the best decision for the science?”

    Easily answered, as we already see it. An increasing number of citations to handy overview papers and a decreasing awareness of the origin of important observations, techniques, and insights. I have noticed many authors choosing to cite a very recent overview paper for a specific concept that was fully developed and presented decades prior in a more specialized paper. I will say as an AE I chastise authors for this, viewing it as unnecessary laziness. To find that paper given the citation, one would end up burrowing down through some number of paper to get to the original. And as with the game of Telephone, the actual observation/technique/insight gets corrupted and is often not what the last paper claimed it to be. Many times, that discrepancy is a fruitful place for new research. So by discouraging proper referencing of the literature, publishers could degrade the quality of the science being done.

    Ironically, publishers themselves might come to rue the day they limit referencing. When there are still citations to a 1920 paper in a journal, that means that there is still value in being able to access that 1920 journal and so libraries would still be willing to pay to access 100 years of a journal. Limit the references and I am willing to bet that most of those old citations will fall away, leaving little value for libraries to pay for the older stuff (when given the option). This can also have an impact on the journal’s reputation as the number of citations to an old, well-established journal will drop off.

    Incidentally, I think the growth of citations reflects a couple of things. One is the maturation of the field: there are precious few places where you can work where there is really no preexisting literature. Another is that plate tectonics reset the counter on citations: an Aegean stable of explanations of miogeosynclines, eugeosynclines and the whole battery of creatures in the bestiary of geosynclinal theory was flushed clean to remain uncited ever since; I rather suspect a more detailed look between 1950 and 1980 might show a plateau in the late 60s to late 70s between upward trends before and after.

  3. I second the remark about print versus online journals. The problem of space is nonexistant in the digital world. Cost of typesetting should be brought back by smart software (such as research paper authoring plugins for text editors combined with reftools as Mendeley or RefWorks or languages such as Latex). Processes could further be simplified by more sysyematically using author ID systems as ORCID and DOIs. In science we could further bring back the number of citation styles from over 5000 to say 10.

    So most of the problems are of administrative or technical/software nature. What is far more interesting is the question whether the article will remain the most effecient vehicle for scholalry communication. We already see new forms emerge, esp. in life science, with nanopublications, allowing for much faster and effective sharing of advancements and findings, while still giving full credit to authors. When we take this forward that might mean first the end of issues, then the end of journals and eventually the end of all those thousands of relevant papers that we never have time to read, while ensuring that each researcher has all the relevant insights and findings at his or her fingertips, integrated, structured, smart and friendly. The future is beautiful.

    BTW I already saw some papers with way over 1000 authors….

    Jeroen Bosman
    Geoscience librarian, Utrecht University

  4. Many of us think the days of printed journals are over, and everything will be online only (or at least predominately) in the not too distant future. Thus, any issue in regard to the size of reference lists will disappear.

    • Randy,

      Thank you for the feedback. You raise an interesting point about the print, but I think it may affect more than that. On the publisher side there are costs associated with references even without taking into account printing and postage. For example, copyediting and coding references takes considerable work, even with the software that we have available today such as eXtyles. Likewise, this could present a strain on researchers. Fifty years ago the average GSA Bulletin author was responsible for citing 30 papers, and now that is up to 90. What kind of chore will it be in another 50 years if the average author is citing 200 papers? Or will the software to track such things be so powerful that this won’t be much of a chore at all? What will the reading experience be like if 25% or 50% of a paper is references? Will that make a difference or no difference? Or will the entire reading experience be so different that none of these concerns will matter?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s