What Will Happen to References?

by Matt Hudson, Publications Manager, Geological Society of America

Most scientists will tell you that there are more and more papers being published by more and more journals. As an example, The Geological Society of America (GSA) published 57% more pages of journal content in 2012 than it did in 2007. This ever-growing mass of literature has put increasing strain on library budgets and reviewers’ time, but another somewhat less obvious result is that a growing pool of literature produces a growing volume of cited references. If you spend even a small portion of your time looking at the references of earth science papers, you may have noticed that they seem to get longer every year. But are they really growing and, if so, what kind of impact is this having on researchers and publishers?

To find out, I examined the reference sections of Geology, GSA Bulletin, and GSA’s Special Papers in ten-year intervals from 1953 to present (1973 to present for Geology since it did not exist until then). I selected 20 random papers from each year and counted the number of papers cited and the number of characters for the entire reference section. For the Special Papers, I strayed into nearby years in order to get a full random sampling of content since some archive years did not contain enough edited volumes.

Are Reference Sections Growing?

Yes. There are several ways to look at this. The number of characters in the average GSA Bulletin paper’s reference section has grown more than 400% since 1953 (see Figure 1). Special Papers have also seen significant growth, but it has recently leveled off. I suspect that is in part because of the variable nature of book content. It is likely that if I had examined more than 20 papers per year this trend would not be as visible.

Geology has seen consistent growth, but not on the same scale as GSA Bulletin or Special Papers. Geology’s strict four-page limit per paper has likely been the limiting factor.

The average number of papers being cited per article has also grown over the years, although Geology and Special Papers have both shown recent drops (Figure 2). For Geology, I suspect that authors have simply run out of room. As mentioned above, the overall size of Geology’s average reference section has continued to increase, albeit slowly. Interestingly, all three journals have shown nearly identical trends in the size of individual references, with each reporting an 84% increase in the number of characters since 1953 (Figure 3). Since individual references are growing faster than the overall reference sections for Geology, the number of citations has actually dropped simply because there is not enough room to include them.

Why Are Reference Sections Growing?

As already discussed, one reason for the growth in references is the vast pool of available literature. Unlike in a lot of scientific fields, such as medicine, the citable value of an earth science paper is known to last a long time. Another reason may be that reference and citation tools like Endnote, Mendelay, and Zotero are making it easier for authors to track and organize their sources.

Both of these scenarios explain why more papers are being cited, but they don’t explain why the size of the average reference has steadily increased. Some of this can be explained by editorial changes. For example, GSA used to use a dash in reference lists when author names were repeated from one reference to another. Likewise, we used to abbreviate journal names and now we spell them out. Over the years we have also added new features to references, such as digital object identifier (doi) numbers and uniform resource locators.

While I did not specifically look at the number of authors appearing in each citation, my guess is that this may be another reason we are seeing steady growth in reference size. In a world of increasing specialization, authors frequently rely on other people to round out the research necessary for a complete paper. I recently heard of a Science paper that had more than 200 authors.

Publishers Respond

Nearly 40 years ago, Bennie Troxel wrote an editorial in Geology titled “What Price Reference Lists?” in which he questioned the value of reference lists and solicited ideas for how to handle what some considered a growing problem. In it, he noted that a recent GSA Bulletin issue contained about 10% references, or the equivalent of three papers. Troxel also noted that Geology was “a medium for experimentation” and that it planned to test a smaller font for references.

The editorial received a flood of comments, and while the vast majority of respondents were okay with the idea of smaller type for references, there was strong resistance to any further tampering of references. The comments highlighted the value of references for further reading lists, tracing the source of an author’s idea, and providing a glimpse of the author’s overall knowledge. As Marvin Lanphere summed up, “The thought of deleting reference lists is appalling.”

Today, Geology’s average reference section is about double the size it was when Troxel wrote that editorial. Even with its smaller font size for references, which was later adopted by GSA Bulletin, there can be little doubt that the growing references are reducing the amount of space available for an author to discuss her research.

For GSA Bulletin and Special Papers, which do not limit an article’s page length, the larger reference sections are still having an impact. In 1953, GSA Bulletin published about 76 pages of references. This year, I anticipate the journal will publish about 250 pages of references, or roughly the equivalent of 2.5 Geology issues. This means that even though a page of GSA Bulletin references today contains about 9500 characters compared to 5500 characters in 1953, the portion of the journal devoted to references has gone from ~4% in 1953 to ~12% today. Is this a problem?

Some publishers think so. At a recent meeting, a biology publisher revealed that they will not accept articles with more than ten references per paper. While this may limit the cost of producing, proofing, and printing large reference sections, is it the best decision for the science?

Other publishers have responded by shortening the amount of information in each reference. The idea here is that, now that we have (supposedly) definitive tools like doi numbers, do we still need to cite the full author list, article title, etc.? Science, for example, publishes full reference sections in its online version, but significantly abbreviated references in their print version. While this does save space, having two versions of a paper’s reference section also creates its own challenges.

Nature frequently eliminates all but the lead author’s name, does not include the article title, and abbreviates journal titles. As a result, the average reference for one of its recent papers was only 83 characters, which is well below the average length GSA was producing in the 1950s and about a third of the size of a current GSA reference. If GSA Bulletin took a similar approach to Nature, the journal would produce ~160 fewer pages per year, or roughly the equivalent of 10 articles.

Some will say that GSA should take this approach so that it can publish 10 more GSA Bulletin papers. Others might say that publishing 10 more papers would only feed this ever-increasing volume of literature and would actually exacerbate the problem. On one hand the idea of adopting a heavily abbreviated reference section is appealing, but on the other hand the future remains pretty uncertain. While it is tempting to think of things like doi numbers as permanent, we have made wrong turns before. It wasn’t until last year that we were finally able to digitize the GSA Bulletin content produced in microfiche only from 1979–1981. In the end it’s not just about what we need today, but also about what we may need in the future.