Blog No. 2012-02 (February 2, 2012)
Key Insight: Research projects are often huge undertakings that lead to more than one publication. How do authors determine whether the papers coming out of one project are sufficiently different from one another to be considered new papers? In this blog, I look at some ex ante methods that authors can use to determine whether a paper is new.
1. Case Examples
Example 1: Two co-authors have a major project underway and want to maximize the number of publications from the project. They recognize that journal editors frown on “slicing and dicing” and want to make sure that the papers are sufficiently different so they reallyare different papers. However, the co-authors don’t know what makes one paper sufficiently different from another. Is it the dataset? The hypotheses? The empirical findings? They search for information on what makes one paper sufficiently different from another and cannot find a definitive answer.
Example 2: A new assistant professor is carving up his dissertation into journal articles. He sends the first article, which has the major theoretical and empirical contributions of the dissertation, to Academy of Management Journal (AMJ). The second article, which looks at two moderators of the main effects in the first paper, is sent to the Journal of International Business Studies (JIBS). Both papers use the same dataset and variables, with the exception that the moderator variables in the JIBS submission are treated as control variables in the AMJ submission. The hypotheses in the JIBS submission include some of the same hypotheses that appear in the AMJ submission, with the addition of new ones for the moderator effects. The AMJ submission goes in first; the JIBS submission follows a month later. In the JIBS submission, the author makes no mention of the prior submission to AMJ, either in the letter to the editor or in the body of the paper. Nor does the author tell AMJ that a second submission to JIBS is planned. The author reasons that he does not need to mention either submission to the other journal because neither submission has been published and, even if both papers should eventually be accepted for publication, they will be so changed during the reviewing process that the likelihood of duplicate material is low.
Example 3: Two years earlier, a professor published an article in JIBS. She has now refined her thinking and has a follow-up article building on the first one, but still using the same dataset. Can she also submit the follow-up paper to JIBS?
Example 4: Three co-authors submitted a paper to AMJ, which was rejected after the first round of review. The co-authors spent a year significantly revising the paper based on the reviewers’ and editor’s feedback. The co-authors believed that the revised paper was sufficiently different that they could submit the revised paper to AMJ as a “new submission”. In their cover letter to the journal, they made no mention of the previously rejected submission.
2. The Problem
All four of the above examples are slightly disguised real-world examples with which I am familiar, either from my own research, my term as a JIBS editor and/or from discussions with other journal editors. I suspect you can add more examples to the ones I have above.
All four examples involve what I refer to as the potential for “slicing and dicing” and the Committee on Publication Ethics (COPE) calls “salami publishing” or “redundancy”; that is, the excessive cutting up of a research project into multiple papers where each paper overlaps significantly with other papers from the same project. Examples 1 and 2 involve situations where the papers come simultaneously out of the same project. Example 3 (closely related papers follow one another in sequence) and Example 4 (the authors revise a previously rejected paper and send the revised paper to the same journal) involve sequential slicing and dicing.
The core issue in all of these cases is determining when a paper is really or sufficiently new. How do we know when it is OK to publish two or more papers out of one project? Where do we cross the line from being OK to engaging in slicing and dicing?
We all know how heavy are the publish-or-perish pressures, especially for junior faculty. (On this topic, see my first THE ETHICIST: RESEARCH blog posting (Eden, August 2012), “Scientists behaving badly: Lessons from the Fraud Triangle”.) An author’s desire to segment his or her big research project into multiple, stand-alone papers aimed at different journals is therefore not surprising. The key issue is where to draw the line between two papers that are “siblings” (same intellectual parents but different children) and those that are “clones”?
In this blog, I address ex ante approaches to handling the ethical dilemma of slicing and dicing. In my May 2012 blog, I will look at ex post approaches as recommended by COPE.
3. Ex Ante Approaches
I see four possible ex ante approaches for handling the slicing-and-dicing problem.
a. Craft Different Papers at Project Inception
Brad Kirkman and Gilad Chen in their article, “Maximizing your data or data slicing? Recommendations for managing multiple submissions from the same dataset” (Management and Organization Review 7:3, 2011, 433-446), provide our first ex ante approach. When authors are starting out at the beginning of a project, it is easier if they “intentionally craft and design [… ] separate papers from the inception of the project” (Kirkman & Chen, 2011: 437). By starting at the beginning, authors have a roadmap that helps keep the papers separate. The papers can, for example, be aimed at different audiences, start with different research questions and theoretical approaches, and/or use different datasets.
b. Follow the Journal’s Instructions
What if “the horse is out of the barn” and you didn’t craft separate papers from the beginning? What can you do? My advice is to first turn to what the journal editors say on this topic. Editors want innovative, thought-provoking, original articles published in their journals. They know about the pressures to engage in slicing and dicing, and that authors may check the box that an article is “original” even if it comes out a big research project. Most journals therefore have an explicit policy defining originality and asking authors to confirm at the time of submission that their paper is original.
The AMJ submission requirements are probably typical of most journals. AMJ requires authors to “check the box” that (1) their manuscript is original, (2) not published or under review at another journal, and (3) will not be submitted to another journal during the review process. In addition, the submission requirements ask authors to check the box to “confirm that their manuscripts have not previously been submitted to AMJ for review.” Some journals go beyond this list to define what they see as an original manuscript; theJIBS Code of Ethics for Authors, for example, devotes several paragraphs to what the editors see as original and what the journal considers to be self-plagiarism or redundancy.
In my Example 1 (carving out papers from a project) and Example 2 (carving up a dissertation into papers), the authors should therefore look to the journals for definitions of originality, both to the Instructions to Authors and to the Code of Ethics (if the journal has one).
Example 3 (sequential papers) and Example 4 (revised and submitted to the same journal after rejection) are slightly different problems. Journals also do provide instructions that are helpful for these situations. Michele Kacmar’s 2009 AMJ “From the Editors” letter, “An Ethical Quiz”, specifically addresses these cases in her Scenario 2: Data Reuse. Kacmar (2009: 432) explains that AMJ requires authors to answer two questions, which I quote below:
- Has another manuscript from this same database ever been previously submitted to AMJ? If yes, please note this in your cover letter, explain how this paper differs from the earlier one, and attach a copy of the previous manuscript.
- Has another manuscript from this same database been accepted by or previously published at AMJ or at another journal? If yes, please note this in your cover letter, explain how this paper differs from the previous one, and attach a copy of the accepted or published manuscript.
The first question addresses papers that have been previously submitted to AMJ (Example 4); the second addresses articles previously published in AMJ (Example 3). In a situation where the author says “yes” to either question, AMJ requires the author to add an explanation to the cover letter at the time of submission and attach the other manuscript.
JIBS also has an FAQ posted on http://www.jibs.net that discourages resubmissions when a manuscript has been rejected after review, except in special circumstances that are outlined in the FAQ. The JIBS Code of Ethics for Authors does not specifically address Example 3 (sequential publications) but I believe the case would fall within the section on Self-Plagiarism.
b. Do an Originality Analysis
Both AMJ and JIBS, interestingly, do allow a bit of wiggle room for exceptions from what might be called the “no second kick at the can” rule deterring authors from making sequential slice-and-dice submissions to the same journal. It is this wiggle room – as mapped out by AMJ and JIBS — that I see as really helpful to authors in determining when carving up a project into papers moves from OK into the unacceptable realm of slicing and dicing.
The JIBS FAQ would allow the authors in Example 4 to make a new submission if “the revised manuscript becomes a new manuscript through significant revision in terms of theory development, empirical work and discussion, and also uses a substantially different dataset”. The FAQ adds that “the addition of one or two new variables to an old dataset does not make a new dataset.”
Example 4 was also directly addressed in a 2009 AMJ “From the Editors” letter by (at the time) editor-in-chief Duane Ireland. His editorial, “When Is a “New” Paper Really New?”, specifically lists three criteria that must be met for a previously rejected manuscript to be considered a new submission to AMJ: “The new manuscript must (1) address modified or new research questions, (2) use new theoretical arguments, and (3) use additional or new data to test the proposed relationships. Satisfying or meeting one or two of the three criteria is not sufficient.” (Ireland, 2009: 10) So, if the authors in Examples 3 and 4 were to meet all three criteria, it would be OK for them to make a new submission to AMJ.
These two editorial policy statements suggest a useful way for authors to determine when a paper that is part of a project is sufficiently new to be separately published. If the authors create an originality matrix comparing the two papers in terms of their component parts, it should be clear both authors and to the journal editors whether there is sufficient differentiation to justify separation of the papers. The statements by the two journals suggest that AMJ would require differentiation in three areas: research question, theoretical arguments and dataset. To this list, JIBS would add empirical tests and discussion. Brad Kirkman and Gilad Chen (2011) develop a similar matrix, which they call a uniqueness analysis, based on five components: research question, theories used, constructs/variables, and theoretical implications and managerial implications. Kirkman and Chen provide helpful two tables, using their own published papers, to show how authors can compare manuscripts in terms of originality.
Based on these three sets of criteria for originality identified by Ireland, Kirkman and Chen, and the JIBS FAQ, I recommend that authors set up a matrix where the columns are papers and the rows are criteria used to judge originality. See Table 1 below.
Table 1: An Originality Matrix (Part 1)
|Constructs / Variables Used|
|Discussion: Theoretical Implications|
|Discussion: Managerial Implications|
To make the comparison even sharper, I argue that it is important not only to fill in the table cells, but also to examine differences andsimilarities or overlap between the two papers. My matrix therefore also has two columns at the end where the author must assess overlap and differences. (When I was JIBS editor-in-chief, I occasionally asked authors to complete a version of Table 1 in cases that appeared to possibly involve slicing and dicing. The authors and I then engaged in a dialogue, based on the matrix, to determine whether the new submission was sufficiently new. These dialogues led to the JIBS FAQ.)
Completing Table 1, of course, forces the author to stand back and be ruthlessly honest about both papers – not an easy task. Authors must use a self-critical eye or the exercise is pointless. Once the exercise has been completed, the author should look hard at the answers, particularly the last two columns on overlaps and differences. These columns may well suggest ways that the two papers could be further revised so as to make them even more separate.
Is Table 1 sufficient to determine originality? Probably not for empirical papers. To do this rigorously and thoroughly, I believe that one must go further than Table 1 and delve into the issue of what makes two datasets different. Journal of Applied Psychology (JAP) has a long-standing policy of publishing original data. JAP requires authors to inform the editorial team, either in their cover letter or in the methods section, if their dataset has or will be used in other journal submissions not only to JAP but to other journals. If an author says “yes”, JAP sends the author a separate form to complete, which I attach below as Table 2. [ii] Based on the completed Table 2, together with any accompanying documentation (e.g., other manuscripts), the JAP editors can more easily and accurately determine whether or not to accept the manuscript as a new submission.
Table 2: JAP Original Data Appendix (Originality Matrix Part 2)
|Instructions: Authors should edit accordingly to describe what has been done and what is planned. Use as many columns as necessary. Provide any additional information necessary to clarify the unique contribution of each manuscript. Please note the STATUS of each manuscript connected to the data collection: under review, in press, published, current ms, planned (anything else).—————————-The data reported in this manuscript have been previously published and / or were collected as part of a larger data collection (at one or more points in time). Findings from the data collection have been reported in separate manuscripts. MS 1 (status) focuses on variables _____; MS 2 (status) focuses on variables _____. MS 3 (status) focuses on variables _____. MS 4 (status) will focus on variables ____. The table below displays where each data variable appears in each study, as well as the current status of each study.
I therefore view a completed originality matrix as consisting of carefully and honestly prepared Tables 1 and 2. A completed originality matrix can help with all four of my examples, whether starting a project and trying to determine the optimal number and content of the papers (Example 1), carving up a dissertation (Example 2), or the sequential issues discussed in Examples 3 and 4. By completing both tables, authors can determine whether two papers are sufficiently different that they can ethically be submitted to the same or different journals and published separately.
d. Transparency Matters
In addition to determining originality, there is an additional ethical issue involved in slicing and dicing: transparency. What should the author tell the journal editors at the time of submission? I recommend that authors “spill the beans”. Transparency is the best policy. Go for full disclosure, as Marshall Schminke, (at the time) chair of the AOM Ethics Education Committee, argued in his AMR Editor’s Comments, on “The Better Angels of Our Nature – Ethics and Integrity in the Publishing Process”. Submitting a cover letter with the completed originality matrix and the relevant papers to the journal editors accomplishes full transparency.
Kirkman and Chen (2011: 442) are also strong advocates for full transparency. They recognize that transparency jeopardizes the double blind review process, but argue that transparency matters more, given that the “ultimate goal of science is to build and advanceour knowledge base”, which requires a clear assessment of unique contribution of each paper. I recognize the problem (see myNovember 2011 THE ETHICIST: RESEARCH blog on double blind review), but also agree with their assessment and have argued so elsewhere (Eden, JIBS Editorial, “Scientists behaving badly, 2010). Transparency – in the case of possible slicing and dicing – matters. Journal editors should be provided with full information by authors, and then the decision on whether to share the originality matrix and papers with the journal’s reviewers (and thus violate one side of double blind review) should be left up to the journal editors.
To recap, I argue that there are established criteria for determining whether two papers from the same research project are sufficiently different to be considered that both can be treated as new papers. Authors should use these criteria to determine whether a paper is original. I summarize the implications for my four Examples below:
- Example 1 (carving up a project into papers): Best to do this at the beginning, not the end, of a project. Authors should deconstruct their papers using the originality matrix (Tables 1 and 2) and share these results with journal editors of the different papers at the time of submission.
- Example 2 (main effect paper followed by moderator paper): The case, as described, would not pass the originality matrix test; moreover, there is a lack of transparency. The author should revise both papers, using the originality matrix, until they are sufficiently different to be treated as separate. He should provide both the matrix and papers to both journals.
- Example 3 (Second paper that grew out of the first one): The answer is this depends on how different the two papers are from one another. Again, an originality matrix is needed and all information should be supplied to the journal if the author decides to go forward with submission.
- Example 4 (Once rejected after review, can I revise and make a new submission?): The answer is probably not, except in exceptional circumstances. These circumstances require an assessment that the two papers were sufficiently different, based on a completed originality matrix, and submission of the matrix and papers to the journal editors.
4. Questions for discussion
I hope that this blog posting provides some food for thought. I return to this topic in my next THE ETHICIST: RESEARCH blog, looking at ex post solutions. Some possible questions for discussion might be:
- Do you think that “slicing and dicing” (or “salami publishing”) is a problem?
- Have you been faced with situations such as the ones described in this blog posting? How did you or you and your co-authors handle them?
- What criteria would you use to determine when a manuscript is new?
- Is the policy advocated here (an originality matrix plus transparency) too onerous a burden to place on authors?
- What advice do you give to your doctoral students and junior faculty about managing a big research project like a dissertation through to publication?
[i] I thank the many members of THE ETHICIST advisory board who provided helpful comments and shared resources on this topic, especially Kathy Lund Dean, Susan Jackson, Deidra Schleicher and Anne Tsui. The views expressed here are my own.
[ii] I thank Deidra Schleicher and Steve Kozlowski at JAP for providing access to this form and allowing its publication in THE ETHICIST.