Slicing and Dicing: Ex Post Approaches

Blog No. 2012-03 (May 4, 2012)

 

KEY INSIGHT:  In my February 2012 THE ETHICIST: RESEARCH posting, I discussed an important issue facing researchers: How do authors determine whether the papers coming out of one project are sufficiently different from one another so that they can be considered to be new papers? In my earlier posting, I looked at ex ante methods that authors could use to determine whether a paper was sufficiently new. Here I follow up with ex post methods for determining novelty; that is, once the paper has been written and submitted for review, how can reviewers and editors be assured of its originality?

1. INTRODUCTION

 

Let me start by saying that I am not against working in teams and developing multiple papers from a big research project that are then published in different journals. In fact, I am very much in favor of team-based research projects, and have been actively doing that for some years now with various co-authors. Clearly, we want scholars to gain the advantages of economies of scale, scope and learning that come from developing big projects that lead to multiple publications. Working in teams offers the advantages we typically associate with strategic alliances: the ability to bring together and leverage complementary resources, the creation of routines and capabilities that generate efficiencies and synergies, and the opportunity for greater flexibility and speed, among other advantages.

 

Big projects with two or more co-researchers are now common, perhaps even dominant, in terms of how we do research now within the academy, at least based on my observation of authorship in our scholarly journals where the norm now appears to be two, three and even four or five co-authors.  The April 2102 issue of AMJ, for example, has 10 articles; only one is single authored.  The April 2012 issue of AMR has six articles; two are single authored.

 

When a team of researchers work together for several years, each new paper builds off the previous ones as the authors develop a deeper understanding of their research area, see more nuances and find more puzzles to solve. As that building and expansion occur, basic ideas are likely to be repeated across the papers, and may look repetitious to an outsider – and to the authors themselves.

In my February blog, I addressed the issue of how authors themselves can determine the “bright line” between “sufficiently new” and “excessive overlap”. I recommended that authors do a originality analysis to compare papers, and that authors share this analysis with journal editors at the time of submission. [1]The originality analysis would then help the editor (and reviewers if the editor chooses to share the information) determine the degree of novelty. My general advice was: be transparent. Assuming an author has been transparent, it is up to the reviewers and editors to assess the extent of overlap and unique contribution.

 

However, what if the authors are not transparent and do not provide this information at the time of submission. How can a journal editor and the paper’s reviewers separate acceptable overlap from “slicing and dicing”?  More generally, how can editors and reviewers identify instances of scholarly dishonesty, and what should they do if they do find such evidence?

 

 

2. CASE EXAMPLE

 

Perhaps you have read the new article by Honig and Bedi, “The Fox in the Hen House”, published in Academy of Management Learning & Education. The authors examined the 279 papers presented at the 2009 AOM meetings, in the International Management division, for evidence of plagiarism. They used regression analysis to test hypotheses about possible antecedents to plagiarism (for example, gender, degrees from non-English speaking countries, junior/untenured faculty, authors located in emerging (“noncore”) economies). The software program Turnitin.com was used to determine the amount of plagiarized material in each paper. [2]

 

The results were startling: One-quarter of the presented papers (71 of the 279 papers) showed some evidence of plagiarism, and 13.6% had an average of 5% or more of the text plagiarized (approximately 1,000 words). The highest offenders appear to have been papers by an author or co-author located in non-core countries.  Over 40% of the papers written by scholars in noncore countries showed some evidence of plagiarism; half of those had more than 5% of the text plagiarized (Honig & Bedi, 2012: 113). For authors from “core” (developed) countries the percentages were lower but still higher than I think most faculty would have expected: 21% of papers showed some evidence of plagiarism and a quarter of those had more than 5% of the text plagiarized. Education in a noncore rather than a core country was also a differentiating factor (27% noncore vs 21% core); however, neither gender (male vs female) nor rank (untenured/junior vs tenured/senior) appeared to matter (Honig & Bedi, 2012:  116).

 

While one might argue that authors are typically less careful on conference submissions than they are on submissions to scholarly journals, the plagiarism issue clearly matters for both. Moreover, the plagiarism estimates of Honig and Bedi are underreports because the authors deliberately excluded self-plagiarism, stating that “If authors used sections from their own previous work or cited the primary source, then it was not considered plagiarism” (p. 112). If the authors had taken “slicing and dicing” into account, I suspect that the percentages – across the board – would have been much higher.

 

 

3. EX POST APPROACHES #1: SOFTWARE SOLUTIONS

 

If plagiarism is a real problem, and it apparently is, what should AOM do? More generally, what should we do as a community of scholars about scholarly dishonesty? Honig and Bedi conclude that “Institutional norms that many of us take for granted are clearly and brazenly being disregarded”; they call for AOM to “implement more rigorous standards in order to reduce plagiarism and to ensure high-quality and original scholarship” (p. 119).

 

Jean Bartunek, in her introduction to the Honig and Bedi article, notes that, starting in the spring of 2012, the AOM journals will begin using the software program CrossCheck to detect plagiarism in submissions.  It is not clear to me whether all submissions or only conditionally accepted papers will go through the CrossCheck process. Obviously putting all papers through would substantially increase the total cost.

 

However, regardless of the financial cost, submitting papers to some form of software such as Turnitin.com or CrossCheck seems to me to have now become a necessary part of the journal review procedure.  We may not like this nor like the psychosocial (lack of trust) message that it sends to prospective authors, but — given the huge numbers of papers now being submitted to our journals and the evidence in Hong & Bedi (2012) —  some form of plagiarism checking software appears to be inevitable.  AOM and other scholarly associations must move to adopt stronger ex post approaches to combating both plagiarism and self-plagiarism.

 

In addition to software options such as CrossCheck, I believe AOM and its scholarly journals would benefit from membership in national and international organizations that are devoted to improving ethics in professional associations and scholarly publishing. Through our memberships, we can signal our commitment to scholarly and professional ethics, learn and adopt best practices, and bring the weight and voice of AOM — 18,000 members from 110 countries — to bear on these issues. One of the best known of these associations is COPE.

 

 

4. EX POST APPROACHES #2: COPE TO THE RESCUE

 

COPE (the Committee on Publication Ethics) is a non-governmental organization that was set up by a group of journal editors to share best practices for handling ethical violations. Over time, the organization has developed a whole set of procedures that it recommends editors follow when faced with problems such as plagiarism and self-plagiarism.  More than 7,000 universities, journals and publishers are now COPE members.[3]

 

As an example of how COPE could be helpful, the Committee has a whole section on “salami publishing,” that is, two overlapping publications arising from the same research project.  (I call this “slicing and dicing”.) The key issues identified by COPE are (1) the degree of overlap between the two publications and (2) whether the author sought to hide the overlap.

 

COPE provides flowcharts to help editors determine best practice in identifying and handling cases of suspected misconduct.  Flowchart 1 deals with suspected redundant publication in a submitted manuscript; flowchart 2 with the same issue in a published article. In both flowcharts, the key issue is whether there is major or minor overlap/redundancy between the two papers. Major redundancy is defined as both papers having the “same dataset with identical findings and/or evidence that authors have sought to hide redundancy, e.g. by changing title, author order or not referring to previous papers”. Minor overlap is defined as “salami publishing with some element of redundancy or legitimate re-analysis (e.g. sub-group/extended follow-up/discussion aimed at different audience)”.  Note that evidence of appearing to hide the overlap can raise the level from minor to major redundancy.

 

Some examples of actual cases submitted to COPE may be helpful here in distinguishing between major and minor redundancy (search the COPE website for other examples):

 

  • Salami publication: Four papers were completed by the same research team, with each paper referencing the prior publications. The fourth paper in the series was rejected on the grounds that there was significant overlap between the new paper and the earlier publications.  COPE recommended, first, distinguishing between salami and redundant publication; arguing that if there were 2/3 overlap between the two papers, this was a redundant publication.  COPE defines a salami publication as covering “the same population, methods, and question”. Second, if the two papers asked related questions, they should be published as one paper; if the two papers asked separate questions, they could be separate publications. Splitting up papers by outcomes was not legitimate.

 

  • Duplicate publication or salami publication? A paper submitted to a journal is discovered by a reviewer to have been already published in another journal. When the editor contacts the author, he responds that the two papers are different.  COPE’s advice was to focus on the overlap between the two papers, determining whether the overlap was major (2/3 would make it a duplicate publication) or minor (a salami publication) and then follow COPE’s rules for one or the other event.

 

  • Duplicate submission:  Two papers based on the same research project on pathogens in school children were submitted to different journals. One analyzed the data by socio-economic class; the other by school attended. Substantial portion of the texts were the same in the two manuscripts, especially in the data description and research methods. COPE recommended determining whether this was a duplicate or salami publication.

 

We have much to learn from other associations and journals that have faced the same or similar issues to the ones identified in Honig and Bedi (2012).  In addition, we have much to offer to other associations and journals by being at the table and exercising voice. I believe that AOM, with its Code of Ethics and well-developed ethical policies and procedures, can help shape international best practices in the research ethics arena.

 

 

5. EX POST APPROACHES #3: EDUCATION

 

A third ex post method to handle scholarly dishonesty is to create a library of resources for sharing key reports among authors, editors and reviewers. For example, in 2009, the OECD’s Global Science Forum published a committee report and practical guide to Best Practices for Ensuring Scientific Integrity and Preventing Research Misconduct. Reports and guides of best practices, such as the OECD one, could be brought together in one location (a website), making them more widely and quickly available to scholars. AOM already does some of this; its Ethics@AOM web page includes links to articles on ethics in research published in AOM journals, the Ethics Education Video Series, and THE ETHICIST blog posts, among other resources.

 

Ultimately, while software programs like CrossCheck and organizations like COPE can help journal editors and reviewers find and evaluate possible cases of scholarly misconduct, the “rubber meets the road” through student and faculty education. This is where AOM’s Ethics and Ethics Education Committees can and does already play an important role, by activities such as:

 

  • Providing ethical training of PhD students and junior faculty in the Professional Development Workshops at the AOM annual meetings;
  • Organizing panels and roundtables on Ethics in the Academy at the AOM meetings and elsewhere;
  • Creating publicly available videos on ethics in research; and
  • Writing for THE ETHICIST!

 

I hope to see you in Boston in August where we can continue this conversation and look forward to your responses posted on THE ETHICIST.

 

 

6. QUESTIONS FOR DISCUSSION

 

  1. What is your response to the Honig and Bedi article?

 

  1. If you are a journal editor, have you been faced with situations similar to those described in this blog? How did you handle them? Do you agree with the COPE templates? Is your journal a member of COPE?

 

  1. As a reviewer, what would you do if you were presented with evidence of two overlapping papers? Should you inform the editors?

 

  1. Should AOM and its journals be a member of COPE?

 

 

REFERENCES

 

Bartunek, Jean. 2012. Introduction: Plagiarism in Submissions to the AOM Conference. Academy of Management Learning & Education, 11.1: 99-100.

 

Eden, Lorraine. 2012. Slicing and Dicing: Ex Ante Approaches. THE ETHICIST. Blog No. 2012-02 (February). http://connect.aomonline.org/posts/c72b3d87ac.

 

Honig, Benson and Akanksha Bedi. 2012. The Fox in the Hen House: A Critical Examination of Plagiarism Among Members of the Academy of Management. Academy of Management Learning & Education, 11.1: 101-123.

 

Kirkman, Brad and Gilad Chen. 2011. Maximizing your data or data slicing? Recommendations for managing multiple submissions from the same dataset.  Management and Organization Review 7:3: 433-446.

 

OECD Global Science Forum.   2009a. Facilitating International Research Misconduct Investigations. Final Report of the Co-ordinating Committee. Paris: OECD. May 7.  Accessed May 3, 2012. http://www.oecd.org/dataoecd/29/4/42713295.pdf

 

OECD Global Science Forum. 2009b. Practical Guide for Investigating Research Misconduct Allegations in International Collaborative Research Projects. Paris: OECD. May 15.  Accessed May 3, 2012. http://www.oecd.org/dataoecd/42/34/42770261.pdf

ACKNOWLEDGEMENTS:  I gratefully thank Chi Anyansi-Archibong, Kathy Lund Dean, Kim Eggleton, Deidra Schleicher and Paul Vaaler for helpful comments in connection with an earlier draft of this blog posting, while acknowledging that the views expressed here are my own.

 

——————————————————————————————————————————-

Download as PDF

 


[1] In my February 2012 blog, I discuss the transparency matrix in Kirkman and Chen (2011) and the originality matrix used by the Journal of Applied Psychology, and how both matrices can be used in an originality analysis.  I am grateful to Deidra Schleicher and Steve Kozlowski for providing access to the JAP form.

[2] A brief aside: I require my undergraduate and graduate (masters and PhD) students to turn their papers into Turnitin.com before they are submitted to me. The students can submit as often as they like, and in this manner, they can learn what is and is not plagiarism. I accept the papers only after they have received a “green” rating from Turnitin.com. My cases of plagiarism have basically disappeared since I started this practice some years ago, and I highly recommend it to other instructors.

[3]  I give away my bias here by noting that the Journal of International Business Studies joined COPE (as did all the Palgrave journals) during my editorship. I believe that AOM should also join COPE and adopt (where appropriate) its strategies and procedures for handling ethical violations.

Author: Lorraine Eden

Research