An 'important contribution' or 'tiresome reading'? A study of evaluation in peer reviews of journal article submissions
DOI:
https://doi.org/10.1558/japl.v1.i3.247Keywords:
peer review, evaluation, publication process, adjectives, corpusAbstract
While the process of peer reviewing journal articles submitted for publication has been extensively investigated, particularly in the biomedical field, the language of peer reviews is relatively unexplored. This paper studies evaluation in an electronic corpus of 228 reviews submitted to the journal English for Specific Purposes (ESP). The research focuses on the things (or entities) evaluated and the adjectives associated with these. Entities and adjectives are categorised and quantified in order to ascertain what things are valued by reviewers and the qualities by which they are judged. The findings suggest that reviewers take on multiple roles, at the same time discouraging the publication of work that fails to meet the required standards and offering encouragement to authors and guiding them towards publication. These findings have implications for authors submitting research papers, those who support authors in this process, and journal editors.
References
Austin, J. L. (1962) How to do Things with Words. Oxford: Clarendon Press.
Bazerman, C. (1988) Shaping Written Knowledge. Madison: University of Wisconsin Press.
Benfield, J. R. and Howard, K. M. (2000) The language of science. European Journal of Cardiothoracic Surgery 18: 642–8.
Berkenkotter, C. and Huckin, T. (1995) Genre Knowledge in Disciplinary Communication. Hillsdale, NJ: Lawrence Erlbaum.
Bordage, G. and Caelleigh, A. S. (2001) A tool for reviewers: review criteria for research manuscripts. Academic Medicine 76: 904–8.
Burrough-Boenisch, J. (2003) Shapers of published NNS research articles. Journal of Second Language Writing 12: 223–43.
Canagarajah, A. S. (1996) ‘Non-discursive’ requirements in academic publishing, material resources of periphery scholars, and the politics of knowledge production. Written Communication 13(4): 435–72.
Canagarajah, A. S. (2002) A Geopolitics of Academic Writing. Pittsburgh: University of Pittsburgh Press.
Canagarajah, A. S. (2003) A somewhat legitimate and very peripheral participation. In C. P. Casanave and S. Vandrick (eds) Writing for Scholarly Publication: behind the scenes in language education 197–210. Mahwah, NJ: Lawrence Erlbaum Associates.
Casanave, C. P. and Vandrick, S. (2003a) Introduction: Issues in writing for publication. In C. P. Casanave and S. Vandrick (eds) Writing for Scholarly Publication: behind the scenes in language education 1–13. Mahwah, NJ: Lawrence Erlbaum Associates.
Casanave, C. P. and Vandrick, S. (2003b) Writing for Scholarly Publication: behind the scenes in language education. Mahwah, NJ: Lawrence Erlbaum Associates.
Chilton, S. (1999) The good reviewer. Academe 85 (6): 54–5. Accessed at http://www. d.umn.edu/~schilton/Articles/Reviewer.html on 21st December 2004.
Cronin, B. (1984) The Citation Process. London: Taylor Graham.
Curry, M. J. and Lillis, T. (2005) Multilingual scholars and the imperative to publish in English: negotiating interests, demands and rewards. TESOL Quarterly 38: 663-88.
Ernst, E. and Resch, K. L. (1994) Reviewer bias: a blinded experimental study. Journal of Laboratory and Clinical Medicine 124: 178–82.
Flowerdew, J. (1999a) Problems in writing for scholarly publication in English: the case of Hong Kong. Journal of Second Language Writing 8(3): 243–64.
Flowerdew, J. (1999b) Writing for scholarly publication in English: the case of Hong Kong. Journal of Second Language Writing, 8(2): 123–145.
Flowerdew, J. (2000) Discourse community, legitimate peripheral participation, and the non-native English-speaking scholar. TESOL Quarterly 34(1): 127–50.
Flowerdew, J. (2001) Attitudes of journal editors to non-native-speaker contributions: an interview study. TESOL Quarterly 35(1): 121–50.
Fox, R. (2003) Centre and periphery revisited. The structures of European science, 1750– 1914. Introduction. Revue de la Maison Française d’Oxford 1, 2. Accessed at http://www. mfo.ac.uk/Publications/Revue%20Fox/introduction.htm on 18th March 2005.
Gilbert, G. N. (1976) The transformation of research findings into scientific knowledge. Social Studies of Science 6: 281–306.
Gilbert, G. N. and Mulkay, M. (1982) Warranting scientific belief. Social Studies of Science 12: 383–408.
Godlee, F., Gale, C. and Martyn, C. (1998) Effect on the quality of peer review of blinding reviewers and asking them to sign their reports. The Journal of the American Medical Association 280(3): 237–40.
Goldbeck-Wood, S. (1998) What makes a good reviewer of manuscripts? British Medical Journal 316: 86.
Gosden, H. (2001) ‘Thank you for your critical comments and helpful suggestions’: compliance and conflict in authors’ replies to referees’ comments in peer reviews of scientific research papers. IBÉRICA 3: 3–17.
Gosden, H. (2002) Thematic content in peer reviews of scientific papers. Seminários de Linguística 5.
Gosden, H. (2003) ‘Why not give us the full story?’: functions of referees’ comments in peer reviews of scientific research papers. Journal of English for Academic Purposes 2(2): 87–102.
Graddol, D. (1997) The Future of English? London: British Council.
Hyland, K. (2000) Disciplinary Discourses: social interactions in academic writing. London: Longman.
Hunston, S. (1993) Evaluation and ideology in scientific writing. In M. Ghadessy (ed.) Register Analysis: theory and practice 57–73. London: Pinter.
Jefferson, T., Wager, E. and Davidoff, F. (2002) Measuring the quality of editorial peer review. Journal of the American Medical Association 287(21): 2786–90.
Kourilová, M. (1998) Communicative characteristics of reviews of scientific paper written by non-native users of English. Endocrine Regulations 32: 107–14.
Kuhn, T. (1970) The Structure of Scientific Revolutions. (Second edition) Chicago: University of Chicago Press.
Latour, B. and Woolgar, S. (1979) Laboratory Life: the social construction of scientific facts. Beverly Hills: Sage.
Lave, J. and Wenger, E. (1991) Situated Learning: legitimate peripheral participation. Cambridge: Cambridge University Press.
McNutt, R. A., Evans, A. T., Fletcher, R. H., et al. (1990) The effects of blinding on the quality of peer review: a randomised trial. Journal of the American Medical Association 263: 1371–6.
Myers, G. (1990) Writing Biology: texts in the construction of scientific knowledge. Madison: University of Wisconsin Press.
Nylenna, M., Riis, P. and Karlsson, Y. (1994) Multiple blinded reviews of the same two manuscripts: effect of referee characteristics and publication language. Journal of the American Medical Association 272(2): 149–51.
Pennycook, A. (1994) The Cultural Politics of English as an International Language. London: Longman.
Phillipson, R. (1992) Linguistic Imperialism. Oxford: Oxford University Press. Rowland, F. (2002) The Peer Review Process. A report to the JISC Scholarly Communication Group. Accessed at http://www.jisc.ac.uk/uploaded_documents/rowland.pdf on 14th December 2004.
Scott, M. (1996) Wordsmith Tools. Oxford: Oxford University Press. Searle, J. R. (1969) Speech Acts. London: Cambridge University Press.
Swales, J. (1981) Aspects of Article Introductions. (Aston ESP Research Reports 1.) Birmingham: Language Studies Unit, University of Aston.
Swales, J. (1996) Occluded genres in the academy: the case of the submission letter. In E. Ventola and A. Mauranen (eds) Academic Writing: intercultural and textual issues 45–58. Amsterdam: John Benjamins.
Thetela, P. (1997) Evaluated entities and parameters of value in academic research articles. English for Specific Purposes 16: 101–18.
Thompson, G. and Hunston, S. (2000) Evaluation: an introduction. In S. Hunston and G. Thompson (eds) Evaluation in Text: authorial stance and the construction of discourse 1–27. Oxford: Oxford University Press.
van Rooyen, S., Godlee, F., Evans, S., Black, N. and Smith, R. (1999) Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. British Medical Journal 318: 23–7.
Walsh, E., Rooney, M., Appleby, L. and Wilkinson, G. (2000) Open peer review: a randomised controlled trial. The British Journal of Psychiatry 176: 47–51.
Wood, D. and Hurst, P. (2000) Online peer review: perceptions in the biological sciences. Learned Publishing 13(2): 95–100.
Bazerman, C. (1988) Shaping Written Knowledge. Madison: University of Wisconsin Press.
Benfield, J. R. and Howard, K. M. (2000) The language of science. European Journal of Cardiothoracic Surgery 18: 642–8.
Berkenkotter, C. and Huckin, T. (1995) Genre Knowledge in Disciplinary Communication. Hillsdale, NJ: Lawrence Erlbaum.
Bordage, G. and Caelleigh, A. S. (2001) A tool for reviewers: review criteria for research manuscripts. Academic Medicine 76: 904–8.
Burrough-Boenisch, J. (2003) Shapers of published NNS research articles. Journal of Second Language Writing 12: 223–43.
Canagarajah, A. S. (1996) ‘Non-discursive’ requirements in academic publishing, material resources of periphery scholars, and the politics of knowledge production. Written Communication 13(4): 435–72.
Canagarajah, A. S. (2002) A Geopolitics of Academic Writing. Pittsburgh: University of Pittsburgh Press.
Canagarajah, A. S. (2003) A somewhat legitimate and very peripheral participation. In C. P. Casanave and S. Vandrick (eds) Writing for Scholarly Publication: behind the scenes in language education 197–210. Mahwah, NJ: Lawrence Erlbaum Associates.
Casanave, C. P. and Vandrick, S. (2003a) Introduction: Issues in writing for publication. In C. P. Casanave and S. Vandrick (eds) Writing for Scholarly Publication: behind the scenes in language education 1–13. Mahwah, NJ: Lawrence Erlbaum Associates.
Casanave, C. P. and Vandrick, S. (2003b) Writing for Scholarly Publication: behind the scenes in language education. Mahwah, NJ: Lawrence Erlbaum Associates.
Chilton, S. (1999) The good reviewer. Academe 85 (6): 54–5. Accessed at http://www. d.umn.edu/~schilton/Articles/Reviewer.html on 21st December 2004.
Cronin, B. (1984) The Citation Process. London: Taylor Graham.
Curry, M. J. and Lillis, T. (2005) Multilingual scholars and the imperative to publish in English: negotiating interests, demands and rewards. TESOL Quarterly 38: 663-88.
Ernst, E. and Resch, K. L. (1994) Reviewer bias: a blinded experimental study. Journal of Laboratory and Clinical Medicine 124: 178–82.
Flowerdew, J. (1999a) Problems in writing for scholarly publication in English: the case of Hong Kong. Journal of Second Language Writing 8(3): 243–64.
Flowerdew, J. (1999b) Writing for scholarly publication in English: the case of Hong Kong. Journal of Second Language Writing, 8(2): 123–145.
Flowerdew, J. (2000) Discourse community, legitimate peripheral participation, and the non-native English-speaking scholar. TESOL Quarterly 34(1): 127–50.
Flowerdew, J. (2001) Attitudes of journal editors to non-native-speaker contributions: an interview study. TESOL Quarterly 35(1): 121–50.
Fox, R. (2003) Centre and periphery revisited. The structures of European science, 1750– 1914. Introduction. Revue de la Maison Française d’Oxford 1, 2. Accessed at http://www. mfo.ac.uk/Publications/Revue%20Fox/introduction.htm on 18th March 2005.
Gilbert, G. N. (1976) The transformation of research findings into scientific knowledge. Social Studies of Science 6: 281–306.
Gilbert, G. N. and Mulkay, M. (1982) Warranting scientific belief. Social Studies of Science 12: 383–408.
Godlee, F., Gale, C. and Martyn, C. (1998) Effect on the quality of peer review of blinding reviewers and asking them to sign their reports. The Journal of the American Medical Association 280(3): 237–40.
Goldbeck-Wood, S. (1998) What makes a good reviewer of manuscripts? British Medical Journal 316: 86.
Gosden, H. (2001) ‘Thank you for your critical comments and helpful suggestions’: compliance and conflict in authors’ replies to referees’ comments in peer reviews of scientific research papers. IBÉRICA 3: 3–17.
Gosden, H. (2002) Thematic content in peer reviews of scientific papers. Seminários de Linguística 5.
Gosden, H. (2003) ‘Why not give us the full story?’: functions of referees’ comments in peer reviews of scientific research papers. Journal of English for Academic Purposes 2(2): 87–102.
Graddol, D. (1997) The Future of English? London: British Council.
Hyland, K. (2000) Disciplinary Discourses: social interactions in academic writing. London: Longman.
Hunston, S. (1993) Evaluation and ideology in scientific writing. In M. Ghadessy (ed.) Register Analysis: theory and practice 57–73. London: Pinter.
Jefferson, T., Wager, E. and Davidoff, F. (2002) Measuring the quality of editorial peer review. Journal of the American Medical Association 287(21): 2786–90.
Kourilová, M. (1998) Communicative characteristics of reviews of scientific paper written by non-native users of English. Endocrine Regulations 32: 107–14.
Kuhn, T. (1970) The Structure of Scientific Revolutions. (Second edition) Chicago: University of Chicago Press.
Latour, B. and Woolgar, S. (1979) Laboratory Life: the social construction of scientific facts. Beverly Hills: Sage.
Lave, J. and Wenger, E. (1991) Situated Learning: legitimate peripheral participation. Cambridge: Cambridge University Press.
McNutt, R. A., Evans, A. T., Fletcher, R. H., et al. (1990) The effects of blinding on the quality of peer review: a randomised trial. Journal of the American Medical Association 263: 1371–6.
Myers, G. (1990) Writing Biology: texts in the construction of scientific knowledge. Madison: University of Wisconsin Press.
Nylenna, M., Riis, P. and Karlsson, Y. (1994) Multiple blinded reviews of the same two manuscripts: effect of referee characteristics and publication language. Journal of the American Medical Association 272(2): 149–51.
Pennycook, A. (1994) The Cultural Politics of English as an International Language. London: Longman.
Phillipson, R. (1992) Linguistic Imperialism. Oxford: Oxford University Press. Rowland, F. (2002) The Peer Review Process. A report to the JISC Scholarly Communication Group. Accessed at http://www.jisc.ac.uk/uploaded_documents/rowland.pdf on 14th December 2004.
Scott, M. (1996) Wordsmith Tools. Oxford: Oxford University Press. Searle, J. R. (1969) Speech Acts. London: Cambridge University Press.
Swales, J. (1981) Aspects of Article Introductions. (Aston ESP Research Reports 1.) Birmingham: Language Studies Unit, University of Aston.
Swales, J. (1996) Occluded genres in the academy: the case of the submission letter. In E. Ventola and A. Mauranen (eds) Academic Writing: intercultural and textual issues 45–58. Amsterdam: John Benjamins.
Thetela, P. (1997) Evaluated entities and parameters of value in academic research articles. English for Specific Purposes 16: 101–18.
Thompson, G. and Hunston, S. (2000) Evaluation: an introduction. In S. Hunston and G. Thompson (eds) Evaluation in Text: authorial stance and the construction of discourse 1–27. Oxford: Oxford University Press.
van Rooyen, S., Godlee, F., Evans, S., Black, N. and Smith, R. (1999) Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. British Medical Journal 318: 23–7.
Walsh, E., Rooney, M., Appleby, L. and Wilkinson, G. (2000) Open peer review: a randomised controlled trial. The British Journal of Psychiatry 176: 47–51.
Wood, D. and Hurst, P. (2000) Online peer review: perceptions in the biological sciences. Learned Publishing 13(2): 95–100.
Downloads
Published
2004-12-05
Issue
Section
Articles
License
Equinox Publishing Ltd.
How to Cite
Hewings, M. (2004). An ’important contribution’ or ’tiresome reading’? A study of evaluation in peer reviews of journal article submissions. Journal of Applied Linguistics and Professional Practice, 1(3), 247-274. https://doi.org/10.1558/japl.v1.i3.247
