In the present academic world peer review is essential: first for scholars and their careers because peer-reviewed articles count for promotion and grants; second for all the interested people, like students, librarians, journalists, and for other scholars, due to the sort of quality that peer review assures to research. However, if this system works well for print scholarship, can it also be applied to digital scholarship? How to validate the quality and credibility of Digital Humanities projects?
In the article Rethinking Peer Review in the Age of Digital Humanities[1] Roopika Risam shows the difference between print and digital scholarships. Digital projects, unlike articles and books, are often collaborative, rarely finished and frequently public: these three aspects prevent a total application of the traditional peer review system. However, a kind of review and quality recognition is needed by DH scholars, especially by the junior ones who are asked to produce peer-reviewed works in order to build an academic career. As Sheila Cavanagh states, peer review is a key aspect to achieve reputation, opportunities and grants in academia and often forces graduate students and junior scholars to sacrifice their DH projects to spend more time producing more prestigious and recognised works, like articles and book chapters.[2] Sadly, in many faculties DH projects don’t receive the same consideration of publications, not taking into account the amount of labour behind a single CV line reporting a DH project.
According to Risam, new review standards should be created, different from the traditional ones because of the different nature of the two scholarships. She suggests metrics like tracking citations and usage statistics to evaluate digital projects: even though they are important indicators, in my opinion they don’t fully assure the quality of the content. Looking for new ways to transfer and adapt peer review to DH projects, I would like to analyse two solutions:
- The Advanced Research Consortium (ARC) Nodes: NINES, 18thConnect, MESA, ModNets, SiRO
ARC has created a mechanism of scholarly peer review to evaluate digital resources and archives which has been adapted to different studies or periods. MESA is built for scholars and researchers of Medieval studies; 18thConnect focuses on the long 18th-century (1660-1820), British and American; NINES concerns the long 19th-century (1770-1920), British and American; ModNets embraces the field of modernist literary and cultural studies, whereas SiRO devotes itself to archival resources for the study of radicalism.
All these sister organisations adopt the same tools and infrastructures, and the same peer review schema. Submitted projects should have integrity, clear conceptual design and a certain amount of content, even if they don’t have to be completed. The peer-review process requires the submission of a document with information about the project’s structure, its technical dependencies, mission and plans for the future, as well as the submission of metadata describing the objects within the resource in the form of RDF. Then the project is evaluated by an editorial board made up of the most respected scholars in the field. The projects are evaluated on the basis of two questions: the first “is the content important and interested to existing scholarship?” analyses the intellectual content and originality of the resource, thus falling back in the traditional concept of peer review; the second one “is the material presented in a clear, accessible, well-organized and well-documented fashion?” aims attention at the technical structure, the adherence to standards developed by the DH community (like TEI), the interface designs and interoperability.
This kind of peer review benefits a DH project in a number of ways because the editorial board guarantees its intellectual quality and validates it as equivalent to a print publication. This means that the work of a DH scholar can be fairly weighted also by faculty members without any knowledge in Digital Humanities. Furthermore, the peer review gives more credibility to the project, which will thus be more trusted by other scholars and by the general public. Last but not less important, peer-reviewed projects are aggregated in the ARC node’s website of competence and their objects become searchable also there, obtaining greater visibility and more users. - Reviews in Digital Humanities, edited by Jennifer Guiliano (IUPUI) and Roopika Risam (Salem State University)
It is a forthcoming pilot of a peer-reviewed journal (the deadline for the first call of projects was September 2019) which aims to facilitate the evaluation of DH projects. The journal will bring together both the projects’ descriptions written by their directors and the reviews. Furthermore, all the peer-reviewed projects will be listed in a projects’ registry. With this journal the two editors want to help DH scholars get peer reviews of their projects, considering the importance of peer review for promotion and future funding. I find the idea of this journal extremely interesting both because DH projects of any field can be submitted and because the review process will be transparent, publishing the reviews.
We look forward to reading it!
[1] R. Risam, ‘Rethinking Peer Review in the Age of Digital Humanities’, Ada: A Journal of Gender, New Media, and Technology, no.4, 2014, doi:10.7264/N3WQ0220.
[2] S. Cavanagh, ‘Living in a Digital World: Rethinking Peer Review, Collaboration, and Open Access’, Journal of Digital Humanities, vol. 1, no. 4, Fall 2012, http://journalofdigitalhumanities.org/1-4/living-in-a-digital-world-by-sheila-cavanagh/, (accessed 08/12/2019).
