Ross Barclay (University of Toronto); Mary Bell (University of Toronto); Susan Rock (University of Toronto)

Purpose

To build a web based tool to coordinate a peer review qualitative system.

Method

Based on an already established web based peer review process, discussions occurred about the adaptation to a web based meta-ethnography tool to review and screen abstracts, retrieve relevant articles, independently score for relevance, and assess for quality. Using this meta-ethnography tool, key analytical metaphors within each study were identified to determine a conceptual model across papers.

Because of the specific requirements of the ethnography process, the proposed system needed to allow and cue for the collection of in excess of 60 qualitative data concepts from multiple articles while allowing for full-text viewing and copying from the articles. The system needed to be easy to use and intuitive for the approximately 10 reviewers who were using a system of novel design for which existing workflows did not exist. The system needed to work on a variety of screen sizes and types (laptops, desktop computers) and allow for the rich display of journal articles including all tables and layout.

To allow for effective and comprehensive analysis of the large amount of qualitative data, a grid-style reporting system was implemented. These grids allowed for the analysis of concepts identified across papers and across reviewers.

Results

This tool is a practical and reliable method for synthesizing reviewer feedback. For example this tool was used to assess 21,489 abstracts across six chronic diseases (cancer, HIV, CVD, asthma, arthritis, diabetes). 1,412 abstracts were selected for full-length review. 58 articles were subjected to quality assessment, of which 25 were included in the meta-ethnography.

Applications

This meta-ethnography tool is a model that is transferable to other healthcare or continuing education research.

Future Directions

This tool will be expanded further to be considered as a potential qualitative research product. It will be possible to allow independent researchers to create projects consisting of articles and reviewers, and to allow them to manage their projects and to follow the review process in real-time using the tool.

Declaration

The authors have no relevant conflicts of interest to declare.