Citizen Science has enormous potential to change knowledge production for the better. To realise these promises (or maybe more importantly, to stay realistic regarding its potentials), it is also important to recognise and be aware of its limitations and challenges.
1. Support structures around Citizen Science are not yet fully established
A significant part of these strategic challenges follows from the fact that Citizen Science is still a relatively young, evolving discipline and community of practice (for a historical overview, see Science Europe’s Briefing paper on Citizen Science).
Consequently, there is a need to establish and strengthen its support instruments, for instance,
- to develop innovative financing models that guarantee sustainable support for Citizen Science projects and meet the needs of a range of different stakeholders – for example, although citizen science is included in European research funding projects as an eligible method to fund, many national funders do not have schemes that can accommodate it.
- to strengthen training around Citizen Science both inside and outside of academia; or
- to strengthen synergies between Citizen Science and science communication (source: the Green Paper Citizen Science Strategy 2020 for Germany )
2. Mixed attitudes towards the value of Citizen Science both among researchers and the participants
Some members of the scientific community remain sceptical about the idea that the public can make useful contributions to science (for a detailed discussion, see e.g. Golumbic et al. 2017). Such concerns are partially explained by one of the main methodological challenges of Citizen Science projects, namely, the reliability of and trust towards citizen-generated data. On the other side of the trust gap, the absence of alignment between community priorities and research agendas can easily keep communities away from Citizen Science projects when they find little resonance between the scholarly investments into Citizen Science and their own community priorities. This is especially true for minorities or groups that have been historically underrepresented in science (see Pandya 2012), the lack of whom can, in turn, seriously undermine the diversity potential of expert/non-expert collaborations and can amplify existing biases in science. The tweets below critically reflect on such potential biases in a machine-learning context.
Carefully designed projects and the increasing number of Citizen Science success stories and working practices, as well as the development of support structure around it, have the potential to enhance mutual trust towards Citizen Science, however.
3. Challenges in managing the ethics of CS
Managing a project that involves non-professional researchers can lead to the rise a number of ethically challenging issues. Done badly, citizen science can lead to unintentional exploitation of volunteer labour, misuse of public data and the public itself, a lack of proper credit being given and, in the end , it could unintentionally strengthen the lay–expert barriers that Citizen Science is being promoted to overcome. Meeting these challenges will be discussed in more detail in the ‘Giving credit’ subsection
4. Methodological pitfalls and unexpected challenges when conducting a Citizen Science project
Opening up the research process for large-scale external collaboration with those who are not familiar with the field-specific framework adds extra dimensions and complexities to the project design. Consequently, academic-led Citizen Science projects are much less flexible for incremental changes and improvisations in design and implementation. In contrast to traditional research projects, as a leader of a Citizen Science project, you may not be able to fully control all aspects of its operation such as the realisation of volunteer contributions (NB: as we saw above, this can also be seen as an advantage of Citizen Science endeavours). Thinking and planning ahead however can significantly reduce the chances of ending up with the unexpected difficulties or accidental bias that poses a risk to meeting your research objectives.
The two most frequently reported sets of methodological issues to resolve in the Citizen Science discourse, regardless of disciplines or project types, are a. Diversifying and optimising participation and b. Ensuring data quality.
a) Diversifying and optimising participation
Finding a good balance between diversifying participation and ensuring that applications are suited to the project is not always easy. As Wiggine et al. (2012:301) remarks, “Despite their broader reach, new technologies may inadvertently create barriers that widen the ‘digital divide’ between those adopting/having the technology and those avoiding/lacking it (Ess and Sudweeks 2001). [..] As citizen-science programs adopt new technologies, sensitivity to social, cultural, economic, and political factors will be critical to the success of projects that cross boundaries and involve local/traditional ecological knowledge (Ballard et al. 2008).”
How and where to engage people to your project, how to train them adequately for their task, how to make the tools they need accessible and easy to use, which kind of support they might need during their contributions, how to keep them motivated and how to reward them for their work (e.g. via web-based acknowledgement by posting names or login names online) are fundamental aspects of Citizen Science project design.
b) Ensuring data quality
Ensuring that the methods used and quality of data produced are robust enough to make participants’ contributions valid for research is a universally recognised challenge in Citizen Science projects. In most cases, the “success” or “failure” of a citizen science project will be determined by the degree to which data collected by participants matches with researchers’ standards and expectations (for an extended discussion, see e.g. Gollan 2013). For instance, building bridges between the expert vocabularies used to describe materials in formal institutional settings and community descriptions is a crucial component is in the design of Citizen Science projects in the Cultural Heritage domain. Starting with small-scale pilot projects, the outcome of which can help fine-tuning the study design, establishing well-explained and concise data protocols (e.g. clear guidelines on the orthography of place names in tagging tasks, transcription protocols such as consistent transcription guidelines for abbreviations), providing training and close supervision for the contributors are all assets to (co)produce more reliable and satisfactory crowdsourced data. Still, as Riesch and Potter (2013) reminds us, the simplest guarantee for quality data collection can be ensured at a much earlier step of the research workflow, namely by asking appropriate research questions. By specifically addressing questions that can be answered through relatively simple and accessible surveys regardless of age and ability, that there is little that can go wrong.
Case Study: Challenges and Unexpected Outcomes of the Transcribe Bentham Project
Prof. Melissa Terras (Transcribe Bentham project) discusses the challenges and unexpected outcomes that have arisen during the Transcribe Bentham project.
To find out more about the Transcribe Bentham project, click here: https://blogs.ucl.ac.uk/transcribe-bentham/
Strategies to enhance data quality and reduce uncertainties in the Citizen-driven Archive Topotek
The aim of the exploreAT! project is to explore, preserve and archive a corpus of Bavarian dialect samples (DBÖ) collected between 1911 and 1998 in the region of the Austro-Hungarian Empire and open it up for analysis and insight in different contexts such as cultural studies or collaborative multilingual explorations via Linked Open Data technology.
One of the community groups connected to the exploreAt! project is the citizen-driven archive Topothek, a network of Wikipedia-like local history collections collecting digitised images/photos, audio, video material and texts about the history of the particular place and its locals. The collections are made searchable and interconnectable with other databases via community tagging. These tags are instrumental as they form a tissue and enable interlinking real-world objects contained in the Topothek collections with the language data and thus enrich the corpus to obtain a more inclusive view on the rural world as portrayed in the data.
A recently published article offers interesting insights into the challenges in using community tags as connection points between the Topothek and other sources such as the DBÖ language data and also introduced the use of new technologies such as computer vision to overcome some of these identified problems. A major factor that affected the tagging quality was a general one, namely, the varying quality and precision of the tags. The different degrees of specificity in community tags however also uncovered another challenge having to do with the difference in what is interesting and relevant for the citizens curating the Topothek database and what is relevant for researchers and archivists. For instance, tagging simple objects shown on the pictures (e.g. animal, fruit etc.) is crucial for finding things within the archive. By contrast, community-generated tags tended to contain much more precise information on places, roles of persons or individual names but these, lacking further information or context, were not always easy to meaningfully integrate into the sake of analysis or keep them to the interest of the broader community.
As a partial solution to these problems, the project designers introduced a range of easy-to-use, commercial computer vision tools that could be harnessed both by citizen scientists and archivists or researchers to align different levels of specificity, to overcome uncertainties in the data generated by tagging and also to better align the needs of different stakeholder groups. You can read more about the project and success rate of using computer vision tools here.
- Scassa and Chung, 2015: Best practices for managing intellectual property rights in citizen science: A guide for researchers and citizen scientists https://www.wilsoncenter.org/sites/default/files/research_brief_guide_for_researchers.pdf
- Riesch and Potter 2013: Citizen science as seen by scientists: Methodological, epistemological and ethical dimensions https://journals.sagepub.com/doi/abs/10.1177/0963662513497324?casa_token=Ide0u0GFUJwAAAAA%3AOkcaZAtLxWEPVEX4m9see0K72uLlq0KmySEpGXxdzBJW6zaPlWjbwblqcH9FtHuYpWAIR_6aY0-r
- A roadmap for broadening e-Infrastructure deployment to support the participation of citizens in Humanities and Heritage Science projects: CIVIC EPISTEMOLOGIES – Development of a Roadmap for Citizen Researchers in the Age of Digital Culture https://www.civic-epistemologies.eu/