Skip to content

Review Article

Jump to bottom
Nic edited this page Jun 22, 2020 · 1 revision

The goal of the review article is the following:

  • Update Hunter, J. (2009). Collaborative semantic tagging and annotation systems. Annual review of information science and technology, 43(1), 1-84.
  • Synthesize and summarize empirical studies of annotation
  • Identify gaps in literature

The paper should contain two types of summaries:

  1. Ad-hoc review of technical specs that have emerged to enable broad scale web-annotation.
  2. Systematic review of literature on annotations.

Technical Specs

For the review of technical specs we will need to catalogue and develop a list of specifications we think are important. This will be our own subjective judgement, but should include things like the W3C standard for web annotations, hypothes.is, etc.

Systematic Review

For the review of empirical studies, we need to develop a protocol for searching, selecting, and analyzing publications. If you are new to the world of systematic reviews see my notes below

Notes on systematic reviews:

Systematic reviews follow an established research protocol (see below for examples). These are often the basis for a meta-analysis or for a synthetic study which attempts to reuse data or research findings to answer a specific (summative) research question. The advantage to a systematic review is that it is, in short, replicable – by documenting the databases selected, your search strategy, establishing criteria for relevance (inclusion and exclusion), and answering a specific set of research questions the findings are considered empirically rigorous (even if you’re not engaged in primary data collection). Think of the systematic review as more deductive of the two types. The two protocols for systematic reviews to look at are PRISM and Software Engineering:

  • Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., ... & Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. Bmj, 349, g7647.
  • I like this overview for SE systematic reviews, but there are a lot of good examples if you google around: https://dl.acm.org/citation.cfm?id=1134500

Some examples from Information Science:

  • Kelly, D., & Sugimoto, C. R. (2013). A systematic review of interactive information retrieval evaluation studies, 1967–2006. Journal of the American Society for Information Science and Technology, 64(4), 745-770.
  • Buettner, R. (2015, January). A systematic literature review of crowdsourcing research from a human resource management perspective. In System Sciences (HICSS), 2015 48th Hawaii International Conference on (pp. 4609-4618). IEEE. (unique for how he stratifies his sample based on reported results)
  • Malhotra, R. (2015). A systematic review of machine learning techniques for software fault prediction. Applied Soft Computing, 27, 504-518. (this is just a really smart idea on how to use low hanging fruit for an impactful systematic review)

Clone this wiki locally