Varying image assessment of pecking injuries in turkeys while performing repetitions

This study investigated variations in assessing potential pecking injuries in turkey hens when annotating image excerpts. Three observers (OBS1, OBS2, OBS3) with different levels of previous knowledge - one with experience in pecking injuries in turkeys and two computer science students - rated a total of 24,912 image excerpts. The image excerpts were evaluated in work packages (2,076 images each) and were classified by the observers as either head injury (HI), skin injury in the feathered area of the body (SI), or no injury (NI). Two observers evaluated three packages (OBS1, OBS2: 6,228 image excerpts each) and OBS3 annnotated six work packages (12,456 excerpts). The percentage of the classifications in the chronological sequence of the observations was analyzed. Inexperienced observers (OBS2 and OBS3) both classified an average of 13% of the shown images as HI, 70% as SI, and 17% as NI. On average, OBS1 classified 12% of the images as HI, 60% as SI, and 28% as NI. Throughout the study, all observers classified more recordings into the NI class. Particularly, OBS1 with the most experience in evaluating pecking injuries showed a different assessment by rating more images (plus 5%) as showing NI over time (OBS2: plus 0.7%; OBS3: plus 2.2%). This result raises questions about whether divergent assessments always occur in repeated judgments and how this effect can be avoided.