Abstract

When experimental philosophers carry out studies on thought experiments, some participants are excluded based on certain exclusion criteria, mirroring standard social science vignette methodology. This involves excluding people that do not pay attention or who miscomprehend the scenario presented in thought experiments. However, experimental philosophy studies sometimes exclude an alarmingly high number of participants. We argue that this threatens the external and internal validity of the conclusions being drawn and we show how a simple visualization of thought experiments can reduce exclusion rates significantly. Furthermore, we argue that focus should not merely be on how many are excluded, but also why they are excluded, and we highlight the role of comprehension questions in this regard. Philosophical thought experiments often rely on the acceptance of certain key premises that may be regarded contestable, and asking comprehension questions involving such key assumptions could be problematic as that may result in some participants being inadvertently excluded from the study, potentially creating a selection bias.