Research on spatial thinking requires reliable and valid measures of individual differences in various component skills. Spatial perspective taking (PT)—the ability to represent viewpoints different from one's own—is one kind of spatial skill that is especially relevant to navigation. This study had two goals. First, the psychometric properties of four PT tests were examined: Four Mountains Task (FMT), Spatial Orientation Task (SOT), Perspective-Taking Task for Adults (PTT-A), and Photographic Perspective-Taking Task (PPTT). Using item response theory (IRT), (...) class='Hi'>item difficulty, discriminability, and efficiency of item information functions were evaluated. Second, the relation of PT scores to general intelligence, working memory, and mental rotation (MR) was assessed. All tasks showed good construct validity except for FMT. PPTT tapped a wide range of PT ability, with maximum measurement precision at average ability. PTT-A captured a lower range of ability. Although SOT contributed less measurement information than other tasks, it did well across a wide range of PT ability. After controlling for general intelligence and working memory, original and IRT-refined versions of PT tasks were each related to MR. PTT-A and PPTT showed relatively more divergent validity from MR than SOT. Tests of dimensionality indicated that PT tasks share one common PT dimension, with secondary task-specific factors also impacting the measurement of individual differences in performance. Advantages and disadvantages of a hybrid PT test that includes a combination of items across tasks are discussed. (shrink)

Using an Item response theory (IRT) analysis, this study examined ethnic and gender groups differences in exposure to content material (i.e. access to curriculum) assessed on the 12th grade NAEP US History 2010 exam. Employing multi-step data analysis procedures, authors examined race and gender using the NAEP Item Mapping Tool available through NCES. Results revealed item-level patterns, which suggest that females and Black students are more likely to answer questions, related to social history, particularly the (...) Civil Rights, when accounting for student achievement. While instructional exposure matters, how students of similar performance differentiate themselves on item response is related to demographics. These findings have substantial implications for social studies, which include the need to develop a more democratic curriculum that reflects the cultural identities of students in order to make history and history knowledge meaningful. Additionally, results have implications for tests. Assessments should be mindful of cultural validity of items and become more inclusive; perhaps aligning with a more culturally relevant curriculum. (shrink)

Multilevel item response theory models are used widely in educational and psychological research. This type of modeling has two or more levels, including an item response theory model as the measurement part and a linear-regression model as the structural part, the aim being to investigate the relation between explanatory variables and latent variables. However, the linear-regression structural model focuses on the relation between explanatory variables and latent variables, which is only from the perspective of (...) the average tendency. When we need to explore the relationship between variables at various locations along the response distribution, quantile regression is more appropriate. To this end, a quantile-regression-type structural model named as the quantile MLIRT model is introduced under the MLIRT framework. The parameters of the proposed model are estimated using the Gibbs sampling algorithm, and comparison with the original MLIRT model is conducted via a simulation study. The results show that the parameters of the Q-MLIRT model could be recovered well under different quantiles. Finally, a subset of data from PISA 2018 is analyzed to illustrate the application of the proposed model. (shrink)

Applying the graded response model within the item response theory framework, the present study analyzes the psychometric properties of Karwowski’s creative self-efficacy scale. With an ethnically diverse sample of US college students, the results suggested that the six items of the CSE scale were well fitted to a latent unidimensional structure. The scale also had adequate measurement precision or reliability, high levels of item discrimination, and an appropriate range of item difficulty. Gender-based differential (...) class='Hi'>item functioning analyses confirmed that there were no differences in the measurement results of the scale concerning gender. Additionally, openness to experience was found to be positively related to the CSE scale scores, providing some support for the scale’s convergent validity. Collectively, these results confirmed the psychometric soundness of the CSE scale for measuring CSE and also identified avenues for future research. (shrink)

This study develops a short Creative Expression Interest Scale among Chinese freshmen based on the perspective of item response theory. Nine hundred fifty-nine valid Chinese freshmen participated in the Creative Expression Interest survey. Researchers applied the initial data for unidimensionality, item fit, discrimination parameter, and differential item functioning to obtain a short CEIS. The results show that the Short CEIS meets the psychometric requirements of the IRT. Pearson correlation coefficient of theta between the short and (...) long CEIS is 0.922. The marginal reliability of the short CEIS is 0.799. These indicate that the short CEIS developed in this study among Chinese freshmen, meets the psychometric requirements. Although the Short CEIS can eliminate redundant, uninformative items, save time, and improve the quality of data collection. However, the validity of this short scale needs further validation. (shrink)

Insight is a construct carried out into different theoretical orientations with increasing application out of the boundaries of clinical psychology. Recent studies have investigated insight also as a promising variable for organizational outcomes. Given the relevance of Insight in promoting change, this paper aimed at describing the psychometric analysis of one of the shortest, most agile, and most versatile tool for measuring some of the characteristics of insight, the Insight Orientation Scale, using Item Response Theory. To achieve (...) this goal, we applied a Mixed Rash Model to the IOS. Data from 1,445 individuals were analyzed by the means of WIN-MIRA and Multilog. Based on the likelihood statistics we assumed a three-class solution for the IOS. Results also indicated that the greater part of items had good discrimination and threshold parameters. These findings confirmed psychometric stability of the IOS highlighting its measurement precision, supporting its utility in both research and practice. (shrink)

Educational assessments tests are often constructed using testlets because of the flexibility to test various aspects of the cognitive activities and broad content sampling. However, the violation of the local item independence assumption is inevitable when tests are built using testlet items. In this study, simulations are conducted to evaluate the performance of item response theory models and testlet response theory models for both the dichotomous and polytomous items in the context of equating tests (...) composed of testlets. We also examine the impact of testlet effect, length of testlet items, and sample size on estimating item and person parameters. The results show that more accurate performance of testlet response theory models over item response theory models was consistently observed across the studies, which supports the benefits of using the testlet response theory models in equating for tests composed of testlets. Further, results of the study indicate that when sample size is large, item response theory models performed similarly to testlet response theory models across all studies. (shrink)

This study explored calibrating a large item bank for use in multidimensional health measurement with computerized adaptive testing, using both item responses and response time (RT) information. The Activity Measure for Post-Acute Care is a patient-reported outcomes measure comprised of three correlated scales (Applied Cognition, Daily Activities, and Mobility). All items from each scale are Likert type, so that a respondent chooses a response from an ordered set of four response options. The most appropriate (...) class='Hi'>item response theory model for analyzing and scoring these items is the multidimensional graded response model (MGRM). During the field testing of the items, an interviewer read each item to a patient and recorded, on a tablet computer, the patient’s responses and the software recorded RTs. Due to the large item bank with over 300 items, data collection was conducted in four batches with a common set of anchor items to link the scale. Van der Linden’s (2007) hierarchical modeling framework was adopted. Several models, with or without interviewer as a covariate and with or without interaction between interviewer and items, were compared for each batch of data. It was found that the model with the interaction between interviewer and item, when the interaction effect was constrained to be proportional, fit the data best. Therefore, the final hierarchical model with lognormal model for RT and the MGRM for response data was fitted to all batches of data via a concurrent calibration. Evaluation of parameter estimates revealed that (1) adding response time information did not affect the item parameter estimates and their standard errors significantly; (2) adding response time information helped reduce the standard error of patients’ multidimensional latent trait estimates, but adding interviewer as a covariate did not result in further improvement. Implications of the findings for follow up adaptive test delivery design are discussed. (shrink)

Item response theory (IRT) is a modern psychometric framework for estimating respondents’ latent traits (e.g., ability, attitude, and personality) based on their responses to a set of questions in psychological tests. The current study adopted an item response tree (IRTree) method, which combines the tree model with IRT models for handling the sequential process of responding to a test item, to score individual moral inclination for the CNI test—a broadly adopted model for examining humans’ (...) moral decision-making with three parameters generated: sensitivity to moral norms, sensitivity to consequences, and inaction preference. Compared to previous models for the CNI test, the resulting EIRTree-CNI Model is able to generate individual scores without increasing the number of items (thus, less subject fatigue or compromised response quality) or employing a post hoc approach that is deemed statistically suboptimal. The model fits the data well, and the subsequent test also supported the concurrent validity and the predictive validity of the model. Limitations are discussed further. (shrink)

Although construct measurement is critical to explanatory research and intervention efforts, rigorous measure development remains a notable challenge. For example, though the primary theoretical model for understanding health disparities among sexual minority (e.g., lesbian, gay, bisexual) adolescents is minority stress theory, nearly all published studies of this population rely on minority stress measures with poor psychometric properties and development procedures. In response, we developed the Sexual Minority Adolescent Stress Inventory (SMASI) with N = 346 diverse adolescents ages 14–17, (...) using a comprehensive approach to de novo measure development designed to produce a measure with desirable psychometric properties. After exploratory factor analysis on 102 candidate items informed by a modified Delphi process, we applied item response theory techniques to the remaining 72 items. Discrimination and difficulty parameters and item characteristic curves were estimated overall, within each of 12 initially derived factors, and across demographic subgroups. Two items were removed for excessive discrimination and three were removed following reliability analysis. The measure demonstrated configural and scalar invariance for gender and age; a three-item factor was excluded for demonstrating substantial differences by sexual identity and race/ethnicity. The final 64-item measure comprised 11 subscales and demonstrated excellent overall (α = 0.98), subscale (α range 0.75–0.96), and test–retest (scale r > 0.99; subscale r range 0.89–0.99) reliabilities. Subscales represented a mix of proximal and distal stressors, including domains of internalized homonegativity, identity management, intersectionality, and negative expectancies (proximal) and social marginalization, family rejection, homonegative climate, homonegative communication, negative disclosure experiences, religion, and work domains (distal). Thus, the SMASI development process illustrates a method to incorporate information from multiple sources, including item response theory models, to guide item selection in building a psychometrically sound measure. We posit that similar methods can be used to improve construct measurement across all areas of psychological research, particularly in areas where a strong theoretical framework exists but existing measures are limited. (shrink)

This article introduces a new hybrid intake procedure developed for posttraumatic stress disorder (PTSD) screening, which combines an automated textual assessment of respondents’ self-narratives and item-based measures that are administered consequently. Text mining technique and item response modeling were used to analyze long constructed response (i.e., self-narratives) and responses to standardized questionnaires (i.e., multiple choices), respectively. The whole procedure is combined in a Bayesian framework where the textual assessment functions as prior information for the estimation of (...) the PTSD latent trait. The purpose of this study is twofold: first, to investigate whether the combination model of textual analysis and item-based scaling could enhance the classification accuracy of PTSD, and second, to examine whether the standard error of estimates could be reduced through the use of the narrative as a sort of routing test. With the sample at hand, the combination model resulted in a reduction in the misclassification rate, as well as a decrease of standard error of latent trait estimation. These findings highlight the benefits of combining textual assessment and item-based measures in a psychiatric screening process. We conclude that the hybrid test design is a promising approach to increase test efficiency and is expected to be applicable in a broader scope of educational and psychological measurement in the future. (shrink)

Colonius suggests that, in using standard set theory as the language in which to express our computational-level theory of human memory, we would need to violate the axiom of foundation in order to express meaningful memory bindings in which a context is identical to an item in the list. We circumvent Colonius's objection by allowing that a list item may serve as a label for a context without being identical to that context. This debate serves to (...) highlight the value of specifying memory operations in set theoretic notation, as it would have been difficult if not impossible to formulate such an objection at the algorithmic level. (shrink)

As a branch of statistical latent variable modeling, multidimensional item response theory plays an important role in psychometrics. Multidimensional graded response model is a key model for the development of multidimensional computerized adaptive testing with graded-response data and multiple traits. This paper explores how to automatically identify the item-trait patterns of replenished items based on the MGRM in MCAT. The problem is solved by developing an exploratory pattern recognition method for graded-response items based (...) on the least absolute shrinkage and selection operator, which is named LPRM-GR and facilitates the subsequent parameter estimation of replenished items and helps maintaining the effectiveness of item replenishment in MCAT. In conjunction with the proposed approach, the regular BIC and weighted BIC are applied, respectively, to select the optimal item-trait patterns. Simulation for evaluating the LPRM-GR in pattern recognition accuracy of replenished items and the corresponding item estimation accuracy is conducted under multiple conditions across different numbers with respect to dimensionality, response-category numbers, latent trait correlation, stopping rules, and item selection criteria. Results show that the proposed method with the two types of BIC both have good performance in pattern recognition for item replenishment in the two- to four-dimensional MCAT with the MGRM, for which the weighted BIC is generally superior to the regular BIC. The proposed method has relatively high accuracy and efficiency in identifying the patterns of graded-response items, and has the advantages of easy implementation and practical feasibility. (shrink)

Design is concerned with the ways in which we deliberately seek to arrange, organize, or structure things. From a philosophy of design perspective, the practice of design raises fundamental questions about the basic ways in which things can and should be organized. I advance a tripartite schema of the basic ways in which things (anything at all) can be organized and offer a triangular model of the “organization space” or “design space” they define. I refer to these three basic forms (...) of organization as “responsive cohesion,” “fixed cohesion,” and “discohesion,” and offer three reasons why responsively cohesive forms of organization are more valuable than the other two; indeed, the other two forms of organization are typically disvaluable. Beyond focusing on the value of individual instances of responsive cohesion, I further consider the fact that every responsively cohesive item exists within a wider context(s), which may itself tend more towards fixed cohesion, responsive cohesion, or discohesion. This raises a number of further issues; for example, what should we do if a responsively cohesive item clashes with – is discohesive with – its responsively cohesive context? I advance a normative theory of contexts to sort out these kinds of issues. In the context of this discussion, I briefly consider a range of other ideas that bear a family resemblance to the idea of responsive cohesion and indicate, equally briefly, why the theory of responsive cohesion approach is superior to these other approaches. I conclude with some guidance on how we can implement the ideas advanced here (“we” because we all design things in our own way) and then, more specifically, on the implications of these ideas for the professional designer-client relationship. (shrink)

There are a number of key data-centric questions that must be answered when developing classifiers for operator functional states. “Should a supervised or unsupervised learning approach be used? What degree of labeling and transformation must be performed on the data? What are the trade-offs between algorithm flexibility and model interpretability, as generally these features are at odds?” Here, we focus exclusively on the labeling of cognitive load data for supervised learning. We explored three methods of labeling cognitive states for three-state (...) classification. The first method labels states derived from a tertiary split of trial difficulty during a spatial memory task. The second method was more adaptive; it employed a mixed-effects stress–strain curve and estimated an individual’s performance asymptotes with respect to the same spatial memory task. The final method was similar to the second approach; however, it employed a mixed-effects Rasch model to estimate individual capacity limits within the context of item response theory for the spatial memory task. To assess the strength of each of these labeling approaches, we compared the area under the curve (AUC) for receiver operating curves (ROCs) from elastic net and random forest classifiers. We chose these classifiers based on a combination of interpretability, flexibility, and past modeling success. We tested these techniques across two groups of individuals and two tasks to test the effects of different labeling techniques on cross-person and cross-task transfer. Overall, we observed that the Rasch model labeling paired with a random forest classifier led to the best model fits and showed evidence of both cross-person and cross-task transfer. (shrink)

Item context effects refer to the impact of features of a test on an examinee’s item responses. These effects cannot be explained by the abilities measured by the test. Investigations typically focus on only a single type of item context effects, such as item position effects, or mode effects, thereby ignoring the fact that different item context effects might operate simultaneously. In this study, two different types of context effects were modeled simultaneously drawing on data (...) from an item calibration study of a multidimensional computerized test (N = 1,632) assessing student competencies in mathematics, science, and reading. We present a generalized linear mixed model (GLMM) parameterization of the multidimensional Rasch model including item position effects (distinguishing between within-block position effects and block position effects), domain order effects, and the interactions between them. Results show that both types of context effects played a role, and that the moderating effect of domain orders was very strong. The findings have direct consequences for planning and applying mixed domain assessment designs. (shrink)

In the context of item response theory, linking the scales of two measurement points is a prerequisite to examine a change in competence over time. In educational large-scale assessments, non-identical test forms sharing a number of anchor-items are frequently scaled and linked using two− or three-parametric item response models. However, if item pools are limited and/or sample sizes are small to medium, the sparser Rasch model is a suitable alternative regarding the precision of parameter (...) estimation. As the Rasch model implies stricter assumptions about the response process, a violation of these assumptions may manifest as model misfit in form of item discrimination parameters empirically deviating from their fixed value of one. The present simulation study investigated the performance of four IRT linking methods—fixed parameter calibration, mean/mean linking, weighted mean/mean linking, and concurrent calibration—applied to Rasch-scaled data with a small item pool. Moreover, the number of anchor items required in the absence/presence of moderate model misfit was investigated in small to medium sample sizes. Effects on the link outcome were operationalized as bias, relative bias, and root mean square error of the estimated sample mean and variance of the latent variable. In the light of this limited context, concurrent calibration had substantial convergence issues, while the other methods resulted in an overall satisfying and similar parameter recovery—even in the presence of moderate model misfit. Our findings suggest that in case of model misfit, the share of anchor items should exceed 20% as is currently proposed in the literature. Future studies should further investigate the effects of anchor item composition regarding unbalanced model misfit. (shrink)

The increasing digitalization in the field of psychological and educational testing opens up new opportunities to innovate assessments in many respects (e.g., new item formats, flexible test assembly, efficient data handling). In particular, computerized adaptive testing provides the opportunity to make tests more individualized and more efficient. The newly developed continuous calibration strategy (CCS) from Fink, Born, Spoden, and Frey (2018) makes it possible to construct computerized adaptive tests in application areas where separate calibration studies are not feasible. Due (...) to the goal of reporting on a common metric across test cycles, the equating is crucial for the CCS. The quality of the equating depends on the common items selected and the scale transformation method applied. Given the novelty of the CCS, the aim of the study was to evaluate different equating setups in the CCS and to derive practical recommendations. The impact of different equating setups on the precision of item parameter estimates and on the quality of the equating was examined in a Monte Carlo simulation, based on a fully crossed design with the factors common item difficulty distribution (bimodal, normal, uniform), scale transformation method (mean/mean, mean/sigma, Haebara, Stocking-Lord), and sample size per test cycle (50, 100, 300). The quality of the equating was operationalized by three criteria (proportion of feasible equatings, proportion of drifted items, and error of transformation constants). The precision of the item parameter estimates increased with increasing sample size per test cycle, but no substantial difference was found with respect to the common item difficulty distribution and the scale transformation method. With regard to the feasibility of the equatings, no differences were found for the different scale transformation methods. However, when using the moment methods (mean/mean, mean/sigma), quite extrem levels of error for the transformation constants A and B occurred. Among the characteristic curve method the performance of the Stocking-Lord method was slightly better than for the Haebara method. Thus, while no clear recommendation can be made with regard to the common item difficulty distribution, the characteristic curve methods turned out to be the most favorable scale transformation methods within the CCS. (shrink)

Implicit theories refer to assumptions people hold about different domains, also known as mindsets. There are two implicit theories on the malleability of one’s ability: entity theory and incremental theory. They constrain and regulate people’s understanding and responses to an individual’s behavior, leading to different social cognitive patterns and behavioral responses. Resilience is a positive adaptation in highly stressful situations that represents mechanisms for coping with and transcending difficult experiences, i.e., a person’s ability to successfully adapt to change, (...) resist the adverse effects of stressors, avoid significant dysfunction, and be chronically affected by considered a protective factor for mental health. Although previous studies showed that individuals’ implicit theories are associated with resilience, this relationship has received little attention in the nursing population. It is unclear which variables may contribute to explaining the relationship between implicit theories and resilience. Therefore, the current study aims to deeply explore the relationship between implicit theories and the resilience of Chinese nurses. In addition, we also seek to demonstrate the chain mediating effects of grit and meaning in life on this relationship. We surveyed 709 Chinese nurses through online questionnaires using the self-made demographic questionnaire, the Implicit Theories Scale, the Short Grit Scale, the Meaning in Life Questionnaire, and the 10-item Connor-Davidson Resilience Scale. After controlling for demographic variables such as age, gender, educational background, marital status, professional title, and working years, the results reveal positive associations between Chinese nurses’ implicit theories and their resilience, and grit and meaning in life play a partial mediating role in this relationship, respectively. Furthermore, grit and meaning in life play a chain mediating role between implicit theories and resilience. These findings contribute to understanding the psychological impact mechanism of implicit theories on nurses’ resilience and provide a theoretical basis for nursing managers to formulate strategies to improve nurses’ psychological resilience. (shrink)

An extension to a rating system for tracking the evolution of parameters over time using continuous variables is introduced. The proposed rating system assumes a distribution for the continuous responses, which is agnostic to the origin of the continuous scores and thus can be used for applications as varied as continuous scores obtained from language testing to scores derived from accuracy and response time from elementary arithmetic learning systems. Large-scale, high-stakes, online, anywhere anytime learning and testing inherently comes with (...) a number of unique problems that require new psychometric solutions. These include (1) the cold start problem, (2) problem of change, and (3) the problem of personalization and adaptation. We outline how our proposed method addresses each of these problems. Three simulations are carried out to demonstrate the utility of the proposed rating system. (shrink)

Mixture item response theory models include a mixture of latent subpopulations such that there are qualitative differences between subgroups but within each subpopulation the measure model based on a continuous latent variable holds. Under this modeling framework, students can be characterized by both their location on a continuous latent variable and by their latent class membership according to Students’ responses. It is important to identify anchor items for constructing a common scale between latent classes beforehand under the (...) mixture IRT framework. Then, all model parameters across latent classes can be estimated on the common scale. In the study, we proposed Q-matrix anchored mixture Rasch model, including a Q-matrix and the traditional mixture Rasch model. The Q-matrix in QAMRM can use class invariant items to place all model parameter estimates from different latent classes on a common scale regardless of the ability distribution. A simulation study was conducted, and it was found that the estimated parameters of the QAMRM recovered fairly well. A real dataset from the Certificate of Proficiency in English was analyzed with the QAMRM, LCDM. It was found the QAMRM outperformed the LCDM in terms of model fit indices. (shrink)

When it comes to determining how healthcare resources should be allocated, there are many factors that could—and perhaps should—be taken into account. One such factor is a patient’s responsibility for his or her illness, or for the behavior that caused it. Policies that take responsibility for the unhealthy lifestyle or its outcomes into account—responsibility-sensitive policies—have faced a series of criticisms. One holds that agents often fail to meet either the control or epistemic conditions on responsibility with regard to their unhealthy (...) lifestyles or their outcomes. Another holds that even if patients sometimes are responsible for these items, we cannot know whether a particular patient is responsible for them. In this article, we propose a type of responsibility-sensitive policy that may be able to surmount these difficulties. Under this type of policy, patients are empowered to change to a healthier lifestyle by being given what we call a ‘Golden Opportunity’ to change. Such a policy would not only avoid concerns about patients’ fulfilment of conditions on responsibility for their lifestyles, it would also allow healthcare authorities to be justified in believing that a patient who does not change her lifestyle is responsible for the unhealthy lifestyle. We conclude with a discussion of avenues for further work, and place this policy in the broader context of the debate on responsibility for health. (shrink)

The Musical Emotion Discrimination Task (MEDT) is a short, non-adaptive test of the ability to discriminate emotions in music. Test-takers hear two performances of the same melody, both played by the same performer but each trying to communicate a different basic emotion, and are asked to determine which one is “happier”, for example. The goal of the current study was to construct a new version of the MEDT using a larger set of shorter, more diverse music clips and an adaptive (...) framework to expand the ability range for which the test can deliver measurements. The first study analysed responses from a large sample of participants (N = 624) to determine how musical features contributed to item difficulty, which resulted in a quantitative model of musical emotion discrimination ability rooted in Item Response Theory (IRT). This model informed the construction of the adaptive MEDT. A second study contributed preliminary evidence for the validity and reliability of the adaptive MEDT, and demonstrated that the new version of the test is suitable for a wider range of abilities. This paper therefore presents the first adaptive musical emotion discrimination test, a new resource for investigating emotion processing which is freely available for research use. (shrink)

This paper signifies the inevitability of the responsible practices of leadership into the relation with the subordinates in the Indian organizations. The theme of this study revolves around the responsibility and ethical approach of the top management which directly or indirectly influence the job-related attitudinal behavior of employees. To analyze it empirically, a questionnaire is designed and 138 middle-level managers from four private sector telecom companies in India rated their superiors on 11 items of leadership scale. This study used internal (...) consistency reliability, Pearson correlation, and exploratory factor analysis as statistical techniques. Factor analysis of the scores on such items yielded two dimensions. The paper examines the impact of leadership of managers upon their subordinates in the context of enhancing job satisfaction, affective commitment, and knowledge performance. The results of the study are not completely substantiating empirically to the theory that ethical attitude of leaders enhance commitment and satisfaction of subordinates in Indian context. (shrink)

This investigation examined how people’s beliefs about the functionality of emotion shape their emotional response and regulatory strategies when encountering distressing events. In Study 1, we present data supporting the reliability and validity of an 8-item instrument, the Help and Hinder Theories about Emotion Measure (HHTEM), designed to assess an individual’s beliefs about the functionality of emotion. Participants who more strongly endorsed a Help Theory reported greater wellbeing, emotional acceptance, and use of reappraisal to regulate emotion. Participants (...) who more strongly endorsed a Hinder Theory reported less wellbeing and more expressive suppression and substance use. In Study 2, we demonstrate that encouraging participants to view emotion as helpful affected their physiological and regulatory response to a distressing event. Participants in the Help Theory condition showed greater physiological reactivity (SCL) during a distressing film than control participants but were more accepting of their emotional response. Shortly after the film, SCL decreased for participants in the Help Theory condition. Compared to control participants, they engaged in less suppression and reported less lingering effect of the film on their mood. Together, these studies suggest that people’s theories about the functionality of emotion influence their reactivity, the strategies they adopt to regulate emotion, and their ability to rebound after distressing events. (shrink)

In “Conceptual Differences Between Children and Adults,” Susan Carey discusses phlogiston theory in order to defend the view that there can be non-translatability between scientific languages. I present an objection to her defence.

In lieu of an abstract, here is a brief excerpt of the content:RESPONSE TO BRUCE MARSHALL GEORGE LINDBECK There is an abundance of il'iches in Bruce Marshalrs essay. He makes me understand hoth myse1f and Aquinas hetter than I biaid done hefore; and, interestingly, it is chiefly hy his exegesis of St. Thomas that he does :bhis. If I had referred more to the Thomistic ideas he elucidates when I wirus writing Natrure of Dootrine1 it would have!been a better (...) hook. What he calls the " somewhat notorious " example of the crusadeT"s Christus est Dominus is a good oase in point. It would ihave helped if I had made dear that I was thinking in medieval [ashion of a an individual rutterance, the product of a pa•riticular :second 1aJct of the intellect, the ·acl of composing and dividing, of judging ·such and such rto ibe rthe ca,se. In Aquinas' intellectual setting, judgments, not sentences in ·abstraiction from 1 acls of affirmation, were propositions capruble of being true or f.a1se. Many of my ·reruders had a more modern or "Platonic " understanding of rpropositions, and therefore missed the force of the e:xJam:ple ·as Marshall so carefully and rightly explains it. Among his contributions, the major one, however, is system- •a1tically to introduce into the disCUJssion the distinction between the " truth " and " justification " of beliefs or pI'oposi- "'" tionS.- Once the point is made, it is evident that " alethiology" and " epistemology " (to mention a cognate, though not ~den­ tical distinction) ·are, 1 at least in some contexts, partially independent variables. There is no one-to-one relation between different meanings or theories of truth and the v:arious views as to how we know such ·and such is true. It was my failure to make this point explicit which confosed Fr. Colman O'Neill, of blessed memory, ias well as a good many other readers (1as Bruce Marshall quite rightly notes, though, with excessive 408 404 GEORGEl LINDBECK kindness to me, he!blames the,readers rather than the 'author for the misunderstanding). Once iclarfiied, as it has lbeen done hy Mr. Marshall, the issue turns on ·whether,a classical " correspondence " theory of truth can be ioombined,with, to empfoy O'Neill's terminology, the use of "coherentist" and " pragmatist " epistemological criteria in justifying rbelieis. I ·am not sure ·that this is possible for those who e:xdude any reference to an idea.I observer or knower (whether real or hypothetical) when defining truth, hut for any theist r.for whom God is prima veritas, as he was for Aquinas, the answer is dearly in the affirmative. In God, and only in God, are knowledge and reality, not only in correspondence, hut directly known to correspond. Only in him do truth·and knowledge of truth, 1 aleth:iofogy and epistemology, ooindde. In 1human knowledge in via, in contrast, there is always a gap. Our ibe1ieTs may correspond to reality, hut we are justified in holding that they do so, not by directly seeing the correspondence, 1but by some other means. That those other means might in part or w:hole be coherentist or pragmatist cannot :be e:xduded a priori. It is true that in the caise of an Aristotelian 1such 1as Aquinas, coherence •and rpmctise 1are not explicitly accorded major roles in the epistemological justification of natural kno·wledge or scientia, yet even here they are not e:xduded. Nothing can quali[y ais 1an item of knowledge unless it coheres with all other scientia, and right pra.ctice (i.e., training in virtue) is indispensible :llor ethical kno,wledge. Nor need one cease fo be an Aris,totelian, 1as far ·as I can see, if one accepts the contempOO 'ary oommonpJaice that sense experience itself is heavily dependent on linguistic and non-linguistic pmctices which are in part ·acquired rthrough variaible forim:s of aicculturation and not simply through the actualization of 1genetically encoded propensities. One could still, despite this increased emphasis on prwctiice.and on coherence 1 with webs of 1belief, ibe 'able to affum in good Aristotelian,fashion tha;t the knowledge naturally accessible to ra:t]onal animals such,as we a.re is primarily justified RESPONCE TO BRUCE MARSHALL 405 iby reference to... (shrink)

This article contributes to the debate around the abilities of large language models such as GPT-3, dealing with: firstly, evaluating how well GPT does in the Turing Test, secondly the limits of such models, especially their tendency to generate falsehoods, and thirdly the social consequences of the problems these models have with truth-telling. We start by formalising the recently proposed notion of reversible questions, which Floridi & Chiriatti propose allow one to ‘identify the nature of the source of their answers’, (...) as a probabilistic measure based on Item Response Theory from psychometrics. Following a critical assessment of the methodology which led previous scholars to dismiss GPT’s abilities, we argue against claims that GPT-3 completely lacks semantic ability. Using ideas of compression, priming, distributional semantics and semantic webs we offer our own theory of the limits of large language models like GPT-3, and argue that GPT can competently engage in various semantic tasks. The real reason GPT’s answers seem senseless being that truth-telling is not amongst them. We claim that these kinds of models cannot be forced into producing only true continuation, but rather to maximise their objective function they strategize to be plausible instead of truthful. This, we moreover claim, can hijack our intuitive capacity to evaluate the accuracy of its outputs. Finally, we show how this analysis predicts that a widespread adoption of language generators as tools for writing could result in permanent pollution of our informational ecosystem with massive amounts of very plausible but often untrue texts. (shrink)

There are two parts to this present article. First, I consider the applied problem of assessing changes in a congregation—how often to assess, why assess, what changes are looked for, what interventions are assessed, why look to the components of interventions, and how to avoid problems interpreting your data. Brief assessments are mandatory. Pastors and lay leaders can make simple but sound assessments of their spiritual formation programs using item response theory, goal attainment scales, single items, parceling (...) single items, or narrative responses as five alternatives. I describe dangers in interpreting the findings such as not knowing the base rate of change and mistaking the assessment for the mission. In the second part of the article, I briefly describe a program labeled a deep dive into humility. I summarize the assessments that were done, which gives a practical example of the technical recommendations I made earlier in the article. (shrink)

This paper discusses a series of important methodological issues in developing targeted health-related quality of life measures in studies of the effects of medical interventions. Such measures cannot be developed unless the evaluator understands the life domains that medical interventions affect. Qualitative discovery methods are needed to obtain this understanding. Once domains are targeted for measurement, careful and systematic laboratory pilot work should be used to select initial scale items. Psychometric evaluation of response patterns in subsequent field tests is (...) needed to assess the measures. Less concern should be directed to internal consistency reliability of scales in the psychometric evaluation and more to the ability of short scales to reproduce total scale variance and to provide precise measurement within the range of the outcome where effects are expected. The paper closes with a discussion of modern methods of item response scaling that can be used to address these issues. (shrink)

To prevent ordinary people from being harmed by natural language processing (NLP) technology, finding ways to measure the extent to which a language model is biased (e.g., regarding gender) has become an active area of research. One popular class of NLP bias measures are bias benchmark datasets—collections of test items that are meant to assess a language model’s preference for stereotypical versus non-stereotypical language. In this paper, we argue that such bias benchmarks should be assessed with models from the psychometric (...) framework of item response theory (IRT). Specifically, we tie an introduction to basic IRT concepts and models with a discussion of how they could be relevant to the evaluation, interpretation and improvement of bias benchmark datasets. Regarding evaluation, IRT provides us with methodological tools for assessing the quality of both individual test items (e.g., the extent to which an item can differentiate highly biased from less biased language models) as well as benchmarks as a whole (e.g., the extent to which the benchmark allows us to assess not only severe but also subtle levels of model bias). Through such diagnostic tools, the quality of benchmark datasets could be improved, for example by deleting or reworking poorly performing items. Finally, in regards to interpretation, we argue that IRT models’ estimates for language model bias are conceptually superior to traditional accuracy-based evaluation metrics, as the former take into account more information than just whether or not a language model provided a biased response. (shrink)

The factor structure of the Multidimensional Ethics Scale (MES; Reidenbach and Robin: 1988, Journal of Business Ethics 7, 871–879; 1990, Journal of Business Ethics 9, 639–653) was examined for the 8-item short form (N = 328) and the original 30-item pool (N = 260). The objectives of the study were: to verify the dimensionality of the MES; to increase the amount of true cross-scenario variance through the use of 18 scenarios varying in moral intensity (Jones: 1991, Academy of (...) Management Review 16, 366–395); and, to examine the items for measurement precision using item-response theory (IRT) methods. Results of confirmatory and exploratory factor analysis failed to conclusively support the hypothesized 3- (short form) or 5-factor (long form) structure; both instruments were instead dominated by a general factor. Item response theory analyses using Samejima’s (1969, Psychometrika Monograph Supplement 34, (4, Pt. 2)) graded response model revealed that many items in the 30-item pool performed very well, and suggested that a different collection of items be used to form a short-form version of the MES. Our proposed 10-item instrument includes more discriminating items than the 8-item version, and has the added advantage of including two items from each of the five ethical philosophies represented in the original 30-item pool. (shrink)

Research that examines unethical interpersonal treatment has received a great deal of attention from scholars and practitioners in recent years due to the remarkable impact of mistreatment in the workplace. However, the literature is incomplete because we have an inadequate understanding of insubordination, which we define as “subordinates’ disobedient behaviors that intentionally exhibit a defiant refusal of their supervisors’ authority.” In our study, we integrate social exchange theory and the advantageous comparison component of moral disengagement within the integrative model (...) of experiencing and responding to mistreatment at work. Then, we explain why subordinates disengage from moral control as they balance experiencing abusive supervision with perpetrating insubordination within negative supervisor–subordinate social exchange relationships. In Studies 1–4, we validate a five-item measure of insubordination and demonstrate its content, convergent, discriminant, criterion-related, and predictive validity. In Study 5, we demonstrate that there is a positive indirect effect of abusive supervision on insubordination through negative social exchange relationship quality that strengthens for subordinates who perceive higher levels of supervisors’ task performance than others. Overall, our study advances the conversation in the business ethics literature by creating a solid conceptual, empirical, and theoretical foundation for a cohesive program of insubordination research that meaningfully builds on prior findings in unethical interpersonal treatment research. (shrink)

Psychometrics is one of the main approaches to social scientific measurement. It is relied upon in drug testing, economic policymaking, recruitment, and other decision-making contexts. The first aim of this article is to introduce philosophers to key aspects of psychometrics, namely, classical test theory, item response theory, and construct validity. The second aim is to show how a debate on the nature of psychological attributes manifests in psychometrics. In this debate, realists claim that psychometric measures are (...) indicators of independently existing qualities, while operationalists argue that a measure defines its target attribute. The third and final aim is to argue that despite its poor reputation among philosophers, operationalism is a viable approach to psychometrics. (shrink)

Within the context of psychological measurement, realist commitments pervade methodology. Further, there are instances where particular scientific practices and decisions are explicable most plausibly against a background assumption of epistemic realism. That psychometrics is a realist enterprise provides a possible toehold for Stephen Jay Gould’s objections to psychometrics in The Mismeasure of Man and Joel Michell’s charges that psychometrics is a “pathological science.” These objections do not withstand scrutiny. There are no fewer than three activities in ongoing psychometric research which (...) presuppose a commitment to a minimal epistemic realism. Those activities include selecting between different models for representing data, estimating ability in the context of item response theory, and the move to make the individual the fundamental unit of analysis in psychometrics thereby calling for a shift in what sorts of data are evidentially relevant. In none of these activities are the commitments and disregard for evidence that Gould and Michell find objectionable or “pathological.”. (shrink)

We report a Monte Carlo study examining the effects of two strategies for handling measurement non-invariance – modeling and ignoring non-invariant items – on structural regression coefficients between latent variables measured with item response theory models for categorical indicators. These strategies were examined across four levels and three types of non-invariance – non-invariant loadings, non-invariant thresholds, and combined non-invariance on loadings and thresholds – in simple, partial, mediated and moderated regression models where the non-invariant latent variable occupied (...) predictor, mediator, and criterion positions in the structural regression models. When non-invariance is ignored in the latent predictor, the focal group regression parameters are biased in the opposite direction to the difference in loadings and thresholds relative to the referent group (i.e., lower loadings and thresholds for the focal group lead to overestimated regression parameters). With criterion non-invariance, the focal group regression parameters are biased in the same direction as the difference in loadings and thresholds relative to the referent group. While unacceptable levels of parameter bias were confined to the focal group, bias occurred at considerably lower levels of ignored non-invariance than was previously recognized in referent and focal groups. (shrink)

In this volume, a diverse group of world experts in personality assessment showcase a range of different viewpoints on response distortion. Contributors consider what it means to "fake" a personality assessment, why and how people try to obtain particular scores on personality tests, and what types of tests people can successfully manipulate. The authors present and discuss the usefulness of a range of traditional and cutting-edge methods for detecting and controlling the practice of faking. These methods include social desirability (...) scales, warnings, affective neutralization, unidimensional and multidimensional pairwise preferences, decision trees, linguistic analysis, situational measures, and methods based on item response theory. The wide range of viewpoints presented in this book are then summarized, synthesized, and evaluated. The authors make practical recommendations and suggest areas for future research. Anyone who wonders whether people exaggerate or lie outright on personality tests -- or questions what psychologists can and should do about it -- will find in this book stimulating questions and useful answers. (shrink)

How can corporate social responsibility initiatives influence brand love? Based on the theory of social identity complexity, we examined whether greater complexity of a sport fan’s multiple identifications with sport leagues led to higher multicultural tolerance and more positive perceptions of leagues’ corporate social responsibility activities. Further, brand authenticity was tested as a variable intervening between perceived corporate social responsibility and brand love. We analyzed this serial mediation effect impacting sport fans’ brand love for their multiple, favored and less (...) favored, sport leagues. Participants answered the scale item questionnaire for model assessment. The hypothesized model was supported as the indirect effect through all mediators was significant. Our results suggest that when sport fans acknowledge a high overlap among league fan groups, their tolerance is more likely to be higher than those who have a low overlap. Such high levels of tolerance influence how fans perceive corporate social responsibility initiatives, and these effects build up for fans to perceive the brand to be more authentic. These antecedents affected brand love through a serial mediation. Sport league managers should consider the diverse aspects across leagues for harmonious coexistence with other leagues and maintain brand authenticity for their social initiatives to result in a greater brand love in the consumer’s mind. (shrink)