This article provides unified axiomatic foundations for the most common optimality criteria in statistical decision theory. It considers a decision maker who faces a number of possible models of the world (possibly corresponding to true parameter values). Every model generates objective probabilities, and von Neumann–Morgenstern expected utility applies where these obtain, but no probabilities of models are given. This is the classic problem captured by Wald’s (Statistical decision functions, 1950) device of risk functions. In an (...) Anscombe–Aumann environment, I characterize Bayesianism (as a backdrop), the statistical minimax principle, the Hurwicz criterion, minimax regret, and the “Pareto” preference ordering that rationalizes admissibility. Two interesting findings are that c-independence is not crucial in characterizing the minimax principle and that the axiom which picks minimax regret over maximin utility is von Neumann–Morgenstern independence. (shrink)

This paper analyzes statistical decisions during the interim analyses of clinical trials. After some general remarks about the ethical and scientific demands of clinical trials, I introduce the notion of a hard-case clinical trial, explain the basic idea behind it, and provide a real example involving the interim analyses of zidovudine in asymptomatic HIV-infected patients. The example leads me to propose a decision analytic framework for handling ethical conflicts that might arise during the monitoring of hard-case clinical trials. (...) I use computer simulations to show how the framework can assist in reconciling certain ethical conflicts. The framework is partial, lacking the precision of a complete systematization of statistical monitoring procedures in practice. (shrink)

Setting optimal significance levels that minimize Type I and Type II errors allows for more transparent and well‐considered statistical decision making compared to the traditional α = 0.05 significance level. We use the optimal α approach to re‐assess conclusions reached by three recently published tests of the pace‐of‐life syndrome hypothesis, which attempts to unify occurrences of different physiological, behavioral, and life history characteristics under one theory, over different scales of biological organization. While some of the conclusions reached using (...) optimal α were consistent to those previously reported using the traditional α = 0.05 threshold, opposing conclusions were also frequently reached. The optimal α approach reduced probabilities of Type I and Type II errors, and ensured statistical significance was associated with biological relevance. Biologists should seriously consider their choice of α when conducting null hypothesis significance tests, as there are serious disadvantages with consistent reliance on the traditional but arbitrary α = 0.05 significance level. (shrink)

Statistical decision theory, whether based on Bayesian principles or other concepts such as minimax or admissibility, relies on minimizing expected loss or maximizing expected utility. Loss and utility functions are generally treated as unit-less numerical measures of value for consequences. Here, we address the issue of the units in which loss and utility are settled and the implications that those units have on the rankings of potential decisions. When multiple currencies are available for paying the loss, one must (...) take explicit account of which currency is used as well as the exchange rates between the various available currencies. (shrink)

Fuzzy sets and fuzzy logics are used to model events with imprecise, incomplete, and uncertain information. Researchers have developed numerous methods and techniques to cope with fuzziness or uncertainty. This research intends to introduce the novel concepts of complex neutrosophic relations and its types based on the idea of complex neutrosophic sets. In addition, these concepts are supported by suitable examples. A CNR discusses the quality of a relationship using the degree of membership, the degree of abstinence, and the degree (...) of nonmembership. Each of these degrees is a complex number from the unit circle in a complex plane. The real part of complex-valued degrees represents the amplitude term, while the imaginary part represents the phase term. This property empowers CNRs to model multidimensional variables. Moreover, some interesting properties and useful results have also been proved. Furthermore, the practicality of the proposed concepts is verified by an application, which discusses the use of the proposed concepts in statistical decision-making. Additionally, a comparative analysis between the novel concepts of CNRs and the existing methods is carried out. (shrink)

The rapidly increasing role of automation throughout the economy, culture and our personal lives has generated a large literature on the risks of algorithmic decision-making, particularly in high-stakes legal settings. Algorithmic tools are charged with bias, shrouded in secrecy, and frequently difficult to interpret. However, these criticisms have tended to focus on particular implementations, specific predictive techniques, and the idiosyncrasies of the American legal-regulatory regime. They do not address the more fundamental unease about the prospect that we might one (...) day replace judges with algorithms, no matter how fair, transparent, and intelligible they become. The aim of this paper is to propose an account of the source of that unease, and to evaluate its plausibility. I trace foundational unease with algorithmic decision-making in the law to the powerful intuition that there is a basic moral and legal difference between showing that something is true of many people just like you and showing that it is true of you. Human judgment attends to the exception; automation insists on blindly applying the rule. I show how this intuitive thought is connected to both epistemological arguments about the value of statistical evidence, as well as to court-centered conceptions of the rule of law. Unease with algorithmic decision-making in the law thus draws on an intuitive principle that underpins a disparate range of views in legal philosophy. This suggests the principle is deeply ingrained. Nonetheless, I argue that the powerful intuition is not as decisive as it may seem, and indeed runs into significant epistemological and normative challenges. At an epistemological level, I show how concerns about statistical evidence's ability to track the truth can be resolved by adopting a probabilistic, rather than modal, conception of truth-tracking. At a normative level, commitment to highly individualized decision-making co-exists with equally ingrained and competing principles, such as consistent application of law. This suggests that the “rule of law” may not identify a discrete set of institutional arrangements, as proponents of a court-centric conception would have it, but rather a more loosely defined set of values that could potentially be operationalized in multiple ways, including through some level of algorithmic adjudication. Although the prospect of replacing judges with algorithms is indeed unsettling, it does not necessarily entail unreasonable verdicts or an attack on the rule of law. (shrink)

In the debate about the legal value of naked statistical evidence, Di Bello argues that (1) the likelihood ratio of such evidence is unknown, (2) the decision-theoretic considerations indicate that a conviction based on such evidence is unacceptable when expected utility maximization is combined with fairness constraints, and (3) the risk of mistaken conviction based on such evidence cannot be evaluated and is potentially too high. We argue that Di Bello’s argument for (1) works in a rather narrow (...) context, and that (1) is not exactly in line with the way expert witnesses are required to formulate their opinions. Consequently, Di Bello’s argument for (2), which assumes (1), does not apply uniformly to all convictions based on naked evidence. Moreover, if Di Bello’s analysis is correct, it applies also to eyewitness testimony, given empirical results about its quality, and so the distinctions drawn by DiBello cut across the distinction between naked statistical evidence and other types of evidence. Finally, if we weaken the rather strong requirement of precise measurability of the risk of mistaken conviction, to the availability of reasonable but imprecise and fallible estimates, many field and empirical studies show that often the risk of mistaken conviction based on naked statistical evidence can be estimated to a similar extent as the risk of mistaken conviction based on any other sort of evidence. (shrink)

A popular informal argument suggests that statistics about the preponderance of criminal involvement among particular demographic groups partially justify others in making defensive mistakes against members of the group. One could worry that evidence-relative accounts of moral rights vindicate this argument. After constructing the strongest form of this objection, I offer several replies: most demographic statistics face an unmet challenge from reference class problems, even those that meet it fail to ground non-negligible conditional probabilities, even if they did, they introduce (...) new costs likely to cancel out any justificatory contribution of the statistic, but even if they didn’t, demographic facts are the wrong sort to make a moral difference to agents’ negative rights. I conclude that the popular argument should be rejected, and evidence-relative theories do not have the worrisome implication. (shrink)

The use of algorithms to support prediction-based decision-making is becoming commonplace in a range of domains including health, criminal justice, education, social services, lending, and hiring. An assumption governing such decisions is that there is a property Y such that individual a should be allocated resource R by decision-maker D if a is Y. When there is uncertainty about whether a is Y, algorithms may provide valuable decision support by accurately predicting whether a is Y on the (...) basis of known features of a. Based on recent work on statistical evidence in epistemology this article presents an argument against relying exclusively on algorithmic predictions to allocate resources when they provide purely statistical evidence that a is Y. The article then responds to the objection that any evidence that will increase the proportion of correct decisions should be accepted as the basis for allocations regardless of its epistemic deficiency. Finally, some important practical aspects of the conclusion are considered. (shrink)

In a recent issue of this journal, Philip Dawid proposes a framework for causal inference that is based on statistical decision theory and that is, in many aspects, compatible with the familiar framework of causal graphs ). This editorial compares the methodological features of the two frameworks as well as their epistemological basis.

To whatever extent the use of a behavioral, not an evidential, interpretation of decisions in the Lindley-Savage argument for Bayesian theory undermines its cogency as a criticism of typical standard practice, it also undermines the Neyman-Pearson theory as a support for typical standard practice. This leaves standard practice with far less theoretical support than Bayesian methods. It does nothing to resolve the anomalies and paradoxes of standard methods. (Similar statements apply to the common protestation that the models are not real (...) anyway.) The appropriate interpretation of tests as evidence, if possible at all, is difficult and counterintuitive. Any attempt to support tests as more than rules of thumb is doomed to failure. (shrink)

I thank Judea Pearl for his discussion of my paper and respond to the points he raises. In particular, his attachment to unaugmented directed acyclic graphs has led to a misapprehension of my own proposals. I also discuss the possibilities for developing a non-manipulative understanding of causality.

We develop a mathematical and interpretative foundation for the enterprise of decision-theoretic (DT) statistical causality, which is a straightforward way of representing and addressing causal questions. DT reframes causal inference as “assisted decision-making” and aims to understand when, and how, I can make use of external data, typically observational, to help me solve a decision problem by taking advantage of assumed relationships between the data and my problem. The relationships embodied in any representation of a causal (...) problem require deeper justification, which is necessarily context-dependent. Here we clarify the considerations needed to support applications of the DT methodology. Exchangeability considerations are used to structure the required relationships, and a distinction drawn between intention to treat and intervention to treat forms the basis for the enabling condition of “ignorability.” We also show how the DT perspective unifies and sheds light on other popular formalisations of statistical causality, including potential responses and directed acyclic graphs. (shrink)

The purpose of this study was to investigate the relationship between organizational ethical context and the individual ethical decision-making process. In addition, a new statistical approach combining cluster and discriminant analyses was developed to overcome violations of regression assumptions, which are commonly not identified and/or ignored in behavioral and psychological research. Using regressions and this new alternative method, the findings indicated that ethical context does indeed influence the various components of ethical reasoning. However, social desirability was the strongest (...) predictor of ethical decision making, which raises new concerns about how this bias can confound business ethics research. Finally, the findings showed that the alternative method provided more useful and interpretive results, indicating that it has the capacity to influence future empirical work in the field of business ethics, particularly when dealing with data that do not satisfy regression assumptions. The implications and limitations of the study are discussed, and several noteworthy suggestions for future research are provided. (shrink)

Can the evidence provided by software systems meet the standard of proof for civil or criminal cases, and is it individualized evidence? Or, to the contrary, do software systems exclusively provide bare statistical evidence? In this paper, we argue that there are cases in which evidence in the form of probabilities computed by software systems is not bare statistical evidence, and is thus able to meet the standard of proof. First, based on the case of State v. Loomis, (...) we investigate recidivism predictions provided by software systems used in the courtroom. Here, we raise problems for software systems that provide predictions that are based on bare statistical evidence. Second, by examining the case of People v. Chubbs, we argue that the statistical evidence provided by software systems in cold hit DNA cases may in some cases suffice for individualized evidence, on a view on which individualized evidence is evidence that normically supports the relevant proposition (Smith, in Mind 127:1193–1218, 2018). (shrink)

PREVIOUS WORK Theoretical discussion of the interval measurement of utility based upon theories of decision making under conditions of risk has been voluminous and will not be reviewed here. Those interested will find extensive ...

This paper offers a new angle on the common idea that the process of science does not support epistemic diversity. Under minimal assumptions on the nature of journal editing, we prove that editorial procedures, even when impartial in themselves, disadvantage less prominent research programs. This purely statistical bias in article selection further skews existing differences in the success rate and hence attractiveness of research programs, and exacerbates the reputation difference between the programs. After a discussion of the modeling assumptions, (...) the paper ends with a number of recommendations that may help promote scientific diversity through editorial decision making. (shrink)

We present a new approach to the old problem of how to incorporate the role of the observer in statistics. We show classical probability theory to be inadequate for this task and take refuge in the epsilon-model, which is the only model known to us caapble of handling situations between quantum and classical statistics. An example is worked out and some problems are discussed as to the new viewpoint that emanates from our approach.

The null-hypothesis significance-test procedure (NHSTP) is defended in the context of the theory-corroboration experiment, as well as the following contrasts: (a) substantive hypotheses versus statistical hypotheses, (b) theory corroboration versus statistical hypothesis testing, (c) theoretical inference versus statistical decision, (d) experiments versus nonexperimental studies, and (e) theory corroboration versus treatment assessment. The null hypothesis can be true because it is the hypothesis that errors are randomly distributed in data. Moreover, the null hypothesis is never used as (...) a categorical proposition. Statistical significance means only that chance influences can be excluded as an explanation of data; it does not identify the nonchance factor responsible. The experimental conclusion is drawn with the inductive principle underlying the experimental design. A chain of deductive arguments gives rise to the theoretical conclusion via the experimental conclusion. The anomalous relationship between statistical significance and the effect size often used to criticize NHSTP is more apparent than real. The absolute size of the effect is not an index of evidential support for the substantive hypothesis. Nor is the effect size, by itself, informative as to the practical importance of the research result. Being a conditional probability, statistical power cannot be the apriori probability of statistical significance. The validity of statistical power is debatable because statistical significance is determined with a single sampling distribution of the test statistic based on H0, whereas it takes two distributions to represent statistical power or effect size. Sample size should not be determined in the mechanical manner envisaged in power analysis. It is inappropriate to criticize NHSTP for nonstatistical reasons. At the same time, neither effect size, nor confidence interval estimate, nor posterior probability can be used to exclude chance as an explanation of data. Neither can any of them fulfill the nonstatistical functions expected of them by critics. (shrink)

Multivariate pattern analysis can reveal new information from neuroimaging data to illuminate human cognition and its disturbances. Here, we develop a methodological approach, based on multivariate statistical/machine learning and time series analysis, to discern cognitive processing stages from functional magnetic resonance imaging (fMRI) blood oxygenation level dependent (BOLD) time series. We apply this method to data recorded from a group of healthy adults whilst performing a virtual reality version of the delayed win-shift radial arm maze (RAM) task. This task (...) has been frequently used to study working memory and decision making in rodents. Using linear classifiers and multivariate test statistics in conjunction with time series bootstraps, we show that different cognitive stages of the task, as defined by the experimenter, namely, the encoding/retrieval, choice, reward and delay stages, can be statistically discriminated from the BOLD time series in brain areas relevant for decision making and working memory. Discrimination of these task stages was significantly reduced during poor behavioral performance in dorsolateral prefrontal cortex (DLPFC), but not in the primary visual cortex (V1). Experimenter-defined dissection of time series into class labels based on task structure was confirmed by an unsupervised, bottom-up approach based on Hidden Markov Models. Furthermore, we show that different groupings of recorded time points into cognitive event classes can be used to test hypotheses about the specific cognitive role of a given brain region during task execution. We found that whilst the DLPFC strongly differentiated between task stages associated with different memory loads, but not between different visual-spatial aspects, the reverse was true for V1. Our methodology illustrates how different aspects of cognitive information processing during one and the same task can be separated and attributed to specific brain regions based on information contained in multivariate patterns of voxel activity. (shrink)

We provide statistical inference for measures of predictive success. These measures are frequently used to evaluate and compare the performance of different models of individual and group decision making in experimental and revealed preference studies. We provide a brief illustration of our findings by comparing the predictive success of different revealed preference tests for models of intertemporal decision making. This demonstrates that it is possible to compare the predictive success of different models in a statistically meaningful way.

I thank Ilya Shpitser for his comments on my article, and discuss the use of models with restricted interventions.

A major problem for interpersonal aggregation is how to compare utility across individuals; a major problem for decision-making under normative uncertainty is the formally analogous problem of how to compare choice-worthiness across theories. We introduce and study a class of methods, which we call statistical normalization methods, for making interpersonal comparisons of utility and intertheoretic comparisons of choice-worthiness. We argue against the statistical normalization methods that have been proposed in the literature. We argue, instead, in favor of (...) normalization of variance: we claim that this is the account that most plausibly gives all individuals or theories ‘equal say’. To this end, we provide two proofs that variance normalization has desirable properties that all other normalization methods lack, though we also show how different assumptions could lead one to axiomatize alternative statistical normalization methods. (shrink)

Determining optimal units of representing morphologically complex words in the mental lexicon is a central question in psycholinguistics. Here, we utilize advances in computational sciences to study human morphological processing using statistical models of morphology, particularly the unsupervised Morfessor model that works on the principle of optimization. The aim was to see what kind of model structure corresponds best to human word recognition costs for multimorphemic Finnish nouns: a model incorporating units resembling linguistically defined morphemes, a whole-word model, or (...) a model that seeks for an optimal balance between these two extremes. Our results showed that human word recognition was predicted best by a combination of two models: a model that decomposes words at some morpheme boundaries while keeping others unsegmented and a whole-word model. The results support dual-route models that assume that both decomposed and full-form representations are utilized to optimally process complex words within the mental lexicon. (shrink)

The debates between Bayesian, frequentist, and other methodologies of statistics have tended to focus on conceptual justifications, sociological arguments, or mathematical proofs of their long run properties. Both Bayesian statistics and frequentist (“classical”) statistics have strong cases on these grounds. In this article, we instead approach the debates in the “Statistics Wars” from a largely unexplored angle: simulations of different methodologies’ performance in the short to medium run. We conducted a large number of simulations using a straightforward decision problem (...) based around tossing a coin with unknown bias and then placing bets. In this simulation, we programmed four players, inspired by Bayesian statistics, frequentist statistics, Jon Williamson’s version of Objective Bayesianism, and a player who simply extrapolates from observed frequencies to general frequencies. The last player functions as a benchmark: a statistical methodology should at least outperform a crude form of induction. We focused on the performance of these methodologies in guiding the players towards good decisions. Unlike an earlier simulation study of this type, we found no systematic difference in performance between the Bayesian and frequentist players, provided the Bayesian used a flat prior and the frequentist used a low confidence level. Unlike that study, we were able to use Big Data methods to mitigate problems of random error in the simulation results. The Williamsonian player, who is a novel element of our study, also had no systematic differences in their performance, provided that they use a low confidence level. These players performed similarly even in the very short run, when players were making different decisions. Our study indicates that all three methodologies should be taken seriously by philosophers and practitioners of statistics. However, the frequentist and Williamsonian performed poorly when their confidence levels were high, and the Bayesian was surprisingly harmed by biased priors, providing some unexpected lessons for these methodologies when facing this type of decision problem. (shrink)

In many learning or inference tasks human behavior approximates that of a Bayesian ideal observer, suggesting that, at some level, cognition can be described as Bayesian inference. However, a number of findings have highlighted an intriguing mismatch between human behavior and standard assumptions about optimality: People often appear to make decisions based on just one or a few samples from the appropriate posterior probability distribution, rather than using the full distribution. Although sampling-based approximations are a common way to implement Bayesian (...) inference, the very limited numbers of samples often used by humans seem insufficient to approximate the required probability distributions very accurately. Here, we consider this discrepancy in the broader framework of statistical decision theory, and ask: If people are making decisions based on samples—but as samples are costly—how many samples should people use to optimize their total expected or worst-case reward over a large number of decisions? We find that under reasonable assumptions about the time costs of sampling, making many quick but locally suboptimal decisions based on very few samples may be the globally optimal strategy over long periods. These results help to reconcile a large body of work showing sampling-based or probability matching behavior with the hypothesis that human cognition can be understood in Bayesian terms, and they suggest promising future directions for studies of resource-constrained cognition. (shrink)

Inductive logic generalizes the idea of logical entailment and provides standards for the evaluation of non-conclusive arguments. A main application of inductive logic is the generalization of observational data to theoretical models. In the empirical sciences, the mathematical theory of statistics addresses the same problem. This paper argues that there is no separable purely logical aspect of statistical inference in a variety of complex problems. Instead, statistical practice is often motivated by decision-theoretic considerations and resembles empirical science.

This article presents a fairness principle for evaluating decision-making based on predictions: a decision rule is unfair when the individuals directly impacted by the decisions who are equal with respect to the features that justify inequalities in outcomes do not have the same statistical prospects of being benefited or harmed by them, irrespective of their socially salient morally arbitrary traits. The principle can be used to evaluate prediction-based decision-making from the point of view of a wide (...) range of antecedently specified substantive views about justice in outcome distributions. (shrink)

An examination of topics involved in statistical reasoning with imprecise probabilities. The book discusses assessment and elicitation, extensions, envelopes and decisions, the importance of imprecision, conditional previsions and coherent statistical models.

This work presents a Decision Making Model riferring to the forecasts about Football World Cup in Brazil. The aim of this work is to demonstrate how it is possible to approach young students to the study of Mathematics through evoking themes that are congenial to them and able to arouse their interest.