Tetris provides a difficult, dynamic task environment within which some people are novices and others, after years of work and practice, become extreme experts. Here we study two core skills; namely, choosing the goal or objective function that will maximize performance and a feature-based analysis of the current game board to determine where to place the currently falling zoid so as to maximize the goal. In Study 1, we build cross-entropy reinforcement learning models to determine whether different (...) goals result in different feature weights. Two of these optimization strategies quickly rise to performance plateaus, whereas two others continue toward higher but more jagged heights. In Study 2, we compare the zoid placement decisions made by our best CERL models with those made by 67 human players. Across 370,131 human game episodes, two CERL models picked the same zoid placements as our lowest scoring human for 43% of the placements and as our three best scoring experts for 65% of the placements. Our findings suggest that people focus on maximizing points, not number of lines cleared or number of levels reached. They also show that goal choice influences the choice of zoid placements for CERLs and suggest that the same is true of humans. Tetris has a repetitive task structure that makes Tetris more tractable and more like a traditional experimental psychology paradigm than many more complex games or tasks. Hence, although complex, Tetris is not overwhelmingly complex and presents a right-sized challenge to cognitive theories, especially those of integrated cognitive systems. (shrink)

Algorithm Decision Making (ADM) systems designed to augment or automate human decision-making have the potential to produce better decisions while also freeing up human time and attention for other pursuits. For this potential to be realised, however, algorithmic decisions must be sufficiently aligned with human goals and interests. We take a Principal-Agent (P-A) approach to the questions of ADM alignment and trust. In a broad sense, ADM is beneficial if and only if human principals can trust (...) algorithmic agents to act faithfully on their behalf. This mirrors the challenge of facilitating P-A relationships among humans, but the peculiar nature of human-machine interaction also raises unique issues. The problem of asymmetric information is omnipresent but takes a different form in the context of ADM. Although the decision-making machinery of an algorithmic agent can in principle be laid bare for all to see, the sheer complexity of ADM systems based on deep learning models prevents straightforward monitoring. We draw on literature from economics and political science to argue that the problem of trust in ADM systems should be addressed at the level of institutions. Although the dyadic relationship between human principals and algorithmic agents is our ultimate concern, cooperation at this level must rest against an institutional environment which allows humans to effectively evaluate and choose among algorithmic alternatives. (shrink)

A deep reinforcement learning-based computational guidance method is presented, which is used to identify and resolve the problem of collision avoidance for a variable number of fixed-wing UAVs in limited airspace. The cooperative guidance process is first analyzed for multiple aircraft by formulating flight scenarios using multiagent Markov game theory and solving it by machine learning algorithm. Furthermore, a self-learning framework is established by using the actor-critic model, which is proposed to train collision avoidance decision-making (...) neural networks. To achieve higher scalability, the neural network is customized to incorporate long short-term memory networks, and a coordination strategy is given. Additionally, a simulator suitable for multiagent high-density route scene is designed for validation, in which all UAVs run the proposed algorithm onboard. Simulated experiment results from several case studies show that the real-time guidance algorithm can reduce the collision probability of multiple UAVs in flight effectively even with a large number of aircraft. (shrink)

We argue that cognitive models can provide a common ground between human users and deep reinforcement learning (Deep RL) algorithms for purposes of explainable artificial intelligence (AI). Casting both the human and learner as cognitive models provides common mechanisms to compare and understand their underlying decision-making processes. This common grounding allows us to identify divergences and explain the learner's behavior in human understandable terms. We present novel salience techniques that highlight the most relevant features in each (...) model's decision-making, as well as examples of this technique in common training environments such as Starcraft II and an OpenAI gridworld. (shrink)

Autonomous underwater vehicles are widely used to accomplish various missions in the complex marine environment; the design of a control system for AUVs is particularly difficult due to the high nonlinearity, variations in hydrodynamic coefficients, and external force from ocean currents. In this paper, we propose a controller based on deep reinforcement learning in a simulation environment for studying the control performance of the vectored thruster AUV. RL is an important method of artificial intelligence that can (...) learn behavior through trial-and-error interactions with the environment, so it does not need to provide an accurate AUV control model that is very hard to establish. The proposed RL algorithm only uses the information that can be measured by sensors inside the AUVs as the input parameters, and the outputs of the designed controller are the continuous control actions, which are the commands that are set to the vectored thruster. Moreover, a reward function is developed for deep RL controller considering different factors which actually affect the control accuracy of AUV navigation control. To confirm the algorithm’s effectiveness, a series of simulations are carried out in the designed simulation environment, which is a method to save time and improve efficiency. Simulation results prove the feasibility of the deep RL algorithm applied to the control system for AUV. Furthermore, our work also provides an optional method for robot control problems to deal with improving technology requirements and complicated application environments. (shrink)

How can anyone be rational in a world where knowledge is limited, time is pressing, and deep thought is often an unattainable luxury? Traditional models of unbounded rationality and optimization in cognitive science, economics, and animal behavior have tended to view decision-makers as possessing supernatural powers of reason, limitless knowledge, and endless time. But understanding decisions in the real world requires a more psychologically plausible notion of bounded rationality. In Simple heuristics that make us smart (...) (Gigerenzer et al. 1999), we explore fast and frugal heuristics – simple rules in the mind's adaptive toolbox for making decisions with realistic mental resources. These heuristics can enable both living organisms and artificial systems to make smart choices quickly and with a minimum of information by exploiting the way that information is structured in particular environments. In this précis, we show how simple building blocks that control information search, stop search, and make decisions can be put together to form classes of heuristics, including: ignorance-based and one-reason decision making for choice, elimination models for categorization, and satisficing heuristics for sequential search. These simple heuristics perform comparably to more complex algorithms, particularly when generalizing to new data – that is, simplicity leads to robustness. We present evidence regarding when people use simple heuristics and describe the challenges to be addressed by this research program. Key Words: adaptive toolbox; bounded rationality; decision making; elimination models; environment structure; heuristics; ignorance-based reasoning; limited information search; robustness; satisficing; simplicity. (shrink)

The Internet of Things (IoT) is a fast-growing technology that connects everyday devices to the Internet, enabling wireless, low-consumption and low-cost communication and data exchange. IoT has revolutionized the way devices interact with each other and the internet. The more devices become connected, the greater the risk of security breaches. There is currently a need for new approaches to algorithms that can detect malware regardless of the size of the network and that can adapt to dynamic changes in the (...) network. Through the use of a multi-agent reinforcement learning algorithm, this paper proposes a novel algorithm for malware detection in IoT devices. The proposed algorithm is not strongly dependent on the size of the IoT network due to the that its training is adapted using time differences if the IoT network size is small or Monte Carlo otherwise. To validate the proposed algorithm in an environment as close to reality as possible, we proposed a scenario based on a real IoT network, where we tested different malware propagation models. Different simulations varying the number of agents and nodes in the IoT network have been developed. The result of these simulations proves the efficiency and adaptability of the proposed algorithm in detecting malware, regardless of the malware propagation model. (shrink)

Rapid and precise air operation mission planning is a key technology in unmanned aerial vehicles autonomous combat in battles. In this paper, an end-to-end UAV intelligent mission planning method based on deep reinforcement learning is proposed to solve the shortcomings of the traditional intelligent optimization algorithm, such as relying on simple, static, low-dimensional scenarios, and poor scalability. Specifically, the suppression of enemy air defense mission planning is described as a sequential decision-making problem and formalized as a (...) Markov decision process. Then, the SEAD intelligent planning model based on the proximal policy optimization algorithm is established and a general intelligent planning architecture is proposed. Furthermore, three policy training tricks, i.e., domain randomization, maximizing policy entropy, and underlying network parameter sharing, are introduced to improve the learning performance and generalizability of PPO. Experiments results show that the model in this work is efficient and stable, and can be adapted to the unknown continuous high-dimensional environment. It can be concluded that the UAV intelligent mission planning model based on DRL has powerful intelligent planning performance, and provides a new idea for researching UAV autonomy. (shrink)

Formal models of argumentation have been investigated in several areas, from multi-agent systems and artificial intelligence (AI) to decision making, philosophy and law. In artificial intelligence, logic-based models have been the standard for the representation of argumentative reasoning. More recently, the standard logic-based models have been shown equivalent to standard connectionist models. This has created a new line of research where (i) neural networks can be used as a parallel computational model for argumentation and (...) (ii) neural networks can be used to combine argumentation, quantitative reasoning and statistical learning. At the same time, non-standard logic models of argumentation started to emerge. In this paper, we propose a connectionist cognitive model of argumentation that accounts for both standard and non-standard forms of argumentation. The model is shown to be an adequate framework for dealing with standard and non-standard argumentation, including joint-attacks, argument support, ordered attacks, disjunctive attacks, meta-level attacks, self-defeating attacks, argument accrual and uncertainty. We show that the neural cognitive approach offers an adequate way of modelling all of these different aspects of argumentation. We have applied the framework to the modelling of a public prosecution charging decision as part of a real legal decision making case study containing many of the above aspects of argumentation. The results show that the model can be a useful tool in the analysis of legal decision making, including the analysis of what-if questions and the analysis of alternative conclusions. The approach opens up two new perspectives in the short-term: the use of neural networks for computing prevailing arguments efficiently through the propagation in parallel of neuronal activations, and the use of the same networks to evolve the structure of the argumentation network through learning (e.g. to learn the strength of arguments from data). (shrink)

Successfully explaining and replicating the complexity and generality of human and animal learning will require the integration of a variety of learning mechanisms. Here, we introduce a computational model which integrates associative learning (AL) and reinforcement learning (RL). We contrast the integrated model with standalone AL and RL models in three simulation studies. First, a synthetic grid‐navigation task is employed to highlight performance advantages for the integrated model in an environment where the reward structure is both (...) diverse and dynamic. The second and third simulations contrast the performances of the three models in behavioral experiments, demonstrating advantages for the integrated model in accounting for behavioral data. (shrink)

Low heart rate causes a risk of death, heart disease, and cardiovascular diseases. Therefore, monitoring the heart rate is critical because of the heart’s function to discover its irregularity to detect the health problems early. Rapid technological advancement allows healthcare sectors to consolidate and analyze massive health-based data to discover risks by making more accurate predictions. Therefore, this work proposes a real-time prediction system for heart rate, which helps the medical care providers and patients avoid heart rate (...) risk in real time. The proposed system consists of two phases, namely, an offline phase and an online phase. The offline phase targets developing the model using different forecasting techniques to find the lowest root mean square error. The heart rate time-series dataset is extracted from Medical Information Mart for Intensive Care. Recurrent neural network, long short-term memory, gated recurrent units, and bidirectional long short-term memory are applied to heart rate time series. For the online phase, Apache Kafka and Apache Spark have been used to predict the heart rate in advance based on the best developed model. According to the experimental results, the GRU with three layers has recorded the best performance. Consequently, GRU with three layers has been used to predict heart rate 5 minutes in advance. (shrink)

The predictive processing framework includes a broad set of ideas, which might be articulated and developed in a variety of ways, concerning how the brain may leverage predictive models when implementing perception, cognition, decision-making, and motor control. This article provides an up-to-date introduction to the two most influential theories within this framework: predictive coding and active inference. The first half of the paper (Sections 2–5) reviews the evolution of predictive coding, from early ideas about efficient coding in the visual (...) system to a more general model encompassing perception, cognition, and motor control. The theory is characterized in terms of the claims it makes at Marr's computational, algorithmic, and implementation levels of description, and the conceptual and mathematical connections between predictive coding, Bayesian inference, and variational free energy (a quantity jointly evaluating model accuracy and complexity) are explored. The second half of the paper (Sections 6–8) turns to recent theories of active inference. Like predictive coding, active inference models assume that perceptual and learning processes minimize variational free energy as a means of approximating Bayesian inference in a biologically plausible manner. However, these models focus primarily on planning and decision-making processes that predictive coding models were not developed to address. Under active inference, an agent evaluates potential plans (action sequences) based on their expected free energy (a quantity that combines anticipated reward and information gain). The agent is assumed to represent the world as a partially observable Markov decision process with discrete time and discrete states. Current research applications of active inference models are described, including a range of simulation work, as well as studies fitting models to empirical data. The paper concludes by considering future research directions that will be important for further development of both models. (shrink)

In reinforcement learning, a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian (...) model and temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty. (shrink)

One most common problem in reinforcement learning systems (e.g. Q-learning) is to reduce the number of trials to converge to an optimal policy. As one of the solution to the problem, k-certainty exploration method was proposed. Miyazaki reported that this method could determine an optimal policy faster than Q-learning in Markov decision processes (MDPs). This method is very efficient learning method. But, we propose an improvement plan that makes this method more efficient. In k-certainty exploration method, (...) in case there is no k-uncertainty rule (rules which aren’t selected more than k times) in current state, an agent sometimes walks randomly until it finds a state where it can select a k-uncertainty rule. We think that this behavior is not efficient. To reduce this useless behavior, we propose combining k-certainty exploration method with Dynamic programming (DP). Miyazaki’s system uses DP after all rules are executed at least k times. But, our method uses k-certainty exploration method along with DP during learning. Our method, takes two pattern actions. In case an agent can select k-uncertainty rules, one of these rules is selected at random as k-certainty exploration method. In another case there is no k-uncertainty rule, behavior of agent is different from the behavior of k-certainty exploration method. In that case, our method uses DP to compute an optimal policy for moving from a current state to a state in which some k-uncertainty rules remain. The model for DP is constructed by using only known states. The outline will be described below. First, an agent makes a map constructed by using only known states. In the map, goals are states in which there are k-uncertainty rules and arbitrary state values are set in these states. Point to which attention should be paid is that the map is not given from outside. It is made from only experience. Next, each state values of states in which there are only k-certainty rules in the map are computed by DP (we used Policy Iteration). Finally, an agent continues to select greedy action until it arrives at a state in which it can select a k-uncertainty rule. By this improvement, we expect it can determine an optimal policy faster than k-certainty exploration method. And we have verified that our exploration method can determine an optimal policy faster than k-certainty exploration method by computer simulation. (shrink)

The book consists of 6 chapters. Chapter One explains the reason why SLA researchers should study the language learning environment in space: population movements associated with internal and external migration and social mobility such as the circuits of commodity production and distribution create much space, in which language learning environment become diverse and uneven. With the spatial perspective, we can fully understand the interactions between language learners and the world or environments.In Chapter Two, by introducing the brief history of Critical (...) Spatial Theory, the author wants to tell us one of the key ideas in the theory: the social production of space, which views space as the product of the social relations of its era. Based on it, the author adds that the production of languages is an integral aspect of the production of space on a global scale. It can help us to understand how social mobility shapes language learning environments. From this perspective, the local environments in which languages are learned are outcomes of the production of space under the influence of mobility. It reiterates the key view that the global mobilities of physical objects (people, goods and information) produce space.In Chapter Three, the author introduces the mainstream views held by modern linguistics: languages are conceptualized as objects in the space. They hold that the spatiality of language is given by providing correct writing systems and standardized rules for various languages. Influenced by mainstream conception of language from modern linguistics, SLA tends to regard languages as self-contained objects in space. However, this view neglects the relation between language and the learning environments. The author holds that the distribution and circulation of languages and their varieties in geographical space produce the spatiality of language, which clears the ground for an environment or object-as-space view of language in Chapter Four.In Chapter Four, the author develops an alternative "objects-as-space" view and introduces three concepts to illustrate spatiality of language: flat ontology, assemblage and mobilities. The concept of flat ontology helps explain what language is and how it comes into being in the world; From assemblage perspective, the author develops the idea that everything is an object, so are human beings and languages. Language (the non-physical object) interacts with various physical objects, forming the language-bearing assemblage. This assemblage is the form of spatiality of language. The mobilities theory explains how language-bearing assemblages develop or change on a global scale because of social mobility, emphasizing the importance of learning environment in SLA. All in all, the author outlines how language makes its way into physical space in the form of assemblages, or assemblages of assemblages.Chapter Five centers on language learning environment. First, the author explains the relation between language learners and their learning environment from an ecological view: the learner is a part of the language learning environment, and second language learning is a spatial interaction between the learners and the semiotic resources provided by environment. Then, the author discusses language environment from two different perspectives. From an 'individual' perspective, environments are assembled by individuals from the spatial resources available; from an 'areal' view, environments are shaped both by local circumstances and global circulations of language-bearing assemblages. Meanwhile, the author also discusses other language environments like the multilingual city and online resources. What the author hopes to develop in this chapter is an environment perspective that grounds the contexts of language learning in space, which helps the readers think about language learning from a spatial perspective.Chapter Six contains three parts. In the first part, it outlines the work of a number of researchers from sociolinguistics and the contribution they made to the spatial theory. In the second part, the author wants to tell us how the idea of language learning environment emerges from his own research. The last part discusses the four main researching routes (areal studies, studies of individuals learning in settings, studies of the construction of individual learning environment and design-based studies) that might influence SLA research on language learning environments and some research methods. The book is in alignment with several recent "turns" occurring in the field of SLA, such as "social turn" (Block, 2003) and "multilingual turn" (May, 2014), both of which bring to the fore the social aspects involved in language learning. While the first "turn" accentuates the role of social context in shaping and influencing learner cognition, the latter highlights how the physical multilingual world remold language learning experiences, and how the mental multilingual world helps raise learners' language awareness, and facilitates their successful construction of an identity of legitimate multi-language users. The book under review joins up the two "turns" in acknowledging that language learners are persons-in-context with organic interaction with social learning environment (Ushioda, 2021). This view has research and pedagogical implications. From the point of doing research, spatial perspectives, advocated in the book, afford huge research potentials for both language teachers and (instructed) SLA researchers to conduct situated investigations, exploring the complex and dynamic nexus between language learners and their surrounding environment (not restricted to people) that play a part in their learning experience, and thus gaining insightful understanding of how individuals and learning contexts are coupled with each other, how they evolve together, and how they constitute an ecosystem that generates affordances, facilitating or constraining language learning. Pedagogically speaking, the spatial perspectives encourage language teachers to hold an ecological view of language teaching and learning, directing their attention to the individuality of each language learner and their complex, dynamic and multifaceted relationality with other learners and the physical realities, rather than solely focusing on the abstract systems (e.g., the development of cognitive aspect). This holistic view allows language teachers to take measures to address concrete questions emerging in and out of the classrooms, and stimulates them to design and experiment with pedagogical interventions that help to create a favorable environment for language learning. This view also invites language teachers to take learner agency into account (Xu & Long, 2020) and to design pedagogical activities that enable learners to take agentive actions to adapt their environments as they actively create and seek their own language learning opportunities, making the world their own (Bronfenbrenner, 1979). In turn, teachers' agency-enabling activities will ultimately help learners to autonomously access a range of resources in this new affordance-rich era to improve their language skills, and thus become more independent in their pathway to life-long learning.Beside the above benefits, the book is also theoretically and methodologically friendly. In illustrating the role of space in SLA, the author invokes a plethora of theories from such disciplines as sociology, sociolinguistics, psychology, etc. This orientation helps readers to build a transdisciplinary framework to capture the multi-layered nature of SLA, and to acquire a nuanced understanding of how language learning is influenced by multiple scales of environmental factors. Methodologically, the author proposes several specific methods that are especially apt to investigate language learning environment. These methods, together with the theoretical foundation laid by the book, will definitely help researchers and language teaching practitioners to further delve into the complex interplay between context and language learners.Admittedly, the book is not without weaknesses. It spends most of its words on illustrating the importance of physical environment in SLA. Another essential component, the virtual environment is insufficiently referenced and discussed. This lacking is undoubtedly a regret the book leaves with us, especially in these years of pandemic, where language learning in virtual spaces is becoming a norm.All in all, the book is an invaluable contribution to the field of SLA, and is worthy of recommending to language teaching practitioners, researchers, and postgraduates in applied linguistics. (shrink)

Environmental landscaping is known to build, plan, and manage landscapes that consider the ecology of a site and produce gardens that benefit both people and the rest of the ecosystem. Landscaping and the environment are combined in landscape design planning to provide holistic answers to complex issues. Seeding native species and eradicating alien species are just a few ways humans influence the region’s ecosystem. Landscape architecture is the design of landscapes, urban areas, or gardens and their modification. It comprises (...) the construction of urban and rural landscapes via coordinating the creation and management of open spaces and economics, finding a job, and working within a confined project budget. There was a lot of discussion about global warming and water shortages. There is a lot of hope to be found even in the face of seemingly insurmountable obstacles. AI is becoming more significant in many urban landscape planning and design elements with the advent of web 4.0 and Human-Centred computing. It created a virtual reality-based landscape to create deep neural networks (DNNs) to make deep learning (DL) more user-friendly and efficient. Users may only manipulate physical items in this environment to manually construct neural networks. These setups are automatically converted into a model, and the real-time testing set is reported and aware of the DNN models that users are producing. This research presents a novel strategy for combining DL-DNN with landscape architecture, providing a long-term solution to the problem of environmental pollution. Carbon dioxide levels are constantly checked when green plants are in and around the house. Plants, on either hand, remove toxins from the air, making it easier to maintain a healthy environment. Human-centered Artificial Intelligence-based web 4.0 may be used to assess and evaluate the data model. The study findings can be sent back into the design process for further modification and optimization. (shrink)

This paper focuses on the theme of the application of deep learning in the field of basketball sports, using research methods such as literature research, video analysis, comparative research, and mathematical statistics to explore deep learning in real-time analysis of basketball sports data. The basketball posture action recognition and analysis system proposed for basketball movement is composed of two parts serially. The first part is based on the bottom-up posture estimation method to locate the joint (...) points and is used to extract the posture sequence of the target in the video. The second part is the analysis and research of the action recognition algorithm based on the convolution of the space-time graph. According to the extracted posture sequence, the basketball action of the set classification is recognized. In order to obtain more accurate and three-dimensional information, a multitraining target method can be used in training; that is, multiple indicators can be detected and feedback is provided at the same time to correct player errors in time; the other is an auxiliary method, which is compared with ordinary training. The method can actively correct technical movements, train players to form muscle memory, and improve their abilities. Through the research of this article, it provides a theoretical basis for promoting the application of deep learning in the field of basketball and also provides a theoretical reference for the wider application of deep learning in the field of sports. At the same time, the designed real-time analysis system of basketball data also provides more actual reference values for coaches and athletes. (shrink)

ABSTRACT I argue that this examination and appreciation for the shift to abductive reasoning should be extended to the intersection of neuroscience and novel brain-computer interfaces too. This paper highlights the implications of applying abductive reasoning to personalized implantable neurotechnologies. Then, it explores whether abductive reasoning is sufficient to justify insurance coverage for devices absent widespread clinical trials, which are better applied to one-size-fits-all treatments. INTRODUCTION In contrast to the classic model of randomized-control trials, often with a large number (...) of subjects enrolled, precision medicine attempts to optimize therapeutic outcomes by focusing on the individual.[i] A recent publication highlights the strengths and weakness of both traditional evidence-based medicine and precision medicine.[ii] Plus, it outlines a tension in the shift from evidence-based medicine’s inductive reasoning style (the collection of data to postulate general theories) to precision medicine’s abductive reasoning style (the generation of an idea from the limited data available).[iii] The paper’s main example is the application of precision medicine for the treatment of cancer.[iv] I argue that this examination and appreciation for the shift to abductive reasoning should be extended to the intersection of neuroscience and novel brain-computer interfaces too. As the name suggests, brain-computer interfaces are a significant advancement in neurotechnology that directly connects someone’s brain to external or implanted devices.[v] Among the various kinds of brain-computer interfaces, adaptive deep brain stimulation devices require numerous personalized adjustments to their settings during the implantation and computation stages in order to provide adequate relief to patients with treatment-resistant disorders. What makes these devices unique is how adaptive deep brain stimulation integrates a sensory component to initiate the stimulation. While not commonly at the level of sophistication as self-supervising or generative large language models,[vi] they currently allow for a semi-autonomous form of neuromodulation. This paper highlights the implications of applying abductive reasoning to personalized implantable neurotechnologies. Then, it explores whether abductive reasoning is sufficient to justify insurance coverage for devices absent widespread clinical trials, which are better applied to one-size-fits-all treatments.[vii] ANALYSIS I. The State of Precision Medicine in Oncology and the Epistemological Shift While a thorough overview of precision medicine for the treatment of cancer is beyond the scope of this article, its practice can be roughly summarized as identifying clinically significant characteristics a patient possesses (e.g., genetic traits) to land on a specialized treatment option that, theoretically, should benefit the patient the most.[viii] However, in such a practice of stratification patients fall into smaller and smaller populations and the quality of evidence that can be applied to anyone outside these decreases in turn.[ix] As inductive logic helps to articulate, the greater the number of patients that respond to a particular therapy the higher the probability of its efficacy. By straying from this logical framework, precision medicine opens the treatment of cancer to more uncertainty about the validity of these approaches to the resulting disease subcategories.[x] Thus, while contemporary medical practices explicitly describe some treatments as “personalized”, they ought not be viewed as inherently better founded than other therapies.[xi] A relevant contemporary case of precision medicine out of Norway focuses on the care of a patient with cancer between the ventricles of the heart and esophagus, which had failed to respond to the standard regimen of therapies over four years.[xii] In a last-ditch effort, the patient elected to pay out-of-pocket for an experimental immunotherapy (nivolumab) at a private hospital. He experienced marked improvements and a reduction in the size of the tumor. Understandably, the patient tried to pursue further rounds of nivolumab at a public hospital. However, the hospital initially declined to pay for it given the “lack of evidence from randomised clinical trials for this drug relating to this [patient’s] condition.”[xiii] In rebuttal to this claim, the patient countered that he was actually similar to a subpopulation of patients who responded in “open‐label, single arm, phase 2 studies on another immune therapy drug” (pembrolizumab).[xiv] Given this interpretation of the prior studies and the patient’s response, further rounds of nivolumab were approved. Had the patient not had improvements in the tumor’s size following a round of nivolumab, then pembrolizumab’s prior empirical evidence in isolation would have been insufficient, inductively speaking, to justify his continued use of nivolumab.[xv] The case demonstrates a shift in reasoning from the traditional induction to abduction. The phenomenon of ‘cancer improvement’ is considered causally linked to nivolumab and its underlying physiological mechanisms.[xvi] However, “the weakness of abductions is that there may always be some other better, unknown explanation for an effect. The patient may for example belong to a special subgroup that spontaneously improves, or the change may be a placebo effect. This does not mean, however, that abductive inferences cannot be strong or reasonable, in the sense that they can make a conclusion probable.”[xvii] To demonstrate the limitations of relying on the abductive standard in isolation, commentators have pointed out that side effects in precision medicine are hard to rule out as being related to the initial intervention itself unless trends from a group of patients are taken into consideration.[xviii] As artificial intelligence (AI) assists the development of precision medicine for oncology, this uncertainty ought to be taken into consideration. The implementation of AI has been crucial to the development of precision medicine by providing a way to combine large patient datasets or a single patient with a large number of unique variables with machine learning to recommend matches based on statistics and probability of success upon which practitioners can base medical recommendations.[xix] The AI is usually not establishing a causal relationship[xx] – it is predicting. So, as AI bleeds into medical devices, like brain-computer interfaces, the same cautions about using abductive reasoning alone should be carried over. II. Responsive Neurostimulation, AI, and Personalized Medicine Like precision medicine in cancer treatment, computer-brain interface technology similarly focuses on the individual patient through personalized settings. In order to properly expose the intersection of AI, precision medicine, abductive reasoning, and implantable neurotechnologies, the descriptions of adaptive deep brain stimulation systems need to deepen.[xxi] As a broad summary of adaptive deep brain stimulation, to provide a patient with the therapeutic stimulation, a neural signal, typically referred to as a local field potential,[xxii] must first be detected and then interpreted by the device. The main adaptive deep brain stimulation device with premarket approval, the NeuroPace Responsive Neurostimulation system, is used to treat epilepsy by detecting and storing “programmer-defined phenomena.”[xxiii] Providers can optimize the detection settings of the device to align with the patient’s unique electrographic seizures as well as personalize the reacting stimulation’s parameters.[xxiv] The provider adjusts the technology based on trial and error. One day machine learning algorithms will be able to regularly aid this process in myriad ways, such as by identifying the specific stimulation settings a patient may respond to ahead of time based on their electrophysiological signatures.[xxv] Either way, with AI or programmers, adaptive neurostimulation technologies are individualized and therefore operate in line with precision medicine rather than standard treatments based on large clinical trials. Contemporary neurostimulation devices are not usually sophisticated enough to be prominent in AI discussions where the topics of neural networks, deep learning, generative models, and self-attention dominate the conversation. However, implantable high-density electrocorticography arrays (a much more sensitive version than adaptive deep brain stimulation systems use) have been used in combination with neural networks to help patients with neurologic deficits from a prior stroke “speak” through a virtual avatar.[xxvi] In some experimental situations, algorithms are optimizing stimulation parameters with increasing levels of independence.[xxvii] An example of neurostimulation that is analogous to the use of nivolumab in Norway surrounds a patient in the United States who was experiencing both treatment-resistant OCD and temporal lobe epilepsy.[xxviii]Given the refractory nature of her epilepsy, implantation of an adaptive deep brain stimulation system was indicated. As a form of experimental therapy, her treatment-resistant OCD was also indicated for the off-label use of an adaptive deep brain stimulation set-up. Another deep brain stimulation lead, other than the one implanted for epilepsy, was placed in the patient’s right nucleus accumbens and ventral pallidum region given the correlation these nuclei had with OCD symptoms in prior research. Following this, the patient underwent “1) ambulatory, patient-initiated magnet-swipe storage of data during moments of obsessive thoughts; (2) lab-based, naturalistic provocation of OCD-related distress (naturalistic provocation task); and (3) lab-based, VR [virtual reality] provocation of OCD-related distress (VR provocation task).”[xxix] Such signals were used to identify when to deliver the therapeutic stimulation in order to counter the OCD symptoms. Thankfully, following the procedure and calibration the patient exhibited marked improvements in their OCD symptoms and recently shared her results publicly.[xxx] In both cases, there is a similar level of abductive justification for the efficacy of the delivered therapy. In the case study in which the patient was treated with adaptive deep brain stimulation, they at least had their neural activity tested in various settings to determine the optimum parameters for treatment to avoid them being based on guesswork. Additionally, the adaptive deep brain stimulation lead was already placed before the calibration trials were conducted, meaning that the patient had already taken on the bulk of the procedural risk before the efficacy could be determined. Such an efficacy test could have been replicated in the first patient’s cancer treatment, had it been biopsied and tested against the remaining immunotherapies in vitro. Yet, in the case of cancer with few options, one previous dose of a drug that appeared to work on the patient may justify further doses. However, as the Norwegian case presents, corroboration with known responses to a similar drug (from a clinical trial) could be helpful to validate the treatment strategy. (It should be noted that both patients were resigned to these last resort options regardless of the efficacy of treatment.) There are some elements of inductive logic seen with adaptive deep brain stimulation research in general. For example, abductively the focus could be that patient X’s stimulation parameters are different from patient Y’s and patient Z’s. In contrast, when grouped as subjects who obtained personalized stimulation, patients X, Y, and Z demonstrate an inductive aspect to this approach’s safety and/or efficacy. The OCD case holds plenty of abductive characteristics in line with precision medicine’s approach to treating cancer and as more individuals try the method, there will be additional data. With the gradual integration of AI into brain-computer interfaces in the name of efficacy, this reliance on abduction will continue, if not grow, over time. Moving forward, if a responsive deep brain stimulation treatment is novel and individualized (like the dose of nivolumab) and there is some other suggestion of efficacy (like clinical similarities to other patients in the literature), then it may justify insurance coverage for the investigative intervention, absent other unrelated reasons to deny it. III. Ethical Implications and Next Steps While AI’s use in oncology and neurology is not yet as prominent as its use in other fields (e.g., radiology), it appears to be on the horizon for both.[xxxi] AI can be found in both the functioning of the neurotechnologies as well as the implementation of precision medicine. The increasing use of AI may serve to further individualize both oncologic and neurological therapies. Given these implications and the handful of publications cited in this article, it is important to have a nuanced evaluation of how these treatments, which heavily rely on abductive justification, ought to be managed. The just use an abductive approach may be difficult as AI infused precision medicine is further pursued. At baseline, such technology relies on a level of advanced technology literacy among the general public and could exclude populations who lack access to basic technological infrastructure or know-how from participation.[xxxii] Even among nations with adequate infrastructure, as more patients seek out implantable neurotechnologies, which require robust healthcare resources, the market will favor patient populations that can afford this complex care.[xxxiii] If patients already have the means to pay for an initial dose/use of a precision medicine product out of pocket, should insurance providers be required to cover subsequent treatments?[xxxiv] That is, if a first dose of a cancer drug or a deep brain stimulator over its initial battery life is successful, patients may feel justified in having the costs of further treatments covered. The Norwegian patient’s experience implies there is a precedent for the idea that some public insurance companies ought to cover successful cancer therapies, however, insurance companies may not all see themselves as obligated to cover neurotechnologies that rely on personalized settings or that are based on precision/abductive research more than on clinical trials. CONCLUSION The fact that the cases outlined above rely on abductive style of reasoning implies that there may not be as strong a justification for coverage by insurance, as they are both experimental and individualized, when compared to the more traditional large clinical trials in which groups have the same or a standardized protocol (settings/doses). If a study is examining the efficacy of a treatment with a large cohort of patients or with different experimental groups/phases, insurance companies may conclude that the resulting symptom improvements are more likely to be coming from the devices themselves. A preference for inductive justification may take priority when ruling in favor of funding someone’s continued use of an implantable neurostimulator. There are further nuances to this discussion surrounding the classifications of these interventions as research versus clinical care that warrant future exploration, since such a distinction is more of a scale[xxxv] than binary and could have significant impacts on the “right-to-try” approach to experimental therapies in the United States.[xxxvi] Namely, given the inherent limitations of conducting large cohort trials for deep brain stimulation interventions on patients with neuropsychiatric disorders, surgically innovative frameworks that blend abductive and inductive methodologies, like with sham stimulation phases, have traditionally been used.[xxxvii] Similarly, for adaptive brain-computer interface systems, if there are no large clinical trials and instead only publications that demonstrate that something similar worked for someone else, then, in addition to the evidence that the first treatment/dose worked for the patient in question, the balance of reasoning would be valid and arguably justify insurance coverage. As precision approaches to neurotechnology become more common, frameworks for evaluating efficacy will be crucial both for insurance coverage and for clinical decision making. ACKNOWLEDGEMENT This article was originally written as an assignment for Dr. Francis Shen’s “Bioethics & AI” course at Harvard’s Center for Bioethics. I would like to thank Dr. Shen for his comments as well as my colleagues in the Lázaro-Muñoz Lab for their feedback. - [i] Jonathan Kimmelman and Ian Tannock, “The Paradox of Precision Medicine,” Nature Reviews. Clinical Oncology 15, no. 6 (June 2018): 341–42, https://doi.org/10.1038/s41571-018-0016-0. [ii] Henrik Vogt and Bjørn Hofmann, “How Precision Medicine Changes Medical Epistemology: A Formative Case from Norway,” Journal of Evaluation in Clinical Practice 28, no. 6 (December 2022): 1205–12, https://doi.org/10.1111/jep.13649. [iii] David Barrett and Ahtisham Younas, “Induction, Deduction and Abduction,” Evidence-Based Nursing 27, no. 1 (January 1, 2024): 6–7, https://doi.org/10.1136/ebnurs-2023-103873. [iv] Vogt and Hofmann, “How Precision Medicine Changes Medical Epistemology,” 1208. [v] Wireko Andrew Awuah et al., “Bridging Minds and Machines: The Recent Advances of Brain-Computer Interfaces in Neurological and Neurosurgical Applications,” World Neurosurgery, May 22, 2024, S1878-8750(24)00867-2, https://doi.org/10.1016/j.wneu.2024.05.104. [vi] Mark Riedl, “A Very Gentle Introduction to Large Language Models without the Hype,” Medium (blog), May 25, 2023, https://mark-riedl.medium.com/a-very-gentle-introduction-to-large-language-models-without-the-hype-5 f67941fa59e. [vii] David E. Burdette and Barbara E. Swartz, “Chapter 4 - Responsive Neurostimulation,” in Neurostimulation for Epilepsy, ed. Vikram R. Rao (Academic Press, 2023), 97–132, https://doi.org/10.1016/B978-0-323-91702-5.00002-5. [viii] Kimmelman and Tannock, 2018. [ix] Kimmelman and Tannock, 2018. [x] Simon Lohse, “Mapping Uncertainty in Precision Medicine: A Systematic Scoping Review,” Journal of Evaluation in Clinical Practice 29, no. 3 (April 2023): 554–64, https://doi.org/10.1111/jep.13789. [xi] Kimmelman and Tannock, “The Paradox of Precision Medicine.” [xii] Vogt and Hofmann, 1206. [xiii] Vogt and Hofmann, 1206. [xiv] Vogt and Hofmann, 1206. [xv] Vogt and Hofmann, 1207. [xvi] Vogt and Hofmann, 1207. [xvii] Vogt and Hofmann, 1207. [xviii] Vogt and Hofmann, 1210. [xix] Mehar Sahu et al., “Chapter Three - Artificial Intelligence and Machine Learning in Precision Medicine: A Paradigm Shift in Big Data Analysis,” in Progress in Molecular Biology and Translational Science, ed. David B. Teplow, vol. 190, 1 vols., Precision Medicine (Academic Press, 2022), 57–100, https://doi.org/10.1016/bs.pmbts.2022.03.002. [xx] Stefan Feuerriegel et al., “Causal Machine Learning for Predicting Treatment Outcomes,” Nature Medicine 30, no. 4 (April 2024): 958–68, https://doi.org/10.1038/s41591-024-02902-1. [xxi] Sunderland Baker et al., “Ethical Considerations in Closed Loop Deep Brain Stimulation,” Deep Brain Stimulation 3 (October 1, 2023): 8–15, https://doi.org/10.1016/j.jdbs.2023.11.001. [xxii] David Haslacher et al., “AI for Brain-Computer Interfaces,” 2024, 7, https://doi.org/10.1016/bs.dnb.2024.02.003. [xxiii] Burdette and Swartz, “Chapter 4 - Responsive Neurostimulation,” 103–4; “Premarket Approval (PMA),” https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpma/pma.cfm?id=P100026. [xxiv] Burdette and Swartz, “Chapter 4 - Responsive Neurostimulation,” 104. [xxv] Burdette and Swartz, 126. [xxvi] Sean L. Metzger et al., “A High-Performance Neuroprosthesis for Speech Decoding and Avatar Control,” Nature 620, no. 7976 (August 2023): 1037–46, https://doi.org/10.1038/s41586-023-06443-4. [xxvii] Hao Fang and Yuxiao Yang, “Predictive Neuromodulation of Cingulo-Frontal Neural Dynamics in Major Depressive Disorder Using a Brain-Computer Interface System: A Simulation Study,” Frontiers in Computational Neuroscience 17 (March 6, 2023), https://doi.org/10.3389/fncom.2023.1119685; Mahsa Malekmohammadi et al., “Kinematic Adaptive Deep Brain Stimulation for Resting Tremor in Parkinson’s Disease,” Movement Disorders 31, no. 3 (2016): 426–28, https://doi.org/10.1002/mds.26482. [xxviii] Young-Hoon Nho et al., “Responsive Deep Brain Stimulation Guided by Ventral Striatal Electrophysiology of Obsession Durably Ameliorates Compulsion,” Neuron 0, no. 0 (October 20, 2023), https://doi.org/10.1016/j.neuron.2023.09.034. [xxix] Nho et al. [xxx] Nho et al.; Erik Robinson, “Brain Implant at OHSU Successfully Controls Both Seizures and OCD,” OHSU News, accessed March 3, 2024, https://news.ohsu.edu/2023/10/25/brain-implant-at-ohsu-successfully-controls-both-seizures-and-ocd. [xxxi] Awuah et al., “Bridging Minds and Machines”; Haslacher et al., “AI for Brain-Computer Interfaces.” [xxxii] Awuah et al., “Bridging Minds and Machines.” [xxxiii] Sara Green, Barbara Prainsack, and Maya Sabatello, “The Roots of (in)Equity in Precision Medicine: Gaps in the Discourse,” Personalized Medicine 21, no. 1 (January 2024): 5–9, https://doi.org/10.2217/pme-2023-0097. [xxxiv] Green, Prainsack, and Sabatello, 7. [xxxv] Robyn Bluhm and Kirstin Borgerson, “An Epistemic Argument for Research-Practice Integration in Medicine,” The Journal of Medicine and Philosophy: A Forum for Bioethics and Philosophy of Medicine 43, no. 4 (July 9, 2018): 469–84, https://doi.org/10.1093/jmp/jhy009. [xxxvi] Vijay Mahant, “‘Right-to-Try’ Experimental Drugs: An Overview,” Journal of Translational Medicine 18 (June 23, 2020): 253, https://doi.org/10.1186/s12967-020-02427-4. [xxxvii] Michael S. Okun et al., “Deep Brain Stimulation in the Internal Capsule and Nucleus Accumbens Region: Responses Observed during Active and Sham Programming,” Journal of Neurology, Neurosurgery & Psychiatry 78, no. 3 (March 1, 2007): 310–14, https://doi.org/10.1136/jnnp.2006.095315. (shrink)

Multi-level institutional designs with distributed power and authority arrangements among federal, state, regional, and local government agencies could lead to the emergence of differential patterns of socioeconomic and infrastructure development pathways in complex social–ecological systems. Both exogenous drivers and endogenous processes in social–ecological systems can lead to changes in the number of “basins of attraction,” changes in the positions of the basins within the state space, and changes in the positions of the thresholds between basins. In an effort (...) to advance the theory and practice of the governance of policy systems, this study addresses a narrower empirical question: how do intergovernmental institutional rules set by federal, state, and regional government agencies generate and sustain basins of attraction in funding infrastructure projects? A pattern-oriented, agent-based model (ABM) of an intergovernmental network has been developed to simulate real-world transportation policy implementation processes across the federal, the state of Vermont, regional, and local governments for prioritizing transportation projects. The ABM simulates baseline and alternative intergovernmental institutional rule structures and assesses their impacts on financial investment flows. The ABM was calibrated with data from multiple focus groups, individual interviews, and analysis of federal, state, and regional scale transportation projects and programs. The results from experimental simulations are presented to test system-wide effects of alternative multi-level institutional designs, in particular different power and authority arrangements between state and regional governments, on the emergence of roadway project prioritization patterns and funding allocations across regions and towns. (shrink)

Decision making often takes place in social environments where other actors influence individuals' decisions. The present article examines how advice affects individual learning. Five social learning models combining advice and individual learning‐four based on reinforcement learning and one on Bayesian learning‐and one individual learning model are tested against each other. In two experiments, some participants received good or bad advice prior to a repeated multioption choice task. Receivers of advice adhered to the advice, so that good (...) advice improved performance. The social learning models described the observed learning processes better than the individual learning model. Of the models tested, the best social learning model assumes that outcomes from recommended options are more positively evaluated than outcomes from nonrecommended options. This model correctly predicted that receivers first adhere to advice, then explore other options, and finally return to the recommended option. The model also predicted accurately that good advice has a stronger impact on learning than bad advice. One‐time advice can have a long‐lasting influence on learning by changing the subjective evaluation of outcomes of recommended options. (shrink)

Theories of decision-making and its neural substrates have long assumed the existence of two distinct and competing valuation systems, variously described as goal-directed vs. habitual, or, more recently and based on statistical arguments, as model-free vs. model-based reinforcement-learning. Though both have been shown to control choices, the cognitive abilities associated with these systems are under ongoing investigation. Here we examine the link to cognitive abilities, and find that individual differences in processing speed covary with (...) a shift from model-free to model-based choice control in the presence of above-average working memory function. This suggests shared cognitive and neural processes; provides a bridge between literatures on intelligence and valuation; and may guide the development of process models of different valuation components. Furthermore, it provides a rationale for individual differences in the tendency to deploy valuation systems, which may be important for understanding the manifold neuropsychiatric diseases associated with malfunctions of valuation. (shrink)

The third volume of “Advancing Uncertain Combinatorics through Graphization, Hyperization, and Uncertainization: Fuzzy, Neutrosophic, Soft, Rough, and Beyond” presents an in-depth exploration of the cutting-edge developments in uncertain combinatorics and set theory. This comprehensive collection highlights innovative methodologies such as graphization, hyperization, and uncertainization, which enhance combinatorics by incorporating foundational concepts from fuzzy, neutrosophic, soft, and rough set theories. These advancements open new mathematical horizons, offering novel approaches to managing uncertainty within complex systems. Combinatorics, a discipline focused on (...) counting, arrangement, and structure, often faces challenges when uncertainty is present. Set theory, which underpins combinatorial problems, has evolved to tackle these challenges. The introduction of fuzzy and neutrosophic sets has expanded the toolkit for modeling uncertainty by incorporating elements of truth, indeterminacy, and falsehood into decision-making processes. These innovations seamlessly intersect with graph theory, providing new ways to represent uncertain structures through "graphized" forms such as hypergraphs and superhypergraphs. This volume also introduces advanced concepts like Neutrosophic Oversets, Undersets, and Offsets, which push the boundaries of classical graph theory and offer deeper insights into the mathematical and practical challenges posed by real-world systems. By blending combinatorics, set theory, and graph theory, the authors have created a robust framework for addressing uncertainty in both mathematical systems and their real-world applications. This foundation sets the stage for future breakthroughs in combinatorics, set theory, and related fields. Each chapter in this volume contributes both theoretical foundations and practical applications, demonstrating the power of integrating graph theory, set theory, and uncertainty models. The new ideas, algorithms, and mathematical tools presented here will drive the future of combinatorial research and its applications in uncertain environments. In the first chapter, “Introduction to Upside-Down Logic: Its Deep Relation to Neutrosophic Logic and Applications”, the authors present Upside-Down Logic, a novel logical framework that systematically transforms truths into falsehoods and vice versa, based on contextual shifts. Introduced by F. Smarandache, this paper provides a mathematical definition of Upside-Down Logic, including applications related to the Japanese language. The chapter also introduces Contextual Upside-Down Logic, an extension that adjusts logical connectives alongside flipped truth values, as well as Indeterm-Upside-Down Logic and Certain Upside-Down Logic to address indeterminacy. A simple algorithm is also proposed to demonstrate the computational aspects of this logic. In the second chapter, “Local-Neutrosophic Logic and Local-Neutrosophic Sets: Incorporating Locality with Applications”, the authors introduce Local-Neutrosophic Logic and Local-Neutrosophic Sets, which integrate the concept of locality into Neutrosophic Logic. By defining locality as the influence of immediate surroundings on an object or system, this chapter explores how it affects indeterminacy in real-world problems. The paper also examines potential applications and provides mathematical definitions for these new concepts. The third chapter, “A Review of Fuzzy and Neutrosophic Offsets: Connections to Some Set Concepts and Normalization Function”, extends the concept of offsets in uncertain set-theoretic frameworks, such as Fuzzy Sets, Neutrosophic Sets, and Plithogenic Sets. This chapter introduces several advanced types of offsets, including Nonstationary Fuzzy Offset, Multi-valued Plithogenic Offset, and Subset-valued Neutrosophic Offset, offering deeper insights into handling uncertainty in mathematical models. In the fourth chapter, “Review of Plithogenic Directed, Mixed, Bidirected, and Pangene OffGraph”, the authors build upon Plithogenic Graphs to propose extensions such as Plithogenic Directed OffGraph, Plithogenic BiDirected OffGraph, and Plithogenic Mixed OffGraph. These new concepts, including the Plithogenic Pangene OffGraph, are explored in detail, with a focus on their mathematical properties and potential applications in uncertain graph theory. The fifth chapter, “Short Note on Neutrosophic Closure Matroids”, explores the extension of matroid concepts into Neutrosophic and Turiyam Neutrosophic set theories, introducing Neutrosophic closure matroids. This concept integrates uncertainty, indeterminacy, and liberal states into matroid theory, enhancing its applicability in optimization and combinatorial problems. In the sixth chapter, “Some Graph Parameters for Superhypertree-width and Neutrosophic Tree-width”, the authors discuss graph parameters such as Superhypertree-width and Neutrosophic tree-width. These parameters play a crucial role in the study of graph characteristics, particularly in algorithms and real-world applications. The chapter explores the generalization of hypergraphs to SuperHyperGraphs and examines how these concepts extend tree-width parameters within the context of Neutrosophic logic. (shrink)

The human masticatory system is a complex functional unit characterized by a multitude of skeletal components, muscles, soft tissues, and teeth. Muscle activation dynamics cannot be directly measured on live human subjects due to ethical, safety, and accessibility limitations. Therefore, estimation of muscle activations and their resultant forces is a longstanding and active area of research. Reinforcement learning (RL) is an adaptive learning strategy which is inspired by the behavioral psychology and enables an agent to learn the (...) dynamics of an unknown system via policy-driven explorations. The RL framework is a well-formulated closed-loop system where high capacity neural networks are trained with the feedback mechanism of rewards to learn relatively complex actuation patterns. In this work, we are building on a deep RL algorithm, known as the Soft Actor-Critic, to learn the inverse dynamics of a simulated masticatory system, i.e., learn the activation patterns that drive the jaw to its desired location. The outcome of the proposed training procedure is a parametric neural model which acts as the brain of the biomechanical system. We demonstrate the model’s ability to navigate the feasible three-dimensional (3D) envelope of motion with sub-millimeter accuracies. We also introduce a performance analysis platform consisting of a set of quantitative metrics to assess the functionalities of a given simulated masticatory system. This platform assesses the range of motion, metabolic efficiency, the agility of motion, the symmetry of activations, and the accuracy of reaching the desired target positions. We demonstrate how the model learns more metabolically efficient policies by integrating a force regularization term in the RL reward. We also demonstrate the inverse correlation between the metabolic efficiency of the models and their agility and range of motion. The presented masticatory model and the proposed RL training mechanism are valuable tools for the analysis of mastication and other biomechanical systems. We see this framework’s potential in facilitating the functional analyses aspects of surgical treatment planning and predicting the rehabilitation performance in post-operative subjects. (shrink)

Prior research has used an Interactive Landslide Simulator tool to investigate human decision making against landslide risks. It has been found that repeated feedback in the ILS tool about damages due to landslides causes an improvement in human decisions against landslide risks. However, little is known on how theories of learning from feedback would account for human decisions in the ILS tool. The primary goal of this paper is to account for human decisions in the ILS tool via computational (...) models based upon reinforcement learning and to explore the model mechanisms involved when people make decisions in the ILS tool. Four different reinforcement-learning models were developed and evaluated in their ability to capture human decisions in an experiment involving two conditions in the ILS tool. The parameters of an Expectancy-Valence model, two Prospect-Valence-Learning models, a combination EV-PU model, and a random model were calibrated to human decisions in the ILS tool across the two conditions. Later, different models with their calibrated parameters were generalized to data collected in an experiment involving a new condition in ILS. When generalized to this new condition, the PVL-2 model’s parameters of both damage-feedback conditions outperformed all other RL models. We highlight the implications of our results for decision making against landslide risks. (shrink)

Making proper decisions in today’s complex world is a challenging task for decision makers. A promising approach that can support decision makers to have a better understanding of complex systems is agent-based modeling. ABM has been developing during the last few decades as a methodology with many different applications and has enabled a better description of the dynamics of complex systems. However, the prescriptive facet of these applications is rarely portrayed. Adding a prescriptive (...) decision-making aspect to ABM can support the decision makers in making better or, in some cases, optimized decisions for the complex problems as well as explaining the investigated phenomena. In this paper, first, the literature of DM with ABM is inquired and classified based on the methods of integration. Performing a scientometric analysis on the relevant literature lets us conclude that the number of publications attempting to integrate DM and ABM has not grown during the last two decades, while analysis of the current methodologies for integrating DM and ABM indicates that they have serious drawbacks. In this regard, a novel nature-inspired model articulation called optimal agent framework has been proposed to ameliorate the disadvantages and enhance the realization of proper decisions in ABM at a relatively low computational cost. The framework is examined with the Bass diffusion model. The results of the simulation for the customized model developed by OAF have verified the feasibility of the framework. Moreover, sensitivity analyses on different agent populations, network structures, and marketing strategies have depicted the great potential of OAF to find the optimal strategies in various stochastic and unconventional conditions which have not been addressed prior to the implementation of the framework. (shrink)

The behaviour of a multi-agent system is driven by messaging. Usually, there is no central dispatcher and each autonomous agent, though resource-bounded, can make less or more rational decisions to meet its own and collective goals. To this end, however, agents must communicate with their fellow agents and account for the signals from their environment. Moreover, in the dynamic, permanently changing world, agents’ behaviour, i.e. their activities, must also be dynamic. By communicating with other fellow (...) agents and with their environment, agents should be able to learn new concepts and enrich their knowledge base. Processes and events that happened in the past may be irrelevant in the present or have a significant impact in the future, and vice versa. Therefore, the fine-grained analysis of agents’ activities as well as events within or beyond the system is very important so that the system can run smoothly without falling into inconsistencies. Moreover, as the system should communicate with its environment, the analysis should be as close to natural language as possible. The goal of this paper is a proposal for such an analysis. To this end, I apply Transparent Intensional Logic (TIL) because TIL is particularly apt for a fine-grained analysis of processes and events specified in the present, past or future tense with reference to the time when they happened, happen or will happen. (shrink)

Beyond the observation that both speakers and listeners rapidly inspect the visual targets of referring expressions, it has been argued that such gaze may constitute part of the communicative signal. In this study, we investigate whether a speaker may, in principle, exploit listener gaze to improve communicative success. In the context of a virtual environment where listeners follow computer-generated instructions, we provide two kinds of support for this claim. First, we show that listener gaze provides a reliable real-time (...) index of understanding even in dynamic and complex environments, and on a per-utterance basis. Second, we show that a language generation system that uses listener gaze to provide rapid feedback improves overall task performance in comparison with two systems that do not use gaze. Aside from demonstrating the utility of listener gaze in situated communication, our findings open the door to new methods for developing and evaluating multi-modal models of situated interaction. (shrink)

Decision making in noisy and changing environments requires a fine balance between exploiting knowledge about good courses of action and exploring the environment in order to improve upon this knowledge. We present an experiment on a restless bandit task in which participants made repeated choices between options for which the average rewards changed over time. Comparing a number of computational models of participants’ behavior in this task, we find evidence that a substantial number of them balanced exploration (...) and exploitation by considering the probability that an option offers the maximum reward out of all the available options. (shrink)

In a multiagent and multi-cultural world, the fine-grained analysis of agents’ dynamic behaviour, i.e. of their activities, is essential. Dynamic activities are actions that are characterized by an agent who executes the action and by other participants of the action. Wh-questions on the participants of the actions pose a difficult particular challenge because the variability of the types of possible answers to such questions is huge. To deal with the problem, we propose the analysis and classification (...) of Wh-questions apt for agents’ communication in a multiagent system (MAS). Our proposal of such a system consists of agents who communicate with their fellow agents by messaging so that each autonomous agent, though resource-bounded, can make less or more rational decisions to meet its own and collective goals. In addition, by communicating with other fellow agents and their environment, agents can learn new concepts and enrich their ontology so that their behaviour is dynamic. We aim to make a general proposal of the system so that the ‘envelope’ of agents’ messages can be formalized in any MAS standard, be it The Foundation for Intelligent Physical Agents - Agent Communication Language (FIPA-ACL) or Knowledge Query and Manipulation Language (KQML). Yet, the content of messages is encoded in a formalized natural language. To this end, we apply Transparent Intensional Logic (TIL) with its procedural semantics which is particularly apt for a fine-grained analysis in which all the semantically salient features of natural language can be plausibly formalized. In this paper, we concentrate on analysing the content of query messages, particularly the content of those that encode Wh-questions and the answers to them. We also summarize TIL deduction system that makes it possible to answer such questions in an intelligent way. Linguists distinguish several subtypes of Wh-questions. Though the linguistic classification is helpful, it is not always suitable for agents’ communication and reasoning. We need the classification based on a logical analysis of Wh-questions so that the agents can infer possible answers to such questions rather than only looking for them by keywords. This paper aims to apply an appropriate classification of the logical types of Wh-questions and the analysis of such questions; we concentrate in particular on questions concerning the participants of activities. The application of these results to the analysis of processes and events based on verb valency frames is another novelty of the paper. (shrink)

With the advent of the Big Data era, information and data are growing in spurts, fueling the deep application of information technology in all levels of society. It is especially important to use data mining technology to study the industry trends behind the data and to explore the information value contained in the massive data. As teaching and learning in higher education continue to advance, student academic and administrative data are growing at a rapid pace. In this paper, we (...) make full use of student academic data and campus behavior data to analyze the data inherent patterns and correlations and use these patterns rationally to provide guidance for teaching activities and teaching management, thus further improving the quality of teaching management. The establishment of a data-mining-technology-based college repetition warning system can help student management departments to strengthen supervision, provide timely warning information for college teaching management as well as leaders and counselors’ decision-making, and thus provide early help to students with repetition warnings. In this paper, we use the global search advantage of genetic algorithm to build a GABP hybrid prediction model to solve the local minimum problem of BP neural network algorithm. The data validation results show that Recall reaches 95% and F1 result is about 86%, and the accuracy of the algorithm prediction results is improved significantly. It can provide a solid data support basis for college administrators to predict retention. Finally, the problems in the application of the retention prediction model are analyzed and corresponding suggestions are given. (shrink)

Recent studies in pedestrian simulation have been able to construct a highly realistic navigation behaviour in many circumstances. However, when replicating the close interactions between pedestrians, the replicated behaviour is often unnatural and lacks human likeness. One of the possible reasons is that the current models often ignore the cognitive factors in the human thinking process. Another reason is that many models try to approach the problem by optimising certain objectives. On the other hand, in real life, humans do (...) not always take the most optimised decisions, particularly when interacting with other people. To improve the navigation behaviour in this circumstance, we proposed a pedestrian interacting model using reinforcement learning. Additionally, a novel cognitive prediction model, inspired by the predictive system of human cognition, is also incorporated. This helps the pedestrian agent in our model to learn to interact and predict the movement in a similar practice as humans. In our experimental results, when compared to other models, the path taken by our model’s agent is not the most optimised in certain aspects like path lengths, time taken and collisions. However, our model is able to demonstrate a more natural and human-like navigation behaviour, particularly in complex interaction settings. (shrink)

This paper addresses weighting and partitioning in complex reinforcement learning tasks, with the aim of facilitating learning. The paper presents some ideas regarding weighting of multiple agents and extends them into partitioning an input/state space into multiple regions with di erential weighting in these regions, to exploit di erential characteristics of regions and di erential characteristics of agents to reduce the learning complexity of agents (and their function approximators) and thus to facilitate the learning overall. It analyzes, in (...) reinforcement learning tasks, di erent ways of partitioning a task and using agents selectively based on partitioning. Based on the analysis, some heuristic methods are described and experimentally tested. We nd that some o -line heuristic methods performed the best, signi cantly better than single-agent models. (shrink)

Real causal systems are complicated. Despite this, causal learning research has traditionally emphasized how causal relations can be induced on the basis of idealized events, i.e. those that have been mapped to binary variables and abstracted from time. For example, participants may be asked to assess the efficacy of a headache-relief pill on the basis of multiple patients who take the pill (or not) and find their headache relieved (or not). In contrast, the current study examines learning via (...) interactions with continuous dynamic systems, systems that include continuous variables that interact over time (and that can be continuously observed in real time by the learner). To explore such systems, we develop a new framework that represents a causal system as a network of stationary Gauss--Markov (``\OU'') processes and show how such \textit{OU networks} can express complex dynamic phenomena such as feedback loops and oscillations. To assess adult's abilities to learn such systems, we conducted an experiment in which participants were asked to identify the causal relationships of a number of OU networks, potentially carrying out multiple, temporally-extended interventions. We compared their judgments to a normative model for learning OU networks as well as a range of alternative and heuristic learning models from the literature. We found that, although participants exhibited substantial learning of such systems, they committed certain systematic errors. These successes and failures were best accounted for by a model that describes people as focusing on pairs of variables, rather than evaluating the evidence with respect to the full space of possible structural models. We argue that our approach provides both a principled framework for exploring the space of dynamic learning environments as well as new algorithmic insights into how people interact successfully with a continuous causal world. (shrink)

Bipolar neutrosophic sets are the extension of neutrosophic sets and are based on the idea of positive and negative preferences of information. Projection measure is a useful apparatus for modelling real life decision making problems. In the paper, we define projection, bidirectional projection and hybrid projection measures between bipolar neutrosophic sets. Three new methods based on the proposed projection measures are developed for solving multi-attribute decision making problems. In the solution process, the ratings of (...) performance values of the alternatives with respect to the attributes are expressed in terms of bipolar neutrosophic values. We calculate projection, bidirectional projection, and hybrid projection measures between each alternative and ideal alternative with bipolar neutrosophic information. All the alternatives are ranked to identify the best alternative. Finally, a numerical example is provided to demonstrate the applicability and effectiveness of the developed methods. Comparison analysis with the existing methods in the literature in bipolar neutrosophic environment is also performed. (shrink)

This paper presents a learning theory pertinent to dynamic decision making (DDM) called instancebased learning theory (IBLT). IBLT proposes five learning mechanisms in the context of a decision‐making process: instance‐based knowledge, recognition‐based retrieval, adaptive strategies, necessity‐based choice, and feedback updates. IBLT suggests in DDM people learn with the accumulation and refinement of instances, containing the decision‐making situation, action, and utility of decisions. As decision makers interact with a dynamic task, they recognize (...) a situation according to its similarity to past instances, adapt their judgment strategies from heuristic‐based to instance‐based, and refine the accumulated knowledge according to feedback on the result of their actions. The IBLT's learning mechanisms have been implemented in an ACT‐R cognitive model. Through a series of experiments, this paper shows how the IBLT's learning mechanisms closely approximate the relative trend magnitude and performance of human data. Although the cognitive model is bounded within the context of a dynamic task, the IBLT is a general theory of decision making applicable to other dynamic environments. (shrink)

This paper presents a human-computer interaction model with a three layers learning mechanism in a pervasive environment. We begin with a discussion around a number of important issues related to human-computer interaction followed by a description of the architecture for a multi-agent cooperative design system for pervasive computing environment. We present our proposed three- layer HCI model and introduce the group formation algorithm, which is predicated on a dynamic sharing niche technology. Finally, we explore the (...) cooperative reinforcement learning and fusion algorithms; the paper closes with concluding observations and a summary of the principal work and contributions of this paper. (shrink)

This book deals with the theoretical, methodological, and empirical implications of bounded rationality in the operation of institutions. It focuses on decisions made under uncertainty, and presents a reliable strategy of knowledge acquisition for the design and implementation of decision-support systems. Based on the distinction between the inner and outer environment of decisions, the book explores both the cognitive mechanisms at work when actors decide, and the institutional mechanisms existing among and within organizations that make decisions fairly predictable. (...) While a great deal of work has been done on how organizations act as patterns of events for (boundedly) rational decisions, less effort has been devoted to study under which circumstances organizations cease to act as such reliable mechanisms. Through an empirical strategy on open-ended response data from a survey among junior judges, the work pursues two main goals. The first one is to explore the limits of "institutional rationality" of the Spanish lower courts on-call service, an optimal scenario to observe decision-making under uncertainty. The second aim is to achieve a better understanding of the kind of uncertainty under which inexperienced decision-makers work. This entails exploring the demands imposed by problems and the knowledge needed to deal with them, making this book also a study on expertise achievement in institutional environments. This book combines standard multivariate statistical methods with machine learning techniques such as multidimensional scaling and topic models, treating text as data. Doing so, the book contributes to the collaboration between empirical social scientific approaches and the community of scientists that provide the set of tools and methods to make sense of the fastest growing resource of our time: data. (shrink)

The paper explores the basis for decision?making and policy with regard to the Environment. Clearly these should be based on knowledge of possible consequences and accompanying risk assessments involving the linked behaviour of the many interacting human actors within a socio?economic system and the ecological, and physical systems in which they are embedded. The paper describes the Complex Systems approach to these problems, showing the kind of models that are required in order to obtain whatever limited knowledge (...) is possible about the co?evolution of the human and environmental systems involved. Several practical examples are described and the models briefly presented. These are shown as examples of what should be required for the creation of the necessary basis for making policy and decision explorations with an integrated view of the system as a whole, instead of separate parts studied in detail by experts of specific disciplines. This provides a framework for making real use of the ?knowledge? of disciplinary experts, and linking their narrow views to the overall, practical consequences in the real world of possible policy options. (shrink)

Individuals make decisions under uncertainty every day based on incomplete information concerning the potential outcome of the choice or chance levels. The choices individuals make often deviate from the rational or mathematically objective solution. Accordingly, the dynamics of human decision-making are difficult to capture using conventional, linear mathematical models. Here, we present data from a two-choice task with variable risk between sure loss and risky loss to illustrate how a simple nonlinear dynamical system can be employed to capture (...) the dynamics of human decision-making under uncertainty (i.e., multi-stability, bifurcations). We test the feasibility of this model quantitatively and demonstrate how the model can account for up to 86% of the observed choice behavior. The implications of using dynamical models for explaining the nonlinear complexities of human decision-making are discussed, as well as the degree to which nonlinear dynamical systems theory might offer an alternative framework for understanding human decision-making processes. (shrink)

Universal Basic Income (UBI) has been proposed as a potential way in which welfare states could be made more responsive to the ever-shifting evolutionary challenges of institutional adaptation in a dynamic environment. It has been proposed as a tool of “real freedom” (Van Parijs) and as a tool of making the welfare state more efficient. (Friedman) From the point of view of complexity theory and evolutionary economics, I argue that only a welfare state model that is “polycentrically” (...) (Polanyi, Hayek) organized as an evolving network of distributed decision-making can respond to the challenges of complexity in an adaptively efficient manner. UBI has the theoretical possibility to facilitate flexible, bottom-up innovation, since a) it embodies the “rule of law” principles of generality, nondiscrimination, simplicity, and transparency, and b) it grants people widespread freedom to experiment, innovate, and deviate from established practices and norms; but whether it can be made to work as intended depends on a number of variables in its design. I situate my research in the contemporary UBI debate around automation and technological development, which I interpret creatively through the lens of contemporary evolutionary political economy. I advance a syncretic model of institutional design based upon the shared insights of the Santa Fe, Neo-Hayekian, and Neo-Schumpeterian schools of evolutionary economics that, following Adam Thierer and Michael Munger, I call the framework of Permissionless Innovation (PI). Under the PI framework, which is a development of the tradition of evolutionary liberalism (Hume, Smith, Mandeville, Mill, Hayek, Polanyi, Schumpeter, Hodgson), individuals are granted the default right to innovate without having to ask for permission, and the right to basic income, which they can use to support their right to innovate, mutate, and experiment. This framework is a liberal framework to the extent it relies on widespread human freedom as a means of fostering innovation and social learning. And it is an evolutionary framework to the extent that it seeks to facilitate evolutionary socioeconomic processes for the sake of social progress and welfare advancement. I proceed to show, through an analysis of how innovation operates in both the economy and the cultural domain, that UBI can be used as an important cornerstone of an evolutionary liberal model of “ecostructural” governance that treats the socioeconomic order as an ecological garden of spontaneous growths, adaptations, and innovations. I call this the Permissionless Innovation Universal Basic Income (PIUBI) framework. In it, the right to innovate, deviate, and mutate is upheld as the “gold standard” of institutional design whereby individuals are granted widespread freedoms combined with the provision of UBI and other innovation-fostering (carefully bound) services and regulations. In the coming decades, UBI may even enable poor people to take better advantage of Human Enhancement Technologies (HETs) and other experimental innovations. Such a framework is compatible with a fast pace of evolutionary development, but the regulatory model accompanying such a framework must be capable of responding to evolutionary lock-ins, maladaptive innovations, self-destructive behaviour on the part of UBI recipients, and the management of catastrophic and existential risks. In the end, I will have shown that the PIUBI framework can be justified as a plausible default institutional mechanism for responding to radical uncertainty, but it needs to be experimentally adapted to changing circumstances with the help of various situation-specific, polycentric, multi-level, approaches. (shrink)

Cognitive models that represent individuals provide many benefits for understanding the full range of human behavior. One way in which individual differences emerge is through differences in knowledge. In dynamic situations, where decisions are made from experience, models built upon a theory of experiential choice (instance-based learning theory; IBLT) can provide accurate predictions of individual human learning and adaptivity to changing environments. Here, we demonstrate how an instance-based learning (IBL) cognitive model, implemented in a cognitive architecture (...) (Adaptive Control of Thought–Rational), can be used to model an individual's decisions in a cybersecurity defense task, accounting for both population average and individual variances. The same IBL model structure with identical architectural parameters generates the full range of human behavior through stochastic memory retrieval processes operating over and contributing to unique experiences. Recurrence quantification analyses allow us to look beyond average behavior between and within individuals to sequential patterns of trial-to-trial behavior. We show how model-tracing and knowledge-tracing techniques can be used to align the model to an individual in real time to drive adaptive and personalized signaling algorithms for a cybersecurity defense system. We also present a method for introspecting into the cognitive model to gain further insight into the cognitive salience of features factored into individual decisions. The combination of techniques provides a blueprint for personalized modeling of individuals. We discuss the results and implications of this adaptive and personalized method for cybersecurity defense and more generally for intelligent artifacts tailored to individual differences in domains such as human–machine teaming. (shrink)

This work presents a post-positivist research framework for explaining any surprising or anomalous fact in the evolutionary path of a complex, dynamic, and contingent social process. Firstly, it elaborates on the reconciliation betweenthe ontological and epistemological assumptions of Critical Realism with the principles of American Pragmatism. Next, the research method is presented: theoretical propositions about a social structure are translated into a set of grammar rules that acknowledge patterns of sequences of events, either involving individual action or interaction (...) between actors within a real social system. The result is a discrete mathematical model for a concrete category of social process based on these rules. Finally, data-grounded refinement of the theory is possible through comparison between cases belonging to the same category, but differing in some contingent pattern of sequences of event outcomes. Consequently, their grammars differ in some pairs of context-sensitive rules that explain this surprising fact, and the derivation of this alternative historical trajectory of event outcomes becomes an extension to the early category of social process. In this sense, the proposed framework suggests a hierarchy of classes of grammars for middle-range explanations based on the ontological assumption of the generative nature of social reality. (shrink)

This article describes and evaluates an approach to create and/or improve tactical agents through direct human interaction in real time through a force-feedback haptic device. This concept takes advantage of a force-feedback joystick to enhance motor skill and decision-making transfer from the human to the agent in real time. Haptic devices have been shown to have high bandwidth and sensitivity. Experiments are described for this new approach, named Instructional Learning. It is used both as (...) a way to build agents from scratch as well as to improve and/or correct agents built through other means. The approach is evaluated through experiments that involve three applications of increasing complexity – chasing a fleer, shepherding a flock of sheep into a pen, and driving a virtual automobile through a simulated road network. The results indicate that in some instances, instructional learning can successfully create agents under some circumstances. However, instructional learning failed to build and/or improve agents in other instances. The Instructional Learning approach, the experiments, and their results are described and extensively discussed. (shrink)

This paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network -based reinforcement learning method. To this end, we rewrite the robotic system model as a state-space form, which will facilitate the realization of optimal tracking control synthesis. To maintain the tracking response, a steady-state control is designed, and then an adaptive optimal tracking control is used to ensure that the tracking error can achieve convergence in (...) an optimal sense. To solve the obtained optimal control via the framework of adaptive dynamic programming, the command trajectory to be tracked and the modified tracking Hamilton-Jacobi-Bellman are all formulated. An online RL algorithm is the developed to address the HJB equation using a critic NN with online learning algorithm. Simulation results are given to verify the effectiveness of the proposed method. (shrink)

This article aims to explore a more suitable prediction method for tourism complex environment, to improve the accuracy of tourism prediction results and to explore the development law of China’s domestic tourism so as to better serve the domestic tourism management and tourism decision-making. This study uses grey system theory, BP neural network theory, and the combination model method to model and forecast tourism demand. Firstly, the GM model is established based on the introduction (...) of grey theory. The regular data series are obtained through the transformation of irregular data series, and the prediction model is established. Secondly, in the structure algorithm of the BP neural network, the BP neural network model is established using the data series of travel time and the number of people. Then, combining BP neural network with the grey model, the grey neural network combination model is established to forecast the number of tourists. The prediction accuracy of the model is analyzed by the actual time series data of the number of tourists. Finally, the experimental analysis shows that the combination forecasting makes full use of the information provided by each forecasting model and obtains the combination forecasting model and the best forecasting result so as to improve the forecasting accuracy and reliability. (shrink)

The area of smart power grids needs to constantly improve its efficiency and resilience, to provide high quality electrical power in a resilient grid, while managing faults and avoiding failures. Achieving this requires high component reliability, adequate maintenance, and a studied failure occurrence. Correct system operation involves those activities and novel methodologies to detect, classify, and isolate faults and failures and model and simulate processes with predictive algorithms and analytics. In this paper, we showcase the application of a (...) class='Hi'>complex-adaptive, self-organizing modeling method, and Probabilistic Boolean Networks, as a way towards the understanding of the dynamics of smart grid devices, and to model and characterize their behavior. This work demonstrates that PBNs are equivalent to the standard Reinforcement Learning Cycle, in which the agent/model has an interaction with its environment and receives feedback from it in the form of a reward signal. Different reward structures were created to characterize preferred behavior. This information can be used to guide the PBN to avoid fault conditions and failures. (shrink)

Deep learning models are powerful tools for representing the complex learning processes and decision-making strategies used by humans. Such neural network models make fewer assumptions about the underlying mechanisms thus providing experimental flexibility in terms of applicability. However, this comes at the cost of involving a larger number of parameters requiring significantly more data for effective learning. This presents practical challenges given that most cognitive experiments involve relatively small numbers of subjects. Laboratory collaborations are a natural way (...) to increase overall dataset size. However, data sharing barriers between laboratories as necessitated by data protection regulations encourage the search for alternative methods to enable collaborative data science. Distributed learning, especially federated learning, which supports the preservation of data privacy, is a promising method for addressing this issue. To verify the reliability and feasibility of applying FL to train neural networks models used in the characterization of decision making, we conducted experiments on a real-world, many-labs data pool including experiment data-sets from ten independent studies. The performance of single models trained on single laboratory data-sets was poor. This unsurprising finding supports the need for laboratory collaboration to train more reliable models. To that end we evaluated four collaborative approaches. The first approach represents conventional centralized learning and is the optimal approach but requires complete sharing of data which we wish to avoid. The results however establish a benchmark for the other three approaches, federated learning, incremental learning, and cyclic incremental learning. We evaluate these approaches in terms of prediction accuracy and capacity to characterize human decision-making strategies. The FL-based model achieves performance most comparable to that of the CL-based model. This indicates that FL has value in scaling data science methods to data collected in computational modeling contexts when data sharing is not convenient, practical or permissible. (shrink)

Improving the resilience of urban road networks suffering from various disruptions has been a central focus for urban emergence management. However, to date the effective methods which may mitigate the negative impacts caused by the disruptions, such as road accidents and natural disasters, on urban road networks is highly insufficient. This study proposes a novel adaptive signal control strategy based on a doubly dynamic learning framework, which consists of deep reinforcement learning and day-to-day traffic dynamic (...) learning, to improve the network performance by adjusting red/green time split. In this study, red time split is regarded as extra traffic flow to discourage drivers to use affected roads, so as to reduce congestion and improve the resilience when urban road networks are subject to different levels of disruptions. In addition, we utilize the convolution neural network as Q-network to approximate Q values, link flow distribution and link capacity are regarded as the state space, and actions are denoted as red/green time split. A small network is utilized as a numerical example, and a fixed time signal control and other two adaptive signal controls are employed for the comparisons with the proposed one. The results show that the proposed adaptive signal control based on deep reinforcement learning can achieve better resilience in most of the cases, particularly in the scenarios of moderate and severe disruptions. This study may shed light on the advantages of the proposed adaptive signal control dealing with major emergencies compared to others. (shrink)

Spam electronic mails (emails) refer to harmful and unwanted commercial emails sent to corporate bodies or individuals to cause harm. Even though such mails are often used for advertising services and products, they sometimes contain links to malware or phishing hosting websites through which private information can be stolen. This study shows how the adaptive intelligent learning approach, based on the visual anti-spam model for multi-natural language, can be used to detect abnormal situations effectively. The application of (...) this approach is for spam filtering. With adaptive intelligent learning, high performance is achieved alongside a low false detection rate. There are three main phases through which the approach functions intelligently to ascertain if an email is legitimate based on the knowledge that has been gathered previously during the course of training. The proposed approach includes two models to identify the phishing emails. The first model has proposed to identify the type of the language. New trainable model based on Naive Bayes classifier has also been proposed. The proposed model is trained on three types of languages (Arabic, English and Chinese) and the trained model has used to identify the language type and use the label for the next model. The second model has been built by using two classes (phishing and normal email for each language) as a training data. The second trained model (Naive Bayes classifier) has been applied to identify the phishing emails as a final decision for the proposed approach. The proposed strategy is implemented using the Java environments and JADE agent platform. The testing of the performance of the AIA learning model involved the use of a dataset that is made up of 2,000 emails, and the results proved the efficiency of the model in accurately detecting and filtering a wide range of spam emails. The results of our study suggest that the Naive Bayes classifier performed ideally when tested on a database that has the biggest estimate (having a general accuracy of 98.4%, false positive rate of 0.08%, and false negative rate of 2.90%). This indicates that our Naive Bayes classifier algorithm will work viably on the off chance, connected to a real-world database, which is more common but not the largest. (shrink)

The question “How can humans learn efficiently to make decisions in a complex, dynamic, and uncertain environment” is still a very open question. We investigate what effects arise when feedback is given in a computer-simulated microworld that is controlled by participants. This has a direct impact on training simulators that are already in standard use in many professions, e.g., for flight simulators for pilots, and a potential impact on a better understanding of human decision making in general. (...) Our study is based on a benchmark microworld with an economic framing, the IWR Tailorshop. N=94 participants played four rounds of the microworld, each 10 months, via a web interface. We propose a new approach to quantify performance and learning, which is based on a mathematical model of the microworld and optimization. Six participant groups receive different kinds of feedback in a training phase, then results in a performance phase without feedback are analyzed. As a main result, feedback of optimal solutions in training rounds improved model knowledge, early learning, and performance, especially when this information is encoded in a graphical representation. (shrink)