Publications

The field of collective intelligence studies how teams can achieve better results than any of the team members alone. The special case of human-machine teams carries unique challenges in this regard. For example, human teams often achieve synergy by communicating to discover their relative advantages, which is not an option if the team partner is an unexplainable deep neural network. Between 2005-2008 a set of "freestyle" chess tournaments were held, in which human-machine teams known as "centaurs", outperformed the best humans and best machines alone. Centaur players reported that they identified relative advantages between themselves and their chess program, even though the program was superhuman. Inspired by this and leveraging recent open-source models, we study human-machine like teams in chess. A human behavioral clone ("Maia") and a pure self-play RL-trained chess engine ("Leela") were composed into a team using a Mixture of Experts (MoE) architecture. By directing our research question at the selection mechanism of the MoE, we could isolate the issue of extracting relative advantages without knowledge sharing. We show that in principle, there is high potential for synergy between human and machine in a complex sequential decision environment such as chess. Furthermore, we show that an expert can identify only a small part of these relative advantages, and that the contribution of its subject matter expertise in doing so saturates quickly. This is probably due to the "curse of knowledge" phenomenon. We also train a network to recognize relative advantages using reinforcement learning, without chess expertise, and it outdoes the expert. Our experiments are repeated in asymmetric teams, in which identifying relative advantages is more challenging. Our findings contribute to the study of collective intelligence and human-centric AI.

The nature of abstract reasoning is a matter of debate. Modern artificial neural network (ANN) models, like large language models, demonstrate impressive success when tested on abstract reasoning problems. However, it has been argued that their success reflects some form of memorization of similar problems (data contamination) rather than a general-purpose abstract reasoning capability. This concern is supported by evidence of brittleness, and the requirement of extensive training. In our study, we explored whether abstract reasoning can be achieved using the toolbox of ANNs, without prior training. Specifically, we studied an ANN model in which the weights of a naive network are optimized during the solution of the problem, using the problem data itself, rather than any prior knowledge. We tested this modeling approach on visual reasoning problems and found that it performs relatively well. Crucially, this success does not rely on memorization of similar problems. We further suggest an explanation of how it works. Finally, as problem solving is performed by changing the ANN weights, we explored the connection between problem solving and the accumulation of knowledge in the ANNs.

This study explores the computational mechanisms underlying the resolution of cognitive dissonances. We focus on scenarios in which an observation violates the expected relationship between objects. For instance, an agent expects object A to be smaller than B in some feature space but observes the opposite. One solution is to adjust the expected relationship according to the new observation and change the expectation to A being larger than B. An alternative solution would be to adapt the representation of A and B in the feature space such that in the new representation, the relationship that A is smaller than B is maintained. While both pathways resolve the dissonance, they generalize differently to different tasks. Using Artificial Neural Networks (ANNs) capable of relational learning, we demonstrate the existence of these two pathways and show that the chosen pathway depends on the dissonance magnitude. Large dissonances alter the representation of the objects, while small dissonances lead to adjustments in the expected relationships. We show that this effect arises from the inherently different learning dynamics of relationships and representations and study the implications.

Background

Impulse control is a critical aspect of cognitive functioning. Intuitively, whether an action is executed prematurely depends on its associated reward, yet the link between value and impulsivity remains poorly understood. Three frameworks for impulsivity offer contrasting views: impulsive behavior may be valuable because it is associated with hidden internal reward (e.g., reduction of mental effort). Alternatively, it can emerge from exploration, which is disadvantageous in the short term but can yield long-term benefits. Finally, impulsivity may reflect Pavlovian bias, an inherent tendency that occurs even when its outcome is negative.

Methods

To test these hypotheses, we trained 17 male mice to withhold licking while anticipating variable rewards. We then measured and optogenetically manipulated dopamine release in the ventral striatum.

Results

We found that higher reward magnitudes correlated with increased impulsivity. This behavior was well explained by a Pavlovian bias model. Furthermore, we observed negative dopamine signals during premature licking, suggesting that in this task, impulsivity is not merely an unsuccessful attempt at obtaining a reward. Rather, it is a failure to overcome the urge to act prematurely despite knowledge of the negative consequences of such impulsive actions.

Conclusions

Our findings underscore the integral role value plays in regulating impulsivity and suggest that the dopaminergic system influences impulsivity through the mediation of value learning.

Humans are capable of solving complex abstract reasoning tests. Whether this ability reflects a learning-independent inference mechanism applicable to any novel unlearned problem or whether it is a manifestation of extensive training throughout life is an open question. Addressing this question in humans is challenging because it is impossible to control their prior training. However, assuming a similarity between the cognitive processing of Artificial Neural Networks (ANNs) and humans, the extent to which training is required for ANNs' abstract reasoning is informative about this question in humans. Previous studies demonstrated that ANNs can solve abstract reasoning tests. However, this success required extensive training. In this study, we examined the learning-independent abstract reasoning of ANNs. Specifically, we evaluated their performance without any pretraining, with the ANNs' weights being randomly-initialized, and only change in the process of problem solving. We found that naive ANN models can solve non-trivial visual reasoning tests, similar to those used to evaluate human learning-independent reasoning. We further studied the mechanisms that support this ability. Our results suggest the possibility of learning-independent abstract reasoning that does not require extensive training.

Decision-making in animals often involves choosing actions while navigating the environment, a process markedly different from static decision paradigms commonly studied in laboratory settings. Even in decision-making assays in which animals can freely locomote, decision outcomes are often interpreted as happening at single points in space and single moments in time, a simplification that potentially glosses over important spatiotemporal dynamics. We investigated locomotor decision-making in Drosophila melanogaster in Y-shaped mazes, measuring the extent to which their future choices could be predicted through space and time. We demonstrate that turn-decisions can be reliably predicted from flies’ locomotor dynamics, with distinct predictability phases emerging as flies progress through maze regions. We show that these predictability dynamics are not merely the result of maze geometry or wall-following tendencies, but instead reflect the capacity of flies to move in ways that depend on sustained locomotor signatures, suggesting an active, working memory-like process. Additionally, we demonstrate that fly mutants known to have sensory and information-processing deficits exhibit altered spatial predictability patterns, highlighting the role of visual, mechanosensory, and dopaminergic signaling in locomotor decision-making. Finally, highlighting the broad applicability of our analyses, we generalize our findings to other species and tasks. We show that human participants in a virtual Y-maze exhibited similar decision predictability dynamics as flies. This study advances our understanding of decision-making processes, emphasizing the importance of spatial and temporal dynamics of locomotor behavior in the lead-up to discrete choice outcomes.

A well-known observation in repeated-choice experiments is that a tendency to prefer one response over the others emerges if the feedback consistently favors that response. Choice bias, a tendency to prefer one response over the others, however, is not restricted to biased-feedback settings and is also observed when the feedback is unbiased. In fact, participant-specific choice bias, known as idiosyncratic choice bias (ICB), is common even in symmetrical experimental settings in which feedback is completely absent. Here we ask whether feedback-induced bias and ICB share a common mechanism. Specifically, we ask whether ICBs reflect idiosyncrasies in choice-feedback associations prior to the measurement of the ICB. To address this question, we compare the long-term dynamics of ICBs with feedback-induced biases. We show that while feedback effectively induced choice preferences, its effect is transient and diminished within several weeks. By contrast, we show that ICBs remained stable for at least 22 months. These results indicate that different mechanisms underlie the idiosyncratic and feedback-induced biases.

It is well-known that cortical areas specializing in the processing of somatosensory information from different parts of the body are arranged in an orderly manner along the cortex. It is also generally accepted that in the cortex, somatosensory information is initially processed in the primary somatosensory cortex and from there, it is hierarchically processed in other cortical regions. Previous studies have focused on the organization of representation at a level of a single or few cortical regions, identifying multiple body maps. However, the question of the large-scale organization of these different maps, and their relation to the hierarchical organization has received little attention. This is primarily because the highly convoluted shape of the cortical surface makes it difficult to characterize the relationship between cortical areas that are centimeters apart. Here, we used functional MRI to characterize cortical responses to full-body light touch stimulation. Our results indicate that the organization of both body representation and hierarchy is radial, with a small number of extrema that reign over a large number of cortical regions. Quantitatively computing the local relationship between the gradients of body and hierarchy maps, we show that the interaction between these two radial geometries, body representation and hierarchy in S1 are approximately orthogonal. However, this orthogonality is restricted to S1. Similar organizational patterns in the visual and auditory systems suggest that radial topography may be a common feature across sensory systems.

Schizophrenia is a severe disruption in cognition and emotion, affecting fundamental human functions. In this study, we applied Multi-Scale Entropy analysis to resting-state Magnetoencephalography data from 54 schizophrenia patients and 98 healthy controls. This method quantifies the temporal complexity of the signal across different time scales using the concept of sample entropy. Results show significantly higher sample entropy in schizophrenia patients, primarily in central, parietal, and occipital lobes, peaking at time scales equivalent to frequencies between 15 and 24 Hz. To disentangle the contributions of the amplitude and phase components, we applied the same analysis to a phase-shuffled surrogate signal. The analysis revealed that most differences originate from the amplitude component in the δ, α, and β power bands. While the phase component had a smaller magnitude, closer examination reveals clear spatial patterns and significant differences across specific brain regions. We assessed the potential of multi-scale entropy as a schizophrenia biomarker by comparing its classification performance to conventional spectral analysis and a cognitive task (the n-back paradigm). The discriminative power of multi-scale entropy and spectral features was similar, with a slight advantage for multi-scale entropy features. The results of the n-back test were slightly below those obtained from multi-scale entropy and spectral features.

Cortical function and the processing of sensory stimuli is remarkably robust against the continuous loss of neurons during aging, but also accelerated loss during prodromal stages of neurodegeneration. Population activity of neurons in sensory cortices builds a representation of the environment in form of a map that is structured in an informative way for guiding behavior. Here, we used the mouse auditory cortex as a model and probed the robustness of a representational map against the removal functionally characterized neurons. Specifically, we tested in how far the structure of the representational map is safeguarded by homeostatic network mechanisms. We combined longitudinal two-photon calcium imaging of population responses evoked by a diverse set of sound stimuli in the mouse auditory cortex with a targeted microablation of individual, functionally characterized neurons. Unilateral microablation of 30 - 40 selected sound-responsive layer 2/3 neurons led to a temporary collapse of the representational map that showed a subsequent recovery. At the level of individual neurons, we observed that the recovery was predominantly driven by neurons that were unresponsive to the sounds before microablation and gained responsiveness during the time course of several days. The remodeling of the spared network was mediated by a shift of the distribution of tuning curves towards the ablated neurons and was accompanied by a shift in the excitation/inhibition balance. Together, our findings provide a link between the plasticity of individual neurons and the population dynamics of sensory representations mediating robustness of cortical function. The dynamic reconstitution of the structure of activity patterns evoked by sensory stimuli despite a permanent loss of neurons in the network demonstrates a homeostatic maintenance of sensory representations in the neocortex.

Adapting to new environments is a hallmark of animal and human cognition, and Reinforcement Learning (RL) models provide a powerful and general framework for studying such adaptation. A fundamental learning component identified by RL models is that in the absence of direct supervision, when learning is driven by trial-and-error, exploration is essential. The necessary ingredients of effective exploration have been studied extensively in machine learning. However, the relevance of some of these principles to humans’ exploration is still unknown. An important reason for this gap is the dominance of the Multi-Armed Bandit tasks in human exploration studies. In these tasks, the exploration component per se is simple, because local measures of uncertainty, most notably visit-counters, are sufficient to effectively direct exploration. By contrast, in more complex environments, actions have long-term exploratory consequences that should be accounted for when measuring their associated uncertainties. Here, we use a novel experimental task that goes beyond the bandit task to study human exploration. We show that when local measures of uncertainty are insufficient, humans use exploration strategies that propagate uncertainties over states and actions. Moreover, we show that the long-term exploration consequences are temporally-discounted, similar to the temporal discounting of rewards in standard RL tasks. Additionally, we show that human exploration is largely uncertainty-driven. Finally, we find that humans exhibit signatures of temporally-extended learning, rather than local, 1-step update rules which are commonly assumed in RL models. All these aspects of human exploration are well-captured by a computational model in which agents learn an exploration “value-function”, analogous to the standard (reward-based) value-function in RL.

How does neuronal activity give rise to cognitive capacities? To address this question, neuroscientists hypothesize about what neurons “represent,” “encode,” or “compute,” and test these hypotheses empirically. This process is similar to the assessment of hypotheses in other fields of science and as such is subject to the same limitations and difficulties that have been discussed at length by philosophers of science. In this paper, we highlight an additional difficulty in the process of empirical assessment of hypotheses that is unique to the cognitive sciences. We argue that, unlike in other scientific fields, comparing hypotheses according to the extent to which they explain or predict empirical data can lead to absurd results. Other considerations, which are perhaps more subjective, must be taken into account. We focus on one such consideration, which is the purposeful function of the neurons as part of a biological system. We believe that progress in neuroscience critically depends on properly addressing this difficulty.

Introduction

Females with Alzheimer's disease (AD) suffer accelerated dementia and loss of cholinergic neurons compared to males, but the underlying mechanisms are unknown. Seeking causal contributors to both these phenomena, we pursued changes in transfer RNS (tRNA) fragments (tRFs) targeting cholinergic transcripts (CholinotRFs).

Methods

We analyzed small RNA-sequencing (RNA-Seq) data from the nucleus accumbens (NAc) brain region which is enriched in cholinergic neurons, compared to hypothalamic or cortical tissues from AD brains; and explored small RNA expression in neuronal cell lines undergoing cholinergic differentiation.

Results

NAc CholinotRFs of mitochondrial genome origin showed reduced levels that correlated with elevations in their predicted cholinergic-associated mRNA targets. Single-cell RNA seq from AD temporal cortices showed altered sex-specific levels of cholinergic transcripts in diverse cell types; inversely, human-originated neuroblastoma cells under cholinergic differentiation presented sex-specific CholinotRF elevations.

Discussion

Our findings support CholinotRFs contributions to cholinergic regulation, predicting their involvement in AD sex-specific cholinergic loss and dementia.

Video prediction models often combine three components: an encoder from pixel space to a small latent space, a latent space prediction model, and a generative model back to pixel space. However, the large and unpredictable pixel space makes training such models difficult, requiring many training examples. We argue that finding a predictive latent variable and using it to evaluate the consistency of a future image enables data-efficient predictions because it precludes the necessity of a generative model training. To demonstrate it, we created sequence completion intelligence tests in which the task is to identify a predictably changing feature in a sequence of images and use this prediction to select the subsequent image. We show that a one-dimensional Markov Contrastive Predictive Coding (M-CPC_1D) model solves these tests efficiently, with only five examples. Finally, we demonstrate the usefulness of M-CPC_1D in solving two tasks without prior training: anomaly detection and stochastic movement video prediction.

In this paper, we aimed to help bridge the gap between human fluid intelligence - the ability to solve novel tasks without prior training - and the performance of deep neural networks, which typically require extensive prior training. An essential cognitive component for solving intelligence tests, which in humans are used to measure fluid intelligence, is the ability to identify regularities in sequences. This motivated us to construct a benchmark task, which we term sequence consistency evaluation (SCE), whose solution requires the ability to identify regularities in sequences. Given the proven capabilities of deep networks, their ability to solve such tasks after extensive training is expected. Surprisingly, however, we show that naive (randomly initialized) deep learning models that are trained on a single SCE with a single optimization step can still solve non-trivial versions of the task relatively well. We extend our findings to solve, without any prior training, realworld anomaly detection tasks in the visual and auditory modalities. These results demonstrate the fluid-intelligent computational capabilities of deep networks. We discuss the implications of our work for constructing fluid-intelligent machines.

Boredom has been defined as an aversive mental state that is induced by the disability to engage in satisfying activity, most often experienced in monotonous environments. However, current understanding of the situational factors inducing boredom and driving subsequent behavior remains incomplete. Here, we introduce a two-alternative forced-choice task coupled with sensory stimulation of different degrees of monotony. We find that human subjects develop a bias in decision-making, avoiding the more monotonous alternative that is correlated with self-reported state boredom. This finding was replicated in independent laboratory and online experiments and proved to be specific for the induction of boredom rather than curiosity. Furthermore, using theoretical modeling we show that the entropy in the sequence of individually experienced stimuli, a measure of information gain, serves as a major determinant to predict choice behavior in the task. With this, we underline the relevance of boredom for driving behavioral responses that ensure a lasting stream of information to the brain.

Negated sentences are known to be more cognitively taxing than positive ones (i.e., polarity effect). We present evidence that two factors contribute to the polarity effect in verification tasks: processing the sentence and verifying its truth value. To quantify the relative contribution of each, we used a delayed verification task. The results show that even when participants are given a considerable amount of time for processing the sentence prior to verification, the polarity effect is not entirely eliminated. We suggest that this sustained effect stems from a retained negation-containing representation in working memory.

In natural settings, many stimuli impinge on our sensory organs simultaneously. Parsing these sensory stimuli into perceptual objects is a fundamental task faced by all sensory systems. Similar to other sensory modalities, increased odor backgrounds decrease the detectability of target odors by the olfactory system. The mechanisms by which background odors interfere with the detection and identification of target odors are unknown. Here we utilized the framework of the Drift Diffusion Model (DDM) to consider possible interference mechanisms in an odor detection task. We first considered pure effects of background odors on either signal or noise in the decision-making dynamics and showed that these produce different predictions about decision accuracy and speed. To test these predictions, we trained mice to detect target odors that are embedded in random background mixtures in a two-alternative choice task. In this task, the inter-trial interval was independent of behavioral reaction times to avoid motivating rapid responses. We found that increased backgrounds reduce mouse performance but paradoxically also decrease reaction times, suggesting that noise in the decision making process is increased by backgrounds. We further assessed the contributions of background effects on both noise and signal by fitting the DDM to the behavioral data. The models showed that background odors affect both the signal and the noise, but that the paradoxical relationship between trial difficulty and reaction time is caused by the added noise.

The mounting evidence for the involvement of astrocytes in neuronal circuits function and behavior stands in stark contrast to the lack of detailed anatomical description of these cells and the neurons in their domains. To fill this void, we imaged >30,000 astrocytes in hippocampi made transparent by CLARITY, and determined the elaborate structure, distribution, and neuronal content of astrocytic domains. First, we characterized the spatial distribution of >19,000 astrocytes across CA1 lamina, and analyzed the morphology of thousands of reconstructed domains. We then determined the excitatory somatic content of CA1 astrocytes, and measured the distance between inhibitory neuronal somata to the nearest astrocyte soma. We find that on average, there are almost 14 pyramidal neurons per domain in the CA1, increasing toward the pyramidal layer midline, compared to only five excitatory neurons per domain in the amygdala. Finally, we discovered that somatostatin neurons are found in close proximity to astrocytes, compared to parvalbumin and VIP inhibitory neurons. This work provides a comprehensive large-scale quantitative foundation for studying neuron-astrocyte interactions.