.

markdown_generator/publications.tsv

@@ -3,9 +3,10 @@ pub_date	title	venue	excerpt	citation	url_slug	paper_url
 2016-10-12	Multi-sensor based state prediction for personal mobility vehicles	PLOS ONE	This paper presents a study on multi-modal human emotional state detection while riding a powered wheelchair (PMV; Personal Mobility Vehicle) in an indoor labyrinth-like environment. The study reports findings on the habituation of human stress response during self-driving. In addition, the effects of “loss of controllability”, change in the role of the driver to a passenger, are investigated via an autonomous driving modality. The multi-modal emotional state detector sensing framework consists of four sensing devices: electroencephalograph (EEG), heart inter-beat interval (IBI), galvanic skin response (GSR) and stressor level lever (in the case of autonomous riding). Physiological emotional state measurement characteristics are organized by time-scale, in terms of capturing slower changes (long-term) and quicker changes from moment-to-moment. Experimental results with fifteen participants regarding subjective emotional state reports and commercial software measurements validated the proposed emotional state detector. Short-term GSR and heart signal characterizations captured moment-to-moment emotional state during autonomous riding (Spearman correlation; ρ = 0.6, p < 0.001). Short-term GSR and EEG characterizations reliably captured moment-to-moment emotional state during self-driving (Classification accuracy; 69.7). Finally, long-term GSR and heart characterizations were confirmed to reliably capture slow changes during autonomous riding and also of emotional state during participant resting state. The purpose of this study and the exploration of various algorithms and sensors in a structured framework is to provide a comprehensive background for multi-modal emotional state prediction experiments and/or applications. Additional discussion regarding the feasibility and utility of the possibilities of these concepts are given.	Jamilah Abdur-Rahim  ,Yoichi Morales ,Pankaj Gupta ,Ichiro Umata ,Atsushi Watanabe ,Jani Even ,Takayuki Suyama,Shin Ishii	multisensormobility	https://journals.plos.org/plosone/article/authors?id=10.1371/journal.pone.0162593
 2017-10-22	What are the visual features underlying human versus machine vision?	2017 IEEE International Conference on Computer Vision Workshops (ICCVW)	Although Deep Convolutional Networks (DCNs) are approaching the accuracy of human observers at object recognition, it is unknown whether they leverage similar visual representations to achieve this performance. To address this, we introduce Clicktionary, a web-based game for identifying visual features used by human observers during object recognition. Importance maps derived from the game are consistent across participants and uncorrelated with image saliency measures. These results suggest that Clicktionary identifies image regions that are meaningful and diagnostic for object recognition but different than those driving eye movements. Surprisingly, Clicktionary importance maps are only weakly correlated with relevance maps derived from DCNs trained for object recognition. Our study demonstrates that the narrowing gap between the object recognition accuracy of human observers and DCNs obscures distinct visual strategies used by each to achieve this performance.	Drew Linsley, Sven Eberhardt, Tarun Sharma, Pankaj Gupta, Thomas Serre	visualfeatureshuman	https://ieeexplore.ieee.org/document/8265530/authors#authors
 2019-11-20	Cortex-wide Computations in Complex Decision Making in Mice	Neuron	Seemingly, a paradox exists between reports of wide-scale task-dependent cortical activity and the causal requirement for only a restricted number of motor and sensory cortical areas in some behavioral studies. In this issue of Neuron, Pinto et al. (2019) indicate that scenarios where mice must accumulate evidence and hold it during a delay period are causally linked to wide regions of cortex.	Pankaj K Gupta, Timothy H Murphy	cotexwidecomputation	https://pubmed.ncbi.nlm.nih.gov/31751543/
-2020-05-14	Real-time selective markerless tracking of forepaws of head fixed mice using deep neural networks	Eneuro	Here, we describe a system capable of tracking specific mouse paw movements at high frame rates (70.17 Hz) with a high level of accuracy (mean = 0.95, SD < 0.01). Short-latency markerless tracking of specific body parts opens up the possibility of manipulating motor feedback. We present a software and hardware scheme built on DeepLabCut—a robust movement-tracking deep neural network framework—which enables real-time estimation of paw and digit movements of mice. Using this approach, we demonstrate movement-generated feedback by triggering a USB-GPIO (general-purpose input/output)-controlled LED when the movement of one paw, but not the other, selectively exceeds a preset threshold. The mean time delay between paw movement initiation and LED flash was 44.41 ms (SD = 36.39 ms), a latency sufficient for applying behaviorally triggered feedback. We adapt DeepLabCut for real-time tracking as an open-source package we term DeepCut2RealTime. The ability of the package to rapidly assess animal behavior was demonstrated by reinforcing specific movements within water-restricted, head-fixed mice. This system could inform future work on a behaviorally triggered “closed loop” brain–machine interface that could reinforce behaviors or deliver feedback to brain regions based on prespecified body movements.	Brandon J. Forys, Dongsheng Xiao, Pankaj Gupta and Timothy H. Murphy	realtimetracking	https://www.eneuro.org/content/7/3/ENEURO.0096-20.2020
+2020-05-14	Real-time selective markerless tracking of forepaws of head fixed mice using deep neural networks	eNeuro	Here, we describe a system capable of tracking specific mouse paw movements at high frame rates (70.17 Hz) with a high level of accuracy (mean = 0.95, SD < 0.01). Short-latency markerless tracking of specific body parts opens up the possibility of manipulating motor feedback. We present a software and hardware scheme built on DeepLabCut—a robust movement-tracking deep neural network framework—which enables real-time estimation of paw and digit movements of mice. Using this approach, we demonstrate movement-generated feedback by triggering a USB-GPIO (general-purpose input/output)-controlled LED when the movement of one paw, but not the other, selectively exceeds a preset threshold. The mean time delay between paw movement initiation and LED flash was 44.41 ms (SD = 36.39 ms), a latency sufficient for applying behaviorally triggered feedback. We adapt DeepLabCut for real-time tracking as an open-source package we term DeepCut2RealTime. The ability of the package to rapidly assess animal behavior was demonstrated by reinforcing specific movements within water-restricted, head-fixed mice. This system could inform future work on a behaviorally triggered “closed loop” brain–machine interface that could reinforce behaviors or deliver feedback to brain regions based on prespecified body movements.	Brandon J. Forys, Dongsheng Xiao, Pankaj Gupta and Timothy H. Murphy	realtimetracking	https://www.eneuro.org/content/7/3/ENEURO.0096-20.2020
 2020-07-12	The value of choice in 3- to 7-year-olds’ use of working memory gating strategies in a naturalistic task	Developmental Science	Rule-guided behavior depends on the ability to strategically update and act on content held in working memory. Proactive and reactive control strategies were contrasted across two experiments using an adapted input/output gating paradigm (Neuron, 81, 2014 and 930). Behavioral accuracies of 3-, 5-, and 7-year-olds were higher when a contextual cue appeared at the beginning of the task (input gating) rather than at the end (output gating). This finding supports prior work in older children, suggesting that children are better when input gating but rely on the more effortful output gating strategy for goal-oriented action selection (Cognition, 155, 2016 and 8). A manipulation was added to investigate whether children's use of working memory strategies becomes more flexible when task goals are specified internally rather than externally provided by the experimenter. A shift toward more proactive control was observed when children chose the task goal among two alternatives. Scan path analyses of saccadic eye movement indicated that giving children agency and choice over the task goal resulted in less use of a reactive strategy than when the goal was determined by the experimenter.	Livia Freier,Pankaj Gupta,David Badre,Dima Amso	workingmemory	https://onlinelibrary.wiley.com/doi/full/10.1111/desc.13017
 2021-09-01	Using Computational Analysis of Behavior To Discover Developmental Change In Memory-Guided Attention Mechanisms In Childhood	PsyArXiv	We tested 4-9.5-year-old children on a naturalistic memory-guided attention visual search task. We measured fixation distribution during a search using wearable eye tracking, and simultaneously recorded depth video data for each participant and used computer vision algorithms to track them during navigation. We manipulated object placement and trial order such that nearby objects would be encountered during initial search for reference objects. We used a computational model of top-down guidance for reference object visual features and examined the use of this top-down attention for reference objects during subsequent nearby object search. The data suggest that the value of physical navigation during initial spatial exploration for subsequent memory-guided attention, specifically in early childhood, is in its association with stronger visual representations of goal reference objects during spatial exploration. By middle childhood, visual search times were not impacted by memory engagement.	Dima Amso, Lakshmi Govindarajan, Pankaj Gupta, Diego Placido, Heidi Baumgartner, Andrew Lynn, Kelley Gunther, Tarun Sharma, Vijay Veerabadran, Kalpit Thakkar, Seung Chan Kim, Thomas Serre	smartplayroom	https://psyarxiv.com/gq4rt/
 2021-02-15	Neuromatch Academy- a 3-week, online summer school in computational neuroscience	Journal of Open Source Education	Neuromatch Academy (https://neuromatch.io/academy) was designed as an online summer school to cover the basics of computational neuroscience in three weeks. The materials cover dominant and emerging computational neuroscience tools, how they complement one another, and specifically focus on how they can help us to better understand how the brain functions. An original component of the materials is its focus on modeling choices, i.e. how do we choose the right approach, how do we build models, and how can we evaluate models to determine if they provide real (meaningful) insight. This meta-modeling component of the instructional materials asks what questions can be answered by different techniques, and how to apply them meaningfully to get insight about brain function.	't Hart, Bernard M., et al.	nma	https://jose.theoj.org/papers/101bd5b60c63dc778dfcb9da787820b1
 2021-03-12	Real-time neural feedback of mesoscale cortical GCAMP6 signals for training mice	CoSyne 2021	Mice can learn to control specific neuronal ensembles using sensory (eg. auditory) cues (Clancy et al. 2014) or even artificial optogenetic stimulation (Prsa et al. 2017). In the present work, we measure mesoscale cortical activity with GCaMP6s and provide graded auditory feedback (within ~100 ms after GCaMP fluorescence) based on changes in dorsal-cortical activation within specified regions of interest (ROI)s with a specified rule. We define a compact, low-cost optical brain-machine-interface (BMI) capable of image acquisition, processing, and conducting closed-loop auditory feedback and rewards, using a Raspberry Pi (Fig. 1). The changes in fluorescence activity (ΔF/F) are calculated based on a running baseline (eg. 5 sec.). Two ROIs (R1, R2) on the dorsal cortical map were selected as targets. We started with a rule of ‘R1-R2’ (ΔF/F of R1 minus ΔF/F of R2) where the activity of R1 relative to R2 was mapped to frequency of the audio feedback (Fig. 1D) and if it were to cross a set threshold, a water drop reward is generated. To investigate learning in this context, water-deprived tetO-GCaMP6s mice (N=8) were trained for 30-minutes every day on the system for several days, with a task to increase audio frequency leading to reward. We found that mice could modulate activity in the rule-specific target ROIs to get an increasing number of rewards over days (Figure 2C). Analysis of the reward-triggered ΔF/F over time indicated that mice progressively learned to activate the cortical ROI to a greater extent (Figure 2B, 2A). In conclusion, we developed an open-source system (to-be released) for closed-loop feedback that can be added to experimental scenarios for brain activity training and could be possibly effective in inducing neuroplasticity.	Pankaj K Gupta, Timothy H Murphy	cosyne2021	https://www.cosyne.org/s/Cosyne2021_program_book.pdf
 2021-04-05	A three-dimensional virtual mouse generates synthetic training data for behavioral analysis	Nature Methods	We developed a three-dimensional (3D) synthetic animated mouse based on computed tomography scans that is actuated using animation and semirandom, joint-constrained movements to generate synthetic behavioral data with ground-truth label locations. Image-domain translation produced realistic synthetic videos used to train two-dimensional (2D) and 3D pose estimation models with accuracy similar to typical manual training datasets. The outputs from the 3D model-based pose estimation yielded better definition of behavioral clusters than 2D videos and may facilitate automated ethological classification.	Luis A. Bolaños, Dongsheng Xiao, Nancy L. Ford, Jeff M. LeDue, Pankaj K. Gupta, Carlos Doebeli, Hao Hu, Helge Rhodin & Timothy H. Murphy	3dmousenaturemeth	https://www.nature.com/articles/s41592-021-01103-9
+	PyMouseTracks: flexible computer vision and RFID-based system for multiple mouse tracking and behavioral assessment	eNeuro	PyMouseTracks (PMT) is a scalable and customizable computer vision and radio frequency identification (RFID)-based system for multiple rodent tracking and behavior assessment that can be set up within minutes in any user-defined arena at minimal cost. PMT is composed of the online Raspberry Pi (RPi)-based video and RFID acquisition with subsequent offline analysis tools. The system is capable of tracking up to six mice in experiments ranging from minutes to days. PMT maintained a minimum of 88% detections tracked with an overall accuracy >85% when compared with manual validation of videos containing one to four mice in a modified home-cage. As expected, chronic recording in home-cage revealed diurnal activity patterns. In open-field, it was observed that novel noncagemate mouse pairs exhibit more similarity in travel trajectory patterns than cagemate pairs over a 10-min period. Therefore, shared features within travel trajectories between animals may be a measure of sociability that has not been previously reported. Moreover, PMT can interface with open-source packages such as DeepLabCut and Traja for pose estimation and travel trajectory analysis, respectively. In combination with Traja, PMT resolved motor deficits exhibited in stroke animals. Overall, we present an affordable, open-sourced, and customizable/scalable mouse behavior recording and analysis system.		pymousetracks	https://www.eneuro.org/content/10/5/ENEURO.0127-22.2023

markdown_generator/talks.tsv

@@ -17,3 +17,4 @@ Can we aid in stroke recovery?	Talk	nph2021	Frontiers in Neurophotonics Highligh
 Introduction to DeepLabCut	Tutorial	phb2022	Precision Health Bootcamp 2022	2022-08-10	Vancouver, Canada	https://tinyurl.com/3d6przm9	The Precision Health Bootcamp will run from July 25th - August 10th, 2022.
 Modeling multi-region cortical interactions using task-specific data-constrained recurrent neural networks	Poster	sfn2022	Society for Neuroscience 2022	2022-11-16	San Diego, USA		Poster presentation at SfN 2022 conference
 Platform for real-time closed-loop feedback of behavior and/or cortical GcaMP activity in mice using an all-optical strategy at the mesoscale	Talk	sfn2023	Society for Neuroscience 2023	2023-11-12	Washington DC, USA	https://www.abstractsonline.com/pp8/#!/10892/presentation/42549	Part of SfN 2023 Nanosymposium Session NANO24 - Neural Coding, Perception, and Plasticity
+Platform for real-time feedback of cortical GCaMP activity and specific body movements in mice	Talk	neuroaiseattle2024	NeuroAI Seattle 2024 conference	###	Eugene, OR, USA	https://www.neuroaiseattle.com/schedule-1	Talk at NeuroAI Seattle 2024 meeting