Human-AI Integration Lab

Misha Montoya Sra

Assistant Professor, UCSB Computer Science
Affiliated Faculty with the Department of Electrical & Computer Engineering, Center for Responsible Machine Learning, Mind & Machine Inteligence, the Media, Art & Tech program and the Cognitive Science Program at UCSB.
Director of the Human-AI Integration Lab
Email: sra@ucsb.edu

As AI decouples intelligence from consciousness, as XR technologies transform perception, and as matter becomes machine manipulable, three key questions about the future arise: 1) what will newer AI-infused experiential interfaces look like where digital bits occupy space and algorithms have faces, 2) what can we learn about human behavior through human-AI interaction and how can AI models learn from human behavior, and 3) how will a shift from data-centric to interactive human-centric AI systems augment our capabilities and enrich our lives? To answer these questions, our current work focuses on Human-Computer Interaction (HCI), eXtended Reality (XR) and Human-Centered Artificial Intelligence (AI). In particular, we investigate the design, engineering and study of experiential interfaces (software and hardware), interactions, tools and systems that focus on skill acquisition, task guidance and creative collaboration.

We are inspired by the writings of Douglas Englebart and JCR Licklider. In both Augmenting Human Intellect and Man-Computer Symbiosis we see a vision of a world where human capabilities are greatly enhanced through close interaction with machines, which we call Human-AI Integration.

RESEARCH

Screen to AR

ARfy: A Pipeline for Adapting 3D Scenes to Augmented Reality. UIST 2022.

Passive Haptics

CardsVR: A Two-Person VR Experience with Passive Haptic Feedback from a Deck of Playing Cards. ISMAR 2022.

Hybrid 3D Models

Mix3D: Assembly and Animation of Seamlessly Stitched Meshes for Creating Hybrid Creatures and Objects. SUI 2022.

Experiential Artifacts

Situated VR: Toward a Congruent Hybrid Reality Without Experiential Artifacts. IEEE CG&A 2022.

EEG Artifact Correction

SRI-EEG: State-Based Recurrent Imputation for EEG Artifact Correction. Frontiers in Neuroscience 2022.

Text Style Transfer

Self-supervised knowledge assimilation for expert-layman text style transfer. AAAI 2022.

Virtual Instructors

AI-Generated Virtual Instructors Based on Liked or Admired People Can Improve Motivation and Foster Positive Emotions for Learning. FIE 2022.

Adaptive Virtual Neuroarchitecture

AI-Generated Virtual Instructors Based on Liked or Admired People Can Improve Motivation and Foster Positive Emotions for Learning. Preprint 2022.

AI-generated Characters

AI-generated characters for supporting personalized learning and well-being. Nature Machine Intelligence 2021.

Facial Paralysis Therapy with AI

FaraPy: An Augmented Reality Feedback System for Facial Paralysis using Action Unity Intensity Estimation. UIST 2021.

AR Storytelling

SceneAR: Scene-based Micro Narratives for Sharing and Remixing in Augmented Reality. ISMAR 2021.

Txt2Vid

Ultra-Low Bitrate Compression of Talking-Head Videos via Text. Arxiv preprint.

No-code VR Narrative Creation

EntangleVR: A Visual Programming Interface for Virtual Reality Interactive Scene Generation. VRST 2021.

Hands-free Gaming

Exploratory Design of a Hands-free Video Game Controller for a Quadriplegic Individual. AHs 2021.

Video to VR

IntoTheVideos: Exploration of Dynamic 3D Space Reconstruction From Single Sports Videos. UIST 2021.

Motion Synthesis for VR

Motion Improvisation: 3D Human Motion Synthesis with a Transformer. UIST 2021.

Thermo-pneumatic Feedback

PneuMod: A Modular Haptic Device with Localized Pressure and Thermal Feedback. VRST 2021.

AR Music Listening

A Spatial Music Listening Experience in Augmented Reality. UIST 2021.

AR Carrom

Flick Gesture Interaction in Augmented Reality: AR Carrom. UIST 2021.

VR Emotion Brushes

Exploring Emotion Brushes for a Virtual Reality Painting Tool. VRST 2021.

Image to Animation

IMAGEimate - An End-to-End Pipeline to Create Realistic Animatable 3D Avatars from a Single Image Using Neural Networks. VRST 2021 (to appear).

Facial Expression Recognition

Design and Evaluation of a Hands-free Video Game Controller for Individuals with Motor Impairments. Frontiers 2021.

AR Zoom Lectures

Multi-View AR Streams for Interactive 3D Remote Teaching. VRST 2021.

Telelife

Telelife: The Future of Remote Living. Frontiers 2021.

Living Bits

Opportunities and Challenges for Integrating Living Microorganisms in Human-Computer Interaction. AHs 2020. Best Paper Award (top 4%).

Consciousness

Designing Interactions Beyond Conscious Control: A New Model for Wearable Interfaces. IMWUT 2020.

World Editor

Augmented Reality World Editor. VRST 2020. Audience Choice Award.

Walking in VR

Walking and Teleportation in Wide-area Virtual Reality Experiences. ISMAR 2020.

MoveU

Electrical stimulation wearable device for providing haptic feedback and reducing motion sickness in VR. CHI 2019.

BreathVR

VR games that use breathing actions to augment controller input. CHI 2018. Best Paper Honorable Mention (top 5%)

Zanzibar

A flexible, portable mat that can sense and track objects, and support multi-touch and hover gestures. CHI 2018. Best Paper Award (top 1%)

Time Travel MIT

An outdoor VR time travel experience that takes you to the MIT of 1916 as well as an envisioned future 100 years from now.

Your Place And Mine

Room-scale mapping techniques to support natural interaction in shared VR spaces for remotely located users. DIS 2018.

GVS

A collaborative interface that allows remote control of a person's walking trajectory through galvanic vestibular stimulation (GVS). VRST 2017.

VMotion

Methods for managing user perception and attention based on the cognitive illusion of Inattentional Blindness. DIS 2018.

Oasis

Automatic generation of VR worlds from 3D reconstruction of real world places. VRST 2016. TVCG 2017. Best Paper Award (top 0.02%)

Full-body Tracking VR

A mocap hack using ViVe controllers before Vive trackers became available. IAP 2017.

Auris

A pipeline for the automatic generation of VR worlds from music. It uses a deep neural network for mood-based image generation. VRST 2017.

SnowballVR

Asymmetric multiplayer VR with roles based on the size of a user's space. UIST 2016.

Amphibian

Multisensory scuba diving experience in VR. CHI 2016. UIST 2016.

Building VR Worlds

Creating immersive and interactive VR worlds using the real world as a template. New Context 2015.

Metaspace I

Room-scale social VR with full-body tracking. It was built before the Vive was announced. UIST 2015.

Lights Out

A sensor-based bedtime alarm and a wallpaper display to promote awareness with sleep data visualization. MobiHealth 2015.

Metaspace II

Room-scale social VR with object and full-body tracking built using Kinects before Vive became available. IEEE VR 2016.

Hands Free

In-air gesture recognition system.

OnTheGo

Gestural input for mobile and wearable devices.

Deep Learning

Handwritten Tamil character recognition with a Convolutional Neural Network (CNN). NEML 2014.

Morph

A continuously morphing 4D geometrical VR world was my first VR project (DK1) back in 2013.

Spellbound

An outdoor fast paced team-based Android game. This was my Master's Thesis at the Media Lab completed in Aug 2013. PUC 2015.

Activ8

Three physical microgames for Google Glass for taking short breaks from sitting. Preprint 2014.

Point & Share

Digital pen that transmits writings on paper to the whiteboard. UIST 2012.

Health Score

An interactive tabletop application that displays a Health Score of a location, neighborhood or a selected area on a map.

Spotz

A location based personality creator and visualizer. Persuasive 2013.

LAB MEMBERS

Misha Sra
Assistant Professor of Computer Science

Sherry Chen
PhD student, CS

Yimeng Liu
PhD student, CS

Atieh Taheri
PhD candidate, ECE

Andrew Huard
PhD student, ECE

Jungah Son
PhD student, MAT

Arthur Caetano
PhD student, CS

Avinash Nargund
PhD student, ECE

Zichen Chen
PhD student, CS

Kojiro Takeyama
Visiting Researcher, Toyota Research

ALUMS

Carlos Gilberto
Visiting PhD student, MechE. UNAM, Mexico City

Giuliana Barrios Dell'Olio
Visiting Master's Student, CS 2020-2021. Now at McKinsey & Company

Jake Guida
Master's Student, CS 2020 (now at Adobe)

Mengyu Chen
PhD'23, MAT

APPLY

We are always looking for exceptional students at the intersection of Human-Computer Interaction, Electrical and Computer Engineering/Electrical Engineering, Cognitive Science and Mechanical Engineering.

If you are interested in applying to the PercLab:

Send me or any PhD student/s an email with your portfolio and CV.
Your online portfolio must show details of your projects and highlight your contributions. We are especially looking for technical expertise that involves: building hardware devices, implementing AR/VR systems, building interactive AI-based systems, and applications in our areas of interest.
Your CV should state your expertise level in areas that are crucial for our lab: have you built AR/VR systems, have you built any hardware, have you designed new AI algorithms or implemented existing ones, do you do more hardware or software or both, etc.
Make sure you have spent enough time on this website and can specify in your email why the PercLab is a good match for your research interests and skills.
Our lab is most suited for folks with some experience in HCI, ECE/EE, and ME.
If you are applying for a summer internship, you must have demonstrated experience in our areas of work.
If you are a UCSB undergraduate, please email me mentioning your area of interest, any prior experience and your transcript (unofficial is fine). Undergrads in the PercLab have so far worked on a volunteer basis but working for credit is also possible.