It’s one thing to study how the relief and albedo of the ice sheets affected weather patterns during the Last Glacial Maximum 20,000 years ago. And it’s a whole other thing to develop an interactive, engaging museum exhibit on the subject for general audiences. But that’s just what teams from the (CIRES), the (NSF NCAR), NOAA and ATLAS managed to do.

Millennia ago, ice sheets formed over huge swaths of North America that were nearly as tall as some of our continent’s highest mountains. They were so massive that they essentially created their own weather.

Former CIRES postdoc Dillon Amaya (now at NOAA’s ) along with Kris Karnauskas, CIRES fellow and associate professor in the Department of Atmospheric and Oceanic Sciences, with NSF funding. 

Researchers long hypothesized that the ice’s massive scale during the Last Glacial Maximum was enough to block the jet stream and change weather patterns sweeping in from the Pacific Ocean. For example, back then the area around what is today Southern California was much wetter while the Pacific Northwest was relatively drier. Today that is reversed.

Through advanced computer simulations, the CIRES team discovered that albedo creates a cooling effect that alters atmospheric circulation in ways that cannot be explained solely by the sheer size of ice sheets. Albedo is a measure of the amount of light reflected off of a surface—and ice sheets reflect a lot of sunlight, significantly impacting wind patterns. The research showed the Pacific Ocean was the driver behind the changes.

Translating Complex Research
In spring 2022, ATLAS offered a class called Design a Science Exhibit for ATLAS and Computer Science students. It centered on designing approachable museum exhibits that translate hard science for everyday people. Led by ATLAS director Mark Gross and adjunct faculty member Wayne Seltzer in collaboration with Eddie Goldstein from the Denver Nature and Science Museum, student teams partnered with researchers and museum specialists to prototype exhibitions that incorporated coding, materials selection, fabrication and storytelling.

Gross notes, “We should be teaching our engineers to communicate with broad audiences, particularly around climate change. We might do good science and engineering, but we’re not always good at communicating it to the public.”

A team of CU Boulder students formed a group to translate the CIRES ice sheet research into an exhibit prototype, including, ATLAS PhD student, David Hunter; Natasha Smith (MS Environment, Environmental Policy); and ATLAS undergraduate students Caileigh Hudson, Logan Turner and Julia Tung.

Seltzer explains, “The that inspired this exhibit is not all that accessible to readers who are not climate scientists. The students focused on what they decided was essential knowledge—the factors that result in an ice age and how computer models can help us predict climate change.”

Experimenting with Form
The team originally conceived of a sandbox as the project medium. As you moved the sand around to build different topographies, visual projections overlaid from above would show how weather patterns change. The idea made sense in theory, but practical stipulations (sand can be challenging to manage in a museum space) pushed the team in a different direction.

Hunter details this evolution, “We made little blocks that represent [topographic features], and then you could put the blocks on top of each other so you could sculpt [a landscape.] As a team, we went about designing and building the whole rig and had a prototype by the end of the semester, and we got to show it alongside everyone else's work at NCAR.” 

NSF NCAR science educators were so impressed with the prototype that they invited the team to work on a permanent installation. 

Making it Real

The biggest challenge then became orchestrating all the different people and components involved in developing a functional exhibit that could live for the long-term with as little ongoing maintenance as possible. Hunter notes, “There’s the digital prototype building, but then there’s the physical make-this-real part as well as the education part and ensuring visitors would get the right message.” 

After two years of iterative collaboration with scientists, curators, coders, fabricators and educators, the exhibit is now officially on permanent display at the Mesa Lab Visitor’s Center. Thousands of guests each year will be able to explore how massive ice sheets can alter the climate in surprising ways.

Amaya related, “This was probably one of the most gratifying experiences of my scientific career. It's not often that a piece of research like this leads to such tangible educational outcomes, so I'm super proud of our team for seeing it through! It's my hope that this exhibit can help illustrate some of these exotic climate interactions so that visitors can leave with a better physical intuition for how and why things were so wildly different.”

If you go:

1850 Table Mesa Drive
Boulder, CO

Free Admission

Hours:
Monday - Friday: 8:00 a.m. – 5:00 p.m. MT
Saturday, Sunday & Holidays: 9:00 a.m. – 4:00 p.m. MT

Off

Off

Off

Partnerships between universities and industry can yield important research and commercial breakthroughs. ATLAS professor Ellen Do has worked to cultivate relationships between CU Boulder and industry players, including as a member of the Pervasive Personalized Intelligence (PPI) Center, to support graduate students and enhance opportunities for commercialization of ATLAS research.

The , which recently concluded its tenure, was founded “with a mission of bringing industry and university talent together to solve the intelligence challenges faced by software and computer engineers in Internet of Things systems." It operated under the supervision of the National Science Foundation and included members from NEC, Intel and Trimble.

“It’s been such a good experience. We’ve learned a lot. Ellen Do and her team have helped to expand our thinking and encouraged us to explore new areas.” - Dr. Haifeng Chen, Head of Data Science Department at NEC Laboratories, and his colleague Kai Ishikawa, Principal Researcher (PPI Center event recap)

The PPI Center’s in Portland, OR, included a research poster session, and ATLAS students were honored with three of the four awards industry attendees voted on at the event. 

• Suibi Che-Chuan Weng, PhD student, won "Most Industry Ready" for Editing Reality: Empowering Users to Manipulate Reality through Addition, Erasing, and Modification with Speech to Prompt in Mixed Reality.
• Rishi Vanukuru, PhD student, won "Most Impactful" for Asynchronous spatial guidance using mobile devices and Augmented Reality.
• Ada Zhao, MS student, won "Most Impactful" for The WizARd and Apprentice: Augmented Reality Expert Capture for Training Novices.

2 more ATLAS PhD students participated: Krithik Ranjan presented PuppetGuide: Tangible Personalized Museum Tour Guides using LLMs and David Hunter presented Tangible Interaction with Object Detection and Large Language Models.

As for the experience participating in the PPI Center, Do says, “it is good to know that the industry is interested in supporting research and considers our research relevant.” She sees ways ATLAS could form partnerships within several industry sectors on a range of themes due to the multidisciplinary nature of the research conducted here.

Since their involvement in PPI started, Do and her team have had a series of meetings with mentors from global technology firms, discussing collaborative research opportunities.

Vanukuru is currently doing an internship at Microsoft Research Cambridge focused on spatial computing in its VR/AR group. Weng and Zhao are working on research in the ACME Lab this summer, extending the Editing Reality (and PuppetGuide), and WizARd and Apprentice projects with interns from the CU SPUR program. Zhao is also conducting a pilot study, interviewing laser cutter operating experts about how they would demonstrate operations and how they can annotate their demonstration using the WizARd prototype for novice learners. Hunter has embarked on an internship with Trimble this summer, while he and Ranjan are also working in the ACME Lab.

Off

Off

Off

Billed as the premier conference for Augmented Reality (AR), Mixed Reality (MR) and Virtual Reality (VR), IEEE ISMAR was the perfect location for ATLAS community members to showcase their work this month.  

ATLAS PhD students Rishi Vanukuru, Torin Hopkins and Suibi Che-Chuan Weng attended in Sydney, Australia, from October 16-20, along with leading researchers in academia and industry.

Vanukuru presented his work on DualStream, a system for mobile phone-based spatial communication employing AR to give people more immersive tools to “share spaces and places.” He also participated in the “1st Joint Workshop on Cross Reality” with his research on using mobile devices to support collaboration.

Meanwhile, Hopkins and Weng displayed their respective research on improving ways for musicians to collaborate remotely. 

Research ATLAS PhD students presented at ISMAR 2023

Rishi Vanukuru, Suibi Che-Chuan Weng, Krithik Ranjan, Torin Hopkins, Amy Banić, Mark D. Gross, Ellen Yi-Luen Do

Abstract: In-person human interaction relies on our spatial perception of each other and our surroundings. Current remote communication tools partially address each of these aspects. Video calls convey real user representations but without spatial interactions. Augmented and Virtual Reality (AR/VR) experiences are immersive and spatial but often use virtual environments and characters instead of real-life representations. Bridging these gaps, we introduce DualStream, a system for synchronous mobile AR remote communication that captures, streams, and displays spatial representations of users and their surroundings. DualStream supports transitions between user and environment representations with different levels of visuospatial fidelity, as well as the creation of persistent shared spaces using environment snapshots. We demonstrate how DualStream can enable spatial communication in real-world contexts, and support the creation of blended spaces for collaboration. A formative evaluation of DualStream revealed that users valued the ability to interact spatially and move between representations, and could see DualStream fitting into their own remote communication practices in the near future. Drawing from these findings, we discuss new opportunities for designing more widely accessible spatial communication tools, centered around the mobile phone.
 

Exploring the use of Mobile Devices as a Bridge for Cross-Reality Collaboration []
Rishi Vanukuru, Ellen Yi-Luen Do 

Abstract: Augmented and Virtual Reality technologies enable powerful forms of spatial interaction with a wide range of digital information. While AR and VR headsets are more affordable today than they have ever been, their interfaces are relatively unfamiliar, and a large majority of people around the world do not yet have access to such devices. Inspired by contemporary research towards cross-reality systems that support interactions between mobile and head-mounted devices, we have been exploring the potential of mobile devices to bridge the gap between spatial collaboration and wider availability. In this paper, we outline the development of a cross-reality collaborative experience centered around mobile phones. Nearly fifty users interacted with the experience over a series of research demo days in our lab. We use the initial insights gained from these demonstrations to discuss potential research directions for bringing spatial computing and cross-reality collaboration to wider audiences in the near future.
 

Investigating the Effects of Limited Field of View on Jamming Experience in Extended Reality []
Suibi Che-Chuan Weng, Torin Hopkins, Shih-Yu Ma, Chad Tobin, Amy Banić, Ellen Yi-Luen Do

Abstract: During musical collaboration, extra-musical visual cues are vital for communication between musicians. Extended Reality (XR) applications that support musical collaboration are often used with headmounted displays such as Augmented Reality (AR) glasses, which limit the field of view (FOV) of the players. We conducted a three part study to investigate the effects of limited FOV on co-presence. To investigate this issue further, we conducted a within-subjects user study (n=19) comparing an unrestricted FOV holographic setup to Nreal AR glasses with a 52◦ limited FOV. In the AR setup, we tested two conditions: 1) standard AR experience with 52◦-limited FOV, and 2) a modified AR experience, inspired by player feedback. Results showed that the holographic setup offered higher co-presence with avatars.
 

Networking AI-Driven Virtual Musicians in Extended Reality [Poster]
Torin Hopkins, Rishi Vanukuru, Suibi Che-Chuan Weng, Chad Tobin, Amy Banić, Mark D. Gross, Ellen Yi-Luen Do

Abstract: Music technology has embraced Artificial Intelligence as part of its evolution. This work investigates a new facet of this relationship, examining AI-driven virtual musicians in networked music experiences. Responding to an increased popularity due to the COVID-19 pandemic, networked music enables musicians to meet virtually, unhindered by many geographical restrictions. This work begins to extend existing research that has focused on networked human-human interaction by exploring AI-driven virtual musicians’ integration into online jam sessions. Preliminary feedback from a public demonstration of the system suggests that despite varied understanding levels and potential distractions, participants generally felt their partner’s presence, were task-oriented, and enjoyed the experience. This pilot aims to open opportunities for improving networked musical experiences with virtual AI-driven musicians and informs directions for future studies with the system.

Off

ATLAS PhD student, Purnendu, presented research at the recent in Delft, Netherlands. Haptics is the science of touch, and this gathering is billed as “the premier international conference covering all aspects of haptics including the fundamental scientific findings, technological developments, algorithms and applications.”

Purnendu developed a wearable electrohydraulic fingertip interface during an internship at Meta’s , with the aim to improve how we perceive and use physical touch in virtual and augmented reality environments. Meta created Reality Labs to “bring together the brightest cross-disciplinary minds in one place to deliver our mission: build tools that help people feel connected, anytime, anywhere.”

[video:https://www.youtube.com/watch?v=sHb_Jq3AXyw]

We connected with Purnendu to hear more details on his haptics work and the conference. Take a look:

What was the initial inspiration behind this research?

The inspiration behind this research was to be able to augment human fingertips with a reliable sense of 'artificial touch' for the Metaverse (Augmented Reality/Virtual Reality Environments). Fingertips are one of the most sensitive regions of human skin and to be able to provide them with desirable tactile cues is an open problem. The most promising pathway is attaching a high density of multimodal actuators with capabilities to render a variety of forces (normal/shear) as well as vibrations.

This research leveraged my prior research on soft electrohydraulic actuators (happened at ATLAS) to develop a high resolution fingertip wearable multimodal haptic interface consisting of 16 individual actuators that can render high intensity pressure as well as a wide range of vibrations (within an area of 1 cm sq).

What was it like to work at Meta?

Meta Reality Labs is an amazing workplace. The realization that many of these research projects happening around me will form the foundation of the products that will come out in the next 5-10 years was thrilling. The depth as well as the breadth of the work happening there is mind boggling. But not only the research being performed was cutting edge, the team and people were really nice. Reality Labs is filled with folks who are super smart, skilled, motivated, and above all very kind. I had a great time working there.

How was your experience presenting at the conference?

IEEE World Haptics Conference is a premiere venue for the haptics community to present, discuss, and demonstrate research ideas and prototypes. My experience was beyond expectation. My presentation went really well; it was an instant hit and a lot of people reached out to me afterwards. The other presentations and demos were quite good (both from industry partners as well as academic labs). The highlight for me was to get to interact closely with the most senior researchers in the haptics community. 

How has this research shaped your ongoing work as a PhD student in ATLAS?

This research has been along the lines of my PhD work at ATLAS and will be included in my dissertation. It had origins in the previous research I performed at ATLAS on soft electrohydraulic actuators. It helped me orient more towards my thesis work on steering soft materials with high-intensity electric fields.

Publication Details

Purnendu, Jess Hartcher‑O’Brien, Vatsal Mehta, Nicholas Colonnese, Aakar Gupta, Carson Bruns, Priyanshu Agarwal

Fingertips are one of the most sensitive regions of the human body and provide a means to dexterously interact with the physical world. To recreate this sense of physical touch in a virtual or augmented reality (VR/AR), high-resolution haptic interfaces that can render rich tactile information are needed. In this paper, we present a wearable electrohydraulic haptic interface that can produce high-fidelity multimodal haptic feedback at the fingertips. This novel hardware can generate high-intensity fine tactile pressure (up to 34 kPa) as well as a wide range of vibrations (up to 700 Hz) through 16 individually controlled electrohydraulic bubble actuators. To achieve such a high intensity multimodal haptic feedback at such a high density (16 bubbles/cm2) at the fingertip using an electrohydraulic haptic interface , we integrated a stretchable substrate with a novel dielectric film and developed a design architecture wherein the dielectric fluid is stored at the back of the fingertip. We physically characterize the static and dynamic behavior of the device. In addition, we conduct psychophysical characterization of the device through a set of user studies. This electrohydraulic interface demonstrates a new way to design and develop high- resolution multimodal haptic systems at the fingertips for AR/VR environments.

Off

Sandra Bae, PhD student and member of the Utility Research Lab and ACME Lab at ATLAS, has been honored with a Best Paper Honorable Mention at VIS 2023 for her research on network physicalizations. 

Billed as “the premier forum for advances in theory, methods and applications of visualization and visual analytics”, will be held in Melbourne, Australia, from October 22-27, and is sponsored by IEEE. The Best Papers Committee bestows honorable mentions on the top 5% of publications submitted. 

The paper introduces a computational design pipeline to 3D print physical representations of networks enabling touch interactivity via capacitive sensing and computational inference.

[video:https://youtu.be/uv0Yu0WUeSQ]

 
S. Sandra Bae, Takanori Fujiwara, Anders Ynnerman, Ellen Yi-Luen Do, Michael L. Rivera, Danielle Albers Szafir

Abstract
Interaction is critical for data analysis and sensemaking. However, designing interactive physicalizations is challenging as it requires cross-disciplinary knowledge in visualization, fabrication, and electronics. Interactive physicalizations are typically produced in an unstructured manner, resulting in unique solutions for a specific dataset, problem, or interaction that cannot be easily extended or adapted to new scenarios or future physicalizations. To mitigate these challenges, we introduce a computational design pipeline to 3D print network physicalizations with integrated sensing capabilities. Networks are ubiquitous, yet their complex geometry also requires significant engineering considerations to provide intuitive, effective interactions for exploration. Using our pipeline, designers can readily produce network physicalizations supporting selection-the most critical atomic operation for interaction-by touch through capacitive sensing and computational inference. Our computational design pipeline introduces a new design paradigm by concurrently considering the form and interactivity of a physicalization into one cohesive fabrication workflow. We evaluate our approach using (i) computational evaluations, (ii) three usage scenarios focusing on general visualization tasks, and (iii) expert interviews. The design paradigm introduced by our pipeline can lower barriers to physicalization research, creation, and adoption.

Bae describes potential use cases for sensing network physicalizations:

• Accessibility visualization - Accessible visualizations (e.g., tactile visualizations) focus on making data visualization more inclusive, particularly for those with low vision or blindness. However, most tactile visualizations are static and non-interactive, which reduces data expressiveness and inhibits data exploration. This technique can create more interactive tactile visualizations. 
• AR/VR - Most AR/VR devices use computer vision (CV), but most devices using CV cannot reproduce the haptic benefits that we naturally leverage (holding, rotating, tracing) with our sense of touch. Past studies confirm the importance of tangible inputs when virtually exploring data. But creating tangible devices for AR/VR requires too much instrumentation to make them interactive. Our technique would enable developers to more easily produce fully functional, responsive controllers right from the printer within a single pass.

The work continues as Bae plans to pursue more complex designs and richer interactivity including:

Fabricating bigger networks - The biggest network Bae has 3D printed so far is 20 nodes and 40 links, but this is rather small for most network datasets. She will scale this technique to support bigger networks.

Supporting output - Interactive objects receive input (e.g., from touch) and produce output (e.g., light, sound, color change) in a controlled manner. The sensing network currently addresses the first part of the interaction loop by responding to touch inputs, but she next wants to explore how to support output.

Bae showcased this research along with fellow ATLAS community members at the Rocky Mountain RepRap Festival earlier this year. We’re excited to see where her innovative research leads next.

Off

ATLAS is well-represented at this year’s conference convening at Carnegie Mellon University in Pittsburgh from July 10-14, 2023. This year’s theme is resilience. 

"Resilience is at once about flexibility, durability, and strength as well as a sense of mutuality and hope where solidaristic modes of engagement make new kinds of worlds possible. 

This also recognizes that resilience takes many forms in design discourse, ranging across: indigenous knowledge, more-than-human perspectives, and the relationship between human, material and artificial intelligences."

It is exciting to see members across more than half of ATLAS labs represented in this year’s proceedings, with broad-reaching research covering microbiomes as materials for interactive design; 3D printing with spent coffee grounds; personal informatics systems; improving cross-disciplinary collaboration among artists and researchers; expressive movement for altering emotions and awareness; and the intersection of crocheting and data. Take a look:

Fiona Bell (ATLAS PhD alum), Michelle Ramsahoye (ATLAS affiliate PhD student), Joshua Coffie (ATLAS MS alum), Julia Tung (ATLAS BS student), and Mirela Alistar (ATLAS Living Matter Lab director, assistant professor)

Our bodies are home to an unseen ecosystem of microbes that live in symbiosis with us. In this work, we extend the “human” in Human-Computer Interaction (HCI) to include these microbes. Specifically, we explore the skin microbiome as an intimate material for interaction design. Viewing the body as a microbial interface, we start by presenting a method to grow our microbiome such that it becomes visible to the human eye. We then present a design space that explores how different environmental parameters, such as temperature and growth media, can be controlled to influence the color of the microbiome. We further investigate how our interactions in a daily uncontrolled environment (e.g., exercising, hugging, typing) can impact the microbiome. We demonstrate several wearable applications that reveal and control the microbiome. Lastly, we address the challenges and opportunities of working with the microbiome as an intimate, living material for interaction design.

Michael L. Rivera (ATLAS Utility Research Lab Director, assistant professor), S. Sandra Bae (ATLAS PhD student)

The widespread adoption of 3D printers exacerbates existing environmental challenges as these machines increase energy consumption, waste output, and the use of plastics. Material choice for 3D printing is tightly connected to these challenges, and as such researchers and designers are exploring sustainable alternatives. Building on these efforts, this work explores using spent coffee grounds as a sustainable material for prototyping with 3D printing. This material, in addition to being compostable and recyclable, can be easily made and printed at home. We describe the material in detail, including the process of making it from readily available ingredients, its material characteristics and its printing parameters. We then explore how it can support sustainable prototyping practices as well as HCI applications. In reflecting on our design process, we discuss challenges and opportunities for the HCI community to support sustainable prototyping and personal fabrication. We conclude with a set of design considerations for others to weigh when exploring sustainable materials for 3D printing and prototyping.

For additional details, see our article on how this and other Utility Research Lab projects won awards at the Rocky Mountain RepRap Festival.

Michael Jeffrey Daniel Hoefer, Stephen Voida, (ATLAS affiliate assistant professor, founding faculty, information science)

A grand challenge for computing is to better understand fundamental human needs and their satisfaction. In this work, we design a personal informatics technology probe that scaffolds reflection on how time-use satisfies Max-Neef's fundamental needs of being, having, doing, and interacting via self-aspects, relationships and organizations, activities, and environments. Through a combination of a think-aloud study (N=10) and a week-long in situ deployment (N=7), participants used the probe to complete self- aspect elicitation and Day Reconstruction Method tasks. Participants then interacted with network visualizations of their daily lives, and discovered insights about their lives. During the study, we collected a dataset of 662 activities annotated with need satisfaction ratings. Despite challenges in operationalizing a theory of need through direct elicitation from individuals, personal informatics systems show potential as a participatory and individually meaningful approach for understanding need satisfaction in everyday life.

Ruhan Yang (ATLAS PhD student), Ellen Yi-Luen Do (ATLAS ACME Lab director, faculty member)

This paper explores the implementation of embedded magnets to enhance paper-based interactions. The integration of magnets in paper-based interactions simplifies the fabrication process, making it more accessible for building soft robotics systems. We discuss various interaction patterns achievable through this approach and highlight their potential applications.

[Workshop]
Laura Devendorf (ATLAS Unstable Design Lab director, assistant professor), Leah Buechley, Noura Howell, Jennifer Jacobs, Hsin-Liu (Cindy) Kao, Martin Murer, Daniela Rosner, Nica Ross, Robert Soden, Jared Tso, Clement Zheng (ATLAS PhD alum)

While cross-disciplinary collaboration has long been, and continues to be a cornerstone of inventive work in interactive design, the infrastructures of academia, as well as barriers to participation imposed by our professional organizations, make collaboration for some groups harder than others. In this workshop, we’ll focus specifically on how artists residencies are addressing (or not) the challenges that artists, craftspeople, and/or independent designers face when collaborating with researchers affiliated with DIS. While focusing on the question “what is mutual benefit”, this workshop seeks to combine the perspectives of artists as well as researchers collaborating with artists (through residencies or otherwise) to (1) reflect on benefits or deficiencies in what we are currently doing and (2) generate resources for our community to effectively structure and evaluate our methods of collaboration with artists. Our hope is to provide recognition of and pathways for equitable inclusion of artists as a first step towards broader infrastructural change. 

Refer to the for more details on this research. 

 

[Demo]
Ruojia Sun (ATLAS PhD student), Althea Vail Wallop (ATLAS MS student), Grace Leslie (ATLAS Brain Music Lab director, assistant professor), Ellen Yi-Luen Do (ATLAS ACME Lab director, faculty member)

Movement forms the basis of our thoughts, emotions, and ways of being in the world. Informed by somaesthetics, we design for "taking up space" (e.g. encouraging expansive body movements), which may in turn alter our emotional experience. We demonstrate SoniSpace, an expressive movement interaction experience that uses movement sonification and visualization to encourage users to take up space with their body. We use a first-person design approach to embed qualities of awareness, exploration, and comfort into the sound and visual design to promote authentic and enjoyable movement expression regardless of prior movement experience. Preliminary results from 20 user experiences with the system show that users felt more comfortable with taking up space and with movement in general following the interaction. We discuss our findings about designing for somatically-focused movement interactions and directions for future work.

[Demo]
Mikhaila Friske (ATLAS affiliate PhD student)

This demo focuses around crocheting and data. In addition to a physical workbook for conference goers to peruse and try, there will be a few small set-ups for specific activities and a small craft circle for people to craft within if they so choose.

Off