My Page

Human Fusions - XPRIZE Semifinals Photo Journal Across the country, working together

Creating an Avatar

We Created the Avatar

As an Avatar

The team first met by teleconference. The entire design and development of the HFi Sensa Avatar was completed over Zoom.

In October of 2020, the core Sensa systems and the Neural Reality Platform were completed, providing the connection infrastructure.

The first physical connection occurred over the internet - shaking hands 2,350 miles across the country.

The ANA Avatar XPRIZE semifinals entry.

Arriving in Miami

The team arrived in Miami, FL on September 8, 2021.

After more than a year of working together, this was the first time the team met in person.

“Wow! you are not the height I had imagined.”

However, from the first meal, the chemistry was immediately obvious. The common thread of quality, competence, and kindness ran through every member.

Within hours, hotel rooms exploded into workshops with Case Western Reserve University and University of California, Los Angeles students working seamlessly side-by-side.

Build Day 1

First day: Discussion and planning

We move to the “garage” and full assembly begins in earnest

The system starts to take shape

The human connection system is fired up.

Human - technology immersion link initiated!

Build Day 2

The team was efficient on day 1 and completed the build. While most of the other teams continued to assemble their systems, Human Fusions went into full test and refine mode.

Systems were tested on multiple people and tweaked to fit all.

We focused on testing the system.

Emphasis was placed on crafting and understanding the human experience.

We adjusted parameters to enable efficient performance of functional tasks

But our main focus thoughout the day was on human interaction

Preparing to train the judges. Goal is to focus on the human connection - to treat the avatar as a person, not as technology.

Developing human embodied experience through multisensory synchrony …

… and human connections through familiar gestures.

After a long day of testing and refinement, we are ready to go. Time to shut down and get some rest…

But then disaster!

At 9:30 pm, the night before judging, a fuse blows during shutdown and fails to accomplish its only purpose of protecting the system. The robot’s brain is fried.

The entire team stayed up, worked together, and had a lot of fun getting the backup robot up and running. By 4 am, we were ready for a 3 hour nap before heading to the judging sessions. Heroic efforts by the team.

Judging

For the next two days, the team trained judges on the Sensa Avatar system and then left the room while the judges performed their tests. Only two team technicians could be in each room, but they were not permitted to talk or interact with the judges.

Operator in Cleveland

The human operator, or the judge with the human interface that would become the Avatar, was greeted in the room we labeled “Cleveland” to keep with the team’s unique development story.

The judge donned the haptic glove and neural stimulation was tuned. Every judge felt sensation in five fingers and the palm.

While Cleveland setup the operator judge, the Los Angeles team (in another room) chatted with the recipient judges about the team’s story. We also told the judge to treat the avatar as a person. Violating the personal space of the robot is felt by the operator as a violation of their personal space.

Once the judge was setup in Cleveland, they were fitted in the headset and asked to close their eyes.

The system was initiated and linked …

… and, the operator was transported to Los Angeles.

“You can open your eyes. Hi, judge X, welcome to LA.”

The judge was the trained about their new Avatar body in LA.

Teaching the operator/avatar judge how to use their new body. This was a lot like traditional rehabilitation following an injury. Real possible applications to medical training!

When they mastered training and were ready, we celebrated a little bit and then left the room.

Of course, before we left, we had to introduce the judges to each other.

We will learn our fate in October of 2021.

Team pictures after the judging!

The real heros who did all the amazing work.

The PIs that appreciated the work.

The Los Angeles team

The hero who performed a robot brain transplant the night before.

The Cleveland team

(The phenomenologist!)

A short look at the Tech

The secret sauce… gloves that provide the neural connections for the sense of touch.

Connected to an Oculus Quest 2 system for audio and visual feedback.

Our approach is to create a system focused on

Mobility
Accessibility
Deployability

We thank Hello Robot for their support and for providing a backup system.

We leveraged our expertise in prosthetics for the gripper. Where else would you get the best anthropomorphic hands designed to be a replacement hand?

Thank you to Taska, LLC for use of the sensorized TASKA Hand.

Sensa’s eyes are a ZED mini camera. The binocular eyes created an important connection with the human interacting with Sensa.

LIDAR for collision avoidance.

The whole, but unimposing, approachable system.

All agreed,

Whether or not we proceed,

We did already succeed!

David Locke, XPRIZE organizer, gives the Human Fusions team a recognition of their accomplishments.

The Human Fusions team that attended the Miami semifinals. We all wish everyone could have attended. The full fusions avatar team consists of more colleagues from Case Western Reserve University, UCLA, Cleveland State University, and the University of Wyoming.The team is continuing to grow, and will build on this first step to develop avatars and avatar applications.

If interested in more information or joining the team, please visit or connect with us at one of the following:

Website: https://www.humanfusions.org

email: HumanFusions@case.edu

Twitter: @HumanFusions

Appreciate

Credits:

Photos by Dustin Tyler; Veronica Santos; Jonathan Bopp; Michael Hill