Are holograms the future of how we capture memories?

When Los Angeles-based actress and interior designer Ashley Martin Scott responded to a casting call for “mom and baby” back in April 2015, details were scant.

“I pretty much came into it blindly, not knowing what to expect, aside from ‘a mom and a baby,’” Scott said. “And that it was something about leaving a message for your child in the future and I thought, ‘That sounds fun.’”

The casting call led her to the 8i studios, then in Culver City, California. That’s when Scott discovered this wasn’t a typical casting call: she and four-month-old Reese were being turned into holograms, a term that’s used (and sometimes overused) to describe volumetric video. 8i was looking to build up its repository of holograms, and a mom-and-baby clip seemed like a good idea for relatable content.

Scott’s first hologram taping led to her crying in a VR headset, and became what the company’s chief executive called a “watershed moment” for the then-fledgling company.

8i’s soundstage is made up of a massive, high-ceilinged green screen, with 41 off-the-shelf cameras aimed at a small circle in the center of the stage. These are the cameras that have captured Ashley and baby Reese, tigers and llamas, NBA stars, wrestlers, actor Jon Hamm, and more. All of the raw image data from the cameras is sent to a computer, where 8i’s proprietary software fills in any data gaps and renders the video. Then it spits out a short, holographic video vignette, one that can be viewed in either a VR headset or in 8i’s mobile app, Holo.

It’s a highly technical process, one that 8i — and other companies — have been trying to perfect as volumetric content has emerged as the “next big thing” in video. But for Scott, her first 8i experience was more emotional than technical. When she first put on a VR headset to view her finished holograms, she started to cry.

Scott attributes her reaction partly to the realness of the volumetric video, which lets you walk around a digital object, get close to it, peer at it, and in a limited way, interact with it. She recalled reaching her arms out, as if to hold her child, and experiencing “phantom” limbs; she knew she was moving them, but they weren’t there in the virtual world. She described the eerie feeling of stepping back into her own body, and looking down at holographic Reese as though she were a real baby.

“I think when you’re dealing with a newborn, it feels like that stage is going to last forever, because your days are long and your nights are longer. But it really goes by so fast,” Scott said. “So to be able to jump into that again was overwhelming.”

“This was really early on in the company, and we weren’t quite to the spot where we thought the tech would all work,” 8i chief executive Steve Raymond said in an interview with The Verge. “But we were looking for ways to test the emotional connectivity that could happen between a real person and a digital person. It was a real watershed moment in the company.”

Scott ended up returning to 8i four times over two years to get a series of holograms made of Reese (now nearly three) and her one-year-old brother, Wilder. Other 8i employees and friends of the company also started to take advantage of their access to the studio, bringing their own kids in to capture holograms: next-generation baby photos that they can revisit in headsets, on their smartphones, or even in their Instagram feeds.

“Everybody wants holograms of their kids,” Nicole St. Jean, 8i’s vice president of content and a former Twitter executive, told me. St. Jean held up her iPhone and showed me an Instagram video of her son Lowell as an example. Only, it wasn’t just one Lowell in the clip: there was a one-year-old Lowell, juxtaposed with an almost two-year-old Lowell. One of these toddlers was a hologram.

“If 8i [brought] what I got to experience to everybody, that would be tremendously fulfilling for me as a parent,” St. Jean said. “If I got to have holographic memories of my parents... it really is like having a memory. It is like seeing the person you remember right in front of you.”

I’d first heard about 8i in January of this year, and spent a day in Los Angeles learning about their approach to making volumetric video. (I even had a hologram of myself made.) It was during that reporting trip that someone mentioned a local actress who had been coming in to make holograms of her kids, so she could revisit her holographic children once they were grown.

I thought the use case sounded mildly bizarre at first — especially since this isn’t a service that 8i offers to the general public — but I was also intrigued by the possibilities that came with it. The vast majority of the AR and VR content I’ve experienced has been centered around gaming or entertainment. Here was an example of personal AR and VR. 3D video, spherical video, volumetric video — whichever format you’re using, the promise of this kind of media is that it’s supposed to offer a more immersive experience, creating environments and people that feel real.

So what happens when you use these tools to capture humans for the sake of posterity? Can holographic videos ever truly replace the human experience?

That is how I ended up back at 8i’s offices last month, following Scott around as we taped an episode of The Verge’s video series Next Level. 8i has since moved its offices from Culver City to Playa Vista, upgrading from a famed Hollywood soundstage to an airy, industrial office space that has a distinctly tech startup feel. Edgy street art is visible from 8i’s all-glass front entrance, with a variation of an Einstein quote scrawled alongside the mural: “Striving for truth and beauty is a sphere of activity in which we are permitted to remain children all our lives.”

We also visited USC’s Shoah Foundation in downtown Los Angeles. Since 1994, the foundation, which was established by Steven Spielberg, has been recording the life stories of genocide survivors. A newer project within the foundation, called “New Dimensions in Testimony,” or NDT, has been experimenting with AR and VR to capture these testimonies.

“So far, virtual reality has been very much about gaming, or has been about creating environments that you can navigate and explore and so forth,” said Steven Smith, the executive director of the Shoah Foundation. “But actually there’s nothing more intimate than speaking another human being, face to face. So we’ve seen multiple ways in which VR can be used for testimony.”

At the Shoah Foundation, I was able to converse with a still-living Holocaust survivor named Pinchas Gutter. Pinchas wasn’t really there, though; I was chatting with a hologram of Pinchas, which appeared on a flat, 2D display in the hallway. The conversation felt almost absurdly natural, due in large part to the foundation’s development of its own natural language processing system. At one point, I realized I felt rude interrupting a video.

“I think the immediacy of this format brings you really close to the subject,” Smith said of the Pinchas video. “And I think it’s going to become the standard way in which we document our history.”

A lot has changed in the broader market for AR and VR video over the past few years, since 8i first started making holograms and the Shoah Foundation began the NDT project. While virtual reality content tends to be less accessible to consumers — requiring either a high-powered, wired VR headset, or a less powerful mobile one — research firm IDC has still predicted that dedicated VR and AR headsets could collectively jump up from just under 10 million units shipped last year, to 100 million units by 2021. The market for augmented reality content hit a critical turning point this fall when both Apple and Google rolled out new smartphone platforms for AR.

Microsoft also recently announcement that it was opening up holographic video studios in San Francisco and London. Microsoft has been an early pioneer in what it calls mixed reality: showing AR objects and applications, but often through the lens of its HoloLens headset. Now, developers, producers, and marketers with the budget to afford it could theoretically go make holographic videos in Microsoft-licensed studios.

Raymond said he believes that holographic video studios will eventually just “be everywhere,” once certain technical hurdles have been overcome and the cost of making high-quality holograms comes down. And this would allow average consumers to create hyper-realistic time capsules of the people closest to us — or, the people we wish were still close to us.

“People are going to want to put their loved ones into their phones for lots of reasons,” he said. “The first step will be capture studios where you have to go to a department store like we did when we were little kids. But then it will become something you can just buy on Amazon and have it installed in your living room and make it at home.”