Niantic, Inc.

🚀Niantic is back at ECCV 2024 with more original research! We’re excited to announce that the Niantic Research and Mapping teams will be presenting four groundbreaking papers that broaden the horizons of 3D reconstruction, depth estimation, and real-time mesh prediction. The team will also be “including an oral presentation of ‘ACE Zero’, a new method for 3D reconstruction using a neural network to efficiently reconstruct high quality scenes.” Some other highlights from our research include: 🔹 DoubleTake: Geometry Guided Depth Estimation for state-of-the-art depth maps and meshes.  🔹 GroundUp: A sketch-based 3D city modeling tool for rapid urban massing.  🔹 HandDGP: Accurate, real-time 3D hand mesh prediction from camera feeds. In addition, we’re hosting a Map-free Visual Relocalization Workshop & Challenge, “where teams and algorithms are tasked with estimating metric camera pose for a query image given a single reference image.” Researchers from top institutions and organizations, including Meta, Google, CTU Prague, and Oxford University, will be presenting their innovative solutions. Cash prizes will be awarded! Computer vision is at the heart of Niantic’s core and we’re proud to be a part of spatial computing research and innovation. Check out our full blog here: https://lnkd.in/gXQWBw2p #ECCV2024 #SpatialComputing #AugmentedReality

We're thrilled to announce a groundbreaking partnership with Meta, bringing even more magic to Peridot’s “Hello, Dot” experience on the Meta Quest 3! We’re pushing the limits of mixed reality with immersive 3D gameplay, fun accessories, and new charming game features. ✨ Check it out: https://lnkd.in/gTcs8wMv

It's an exciting time for augmented reality and spatial computing with many developments just this month! Everyone is staking their ground, and we expect more momentum heading into next year – a year where AR, AI, and games are converging, and a new computing platform is forming. What’s become even clearer is that spatial computing will require a new map. One that we've been pouring nearly a decade into building since the very early days of Niantic. Our considerable investment in our geospatial games, like Pokémon GO and more, has paved the way for this 3D platform to emerge, including our 3D scanning app Scaniverse. You can read more about our work building towards an outdoor spatial computing future in this post: https://lnkd.in/gP-k_ncJ

I just published my Voices of VR podcast deep dive into my coverage of the Snap Spectacles and Snapchat's AR ecosystem with over 8 hours of interviews across 15 Voices of VR podcast and featuring 26 different insights from indie AR devs Snap reps (link at the bottom links to each episode). I was pleasantly surprised with how viable of an AR dev kit the Spectacles is with a 13ms motion-to-photon latency, very responsive and intuitive hand-tracking, and Snap's top notch ML. But more than anything else, Snap has the most creative, exciting, and talented set of indie AR devs who have a viable platform to hone their AR lens skills, and that same lens workflow with Lens Studio ports over into building experiences for the Spectacles. Lots of concern about the 45-min battery, but you can also plug it into a battery pack & then you're limited by how comfortable the thermals will be. Snap has optimized for outdoor use, and their Connected Lens framework enables shared experiences, which the Es Devlin Council experience demonstrates powerfully. Much more of my thoughts in the first & last episodes, but each episode is a deep dive into another corner of the Snap ecosystem: Kickoff of Snap Spectacles Coverage with First Impressions with XR Analyst Anshel Sag Niantic Launches Virtual Pet AR Game "Peridot Beyond" on Snap's Spectacles - Erin Schaefer and Asim Ahmed Wabisabi Game Studio Leverages AR Connected Lenses on Snap Spectacles for Outdoor Capture the Flag Game - Anwar Noriega and Julio Martinez Turning Your Phone into a Spectacles AR controller with DB Creations' "Tiny Motor Arcade" - Dustin Kochensparger and Blake Gross Snap's Lens Studio, Spectacles, & "Polygon Studio" Open Source Low-Poly Drawing App - Max van Leeuwen and Liisi Mononen Building a Geo-Located, Dungeon-Raiding, AR Game for Spectacles with Aidan Wolf Snap Co-Founders Share Vision for AR & Spectacles at Lens Fest Q&A Panel - Evan Spiegel and Bobby Murphy Snap AR Platform Lead Sophia Dominguez on Spectacles and AR Ecosystem AR Lens Genres, Spectacles Impressions, & SelfReflect VTubing App Using Snap's Camera Kit with Brielle Garcia XR Industry Reality Checks with Cix Liv and Turning Your Phone into an AI-Driven Exercise Routine Arguing for Utility-Driven AR Minimalism with Lucas Rizzotto Reflecting upon the XR Market Snap's Head of Spectacles Software Engineering Shares Technical Break Down of Design Tradeoffs - Daniel Wagner Team of Women AR Artists Explore Dreams in Spectacles Hackathon - Alexis Zerafa, Lafiya Watson Ramirez, Meredith Binnette, Heather Dunaway Smith, and Nella Freelance XR Dev Veronica Flint Shares Thoughts on Spectacles and Ray-Ban Meta Smart Glasses - Veronica F. AR Medical Training App Wins Spectacles Hackathon My Concluding Thoughts on Snap Spectacles - Sam Jones, Kathryn Hicks, Lauren Cason, Emma Sofija, and Paige Piskin https://lnkd.in/gex75z9p

Experience the future of spatial computing 🌍🚀 With Scaniverse, it's easier than ever to get started with splatting—capturing life in 3D, scanning the world around you, and enhancing your creations. By sharing your splats, you’re not just engaging with your community—you’re helping lay the groundwork for a new era of immersive apps and games. Let’s build a more interactive world, one splat at a time. FUN today, FUNdamental for tomorrow. Download Scaniverse now: https://scaniverse.com/ #Scaniverse #SpatialComputing #AR #Niantic #TechInnovation #FutureOfGaming

We’re excited to partner with Snap to bring to life amazing augmented reality experiences. We both share a common philosophy that technology can bring us closer to the real world around us — and not close us off. We also introduced Scaniverse support for Snap Spectacles, which is a foundational technology for anchoring spatial content for AR glasses. It was a big day!

One More Step Towards Consumer AR I’ve spent the better part of the last decade working on AR, and during that time, the industry has seen its share of false starts. But this month, I believe we’re finally starting to see the arrival of hardware that will ultimately power the future of personal connectivity and computing. Across the industry, we see companies shifting their focus from VR to lightweight AR glasses that can be worn outside. Why? It's becoming more and more clear that consumers don’t want to be trapped behind heavy goggles, cut off from the world and from other people. Devices that come with us wherever we go and help us accomplish more hold far more promise in terms of usefulness and broad appeal. Today marks an important milestone with the debut of Snap’s fifth generation of its AR wearable product, Spectacles, and our collaboration with them. Although not yet a finished consumer product, this latest iteration of Spectacles (targeted primarily at developers) clearly demonstrates a path to future outdoor AR glasses.  In line with this, we are launching two key product integrations.  Peridot Beyond – An exclusive experience for Spectacles based on a reimagined version of our AR pet game. With Dots by your side, you can explore the world together - finding food, playing catch, and soon, connecting with friends who have Spectacles, so your Dots can meet. I think virtual assistants will be a killer app in AR, and pets will be too!  Perhaps a virtual assistant and a pet can be the same thing? 🤔 Niantic Scaniverse – We just launched Niantic Scaniverse 4.0, featuring groundbreaking Gaussian splats for creating and sharing 3D scans of places and objects. Now, you can import your 3D scans into Lens Studio, so that any place, moment, or memory can be experienced through Spectacles. Want to see how that new cactus would look in your garden? Scan it with Niantic Scaniverse and view it through Spectacles, right there in front of you. This is just the start as companies strive to solve the remaining challenges to bring truly consumer friendly AR hardware to market. But it feels like the industry has definitely turned a corner. A big part of this, in my opinion, is the rise of AI and clear use cases for allowing easy and frictionless access to an AI assistant through wearable technology such as AR glasses. To support this shift, we’re focused on building the world’s most advanced spatial platform and map. This new map, geared for computer vision, enables precise alignment of digital creations with the physical world.  It can also provide rich spatial context for AI assistants through continuous localization and semantic analysis of video inputs. The fusion of glasses, AI, and games is laying the groundwork for the next computing platform, and we’re excited to be leading the way.

We’re partnering with Snap for their 5th gen Spectacles! Enjoy hands-free AR with Peridot—more cuddles, less phone—and Scaniverse, letting you explore your 3D scans anywhere. Together, we're shaping the future of immersive, real-world experiences. Read on: https://lnkd.in/gZhA-u-w

🌀✨ Stepping into the future with Gaussian Splat portals🚪🔮 I had a blast developing these real-time augmented reality portals. Imagine walking through a doorway and ending up... anywhere! Where would you want your AR portal to lead? 🌎 Here’s how I created it: 1. Capture: Scan the environment using a 3D scanning app to create a Gaussian splat (Scaniverse) 2. Create: Import the splat into an AR development platform and add portal effects (Niantic Studio) 3. Experience: Deploy and step through your personal AR portal! (WebAR) #AugmentedReality #GaussianSplat 

Community and mapping are core pillars of Niantic’s mission. With the new Scaniverse 4, we can usher in a new era of spatial computing. The Scaniverse community is filled with XR enthusiasts, developers, 3D photographers and explorers. We can now scan, create, and play in Scaniverse to build the next monumental map of our generation. Get the full scoop on Niantic Scaniverse 4 and the new mapping feature straight from Brian McClendon's keynote speech at AWE Asia 2024. https://lnkd.in/gKs98wqj