What is Meta Codec Avatars?

Lex Fridman conducted a virtual reality interview with Mark Zuckerberg using Meta's cutting-edge prototype photorealistic Codec Avatars. During the interview, Fridman and Zuckerberg used photorealistic avatars from the ongoing development of Codec Avatars.

Meta's Codec Avatars represent a significant and ongoing research project aimed at transforming remote communication. The goal is to create remarkably convincing, real-time avatars driven by headsets equipped with face-tracking sensors, and eventually extend the technology to glasses as well.

Click the image above to be taken to YouTube, where you can watch the full interview. For reference, this is the video outline:

0:00 - Introduction | 0:52 - Metaverse | 15:27 - Quest 3 | 30:16 - Nature of reality | 34:54 - AI in the Metaverse | 51:51 - Large language models | 57:49 - Future of humanity

Ray-Ban | Meta Smart Glasses

Improved audio: Speakers with extended bass, higher maximum volume, and improved directional audio that provides reduced audio leakage for improved calls, music, and podcasts — even in noisy or windy environments. When recording a video, the five-microphone array supports immersive audio recording, so sounds are recording accurately from five directions.

Video: Ultra-wide 12 MP camera translates to improved quality for photos and 1080p videos up to 60 seconds. You can directly share photos to friends and family from your glasses with a simple “send a photo” voice command.

Processing: Are powered by the Qualcomm Snapdragon AR1 Gen1 Platform, which enables higher quality photo and video processing and faster compute.

Livestreaming: Can now livestream from the glasses to Facebook or Instagram.  And users can see the comments in their preview, or tap and hold on the side of their glasses to hear them out loud.

"Meta AI: Integrated Meta AI on Ray-Ban Meta smart glasses. By saying “Hey Meta,” you can engage with Meta AI to get information, and control features — just by using your voice. Meta AI features will be available in the US in beta only at launch."

According to the official Meta post, "Ray-Ban | Meta Smart Glasses Collection will be available in 15 countries, including the US, Canada, Australia, and throughout Europe."

Click the image above to be taken to YouTube, where you can watch the full video.

ChatGPT can now see, hear, and speak

"To get started with voice, head to Settings → New Features on the mobile app and opt into voice conversations. Then, tap the headphone button located in the top-right corner of the home screen and choose your preferred voice out of five different voices.

The new voice capability is powered by a new text-to-speech model, capable of generating human-like audio from just text and a few seconds of sample speech. We collaborated with professional voice actors to create each of the voices. We also use Whisper, our open-source speech recognition system, to transcribe your spoken words into text."

Read the full release announcement here, which also includes voice samples and a lot of useful information: Click Here