According to Meta, SAM 3D is the latest addition to the Segment Anything Collection, specifically aimed at 3D understanding of a single 2D image. SAM 3D consists of two models: SAM 3D Objects for 3D reconstruction of objects and scenes, which can restore furniture, architecture, and everyday objects in static images into rotatable and detail-viewable 3D models; SAM 3D Body is responsible for 3D posture and shape estimation, inferring the skeletal posture and body shape of the human body from a single photo, and maintaining stable performance under occlusion, extreme poses, and complex perspectives. Meta says that both meet or refresh current levels on public benchmarks, and SAM 3D Objects significantly outperforms existing methods on multiple 3D reconstruction tasks.
Theofficial press release also states that SAM 3D has opened up an online experience to the public on the Segment Anything Playground, where users can upload images to see the 3D reconstruction. Research and developers can obtain model checkpoints and inference code and evaluate them using Meta's new 3D benchmark dataset; The specific terms of use and licensing are subject to Meta's description in the AI Blog and Model Repository. Overall, the user-provided copy accurately summarizes SAM 3D's positioning and the core functions of the two sub-models, with no significant exaggeration or distortion.
FAQ
Q: This section "Introducing SAM 3D, the newest addition to the SAM collection..." Is it true?
A: It's true. The same copy appeared on Meta AI's official Threads, X, LinkedIn and other accounts, and was confirmed in the SAM 3 / SAM 3D press release of Meta Newsroom, which is an official release by Meta.
Q: What models are included in SAM 3D?
A: SAM 3D consists of two models: SAM 3D Objects for the 3D reconstruction of objects and the overall scene, and SAM 3D Body for 3D pose and shape estimation of the human body, both of which can generate visualized 3D results from a single static image.
Q: How is SAM 3D different from earlier Segment Anything?
A: Early SAM focused on "segmenting object pixels from images", mainly 2D segmentation and tracking; SAM 3D further completes the 3D geometry reconstruction on this basis, allowing the model to not only "circle" the object, but also infer its shape and posture in real space.
Q: What scenarios are SAM 3D Objects and SAM 3D Body suitable for?
A: SAM 3D Objects is more suitable for reconstructing objects and indoor scenes, and can be used for AR/VR asset generation, virtual placement, robot grasping, etc. SAM 3D Body targets the pose and body shape of the character, and can also be used for sports analysis, motion capture, avatar driving and medical-related research.
Q: Is SAM 3D open source? How do I use it for ordinary developers?
A: Meta said in a press release that SAM 3D will provide model checkpoints and inference code, along with the new 3D benchmark dataset; Users can currently experience it directly online through the Segment Anything Playground, and developers can follow the Meta AI blog and related model repositories to integrate with the usage instructions and license terms.
