Canon’s Dual Pixel CMOS AF sensor has been doing something quietly remarkable for years: recording two slightly offset photodiode readings for every pixel across the sensor plane. The system was designed for phase-detection autofocus, and it’s very good at that job. What Canon’s Dual Pixel 3D research project recognized is that those paired readings contain something else entirely — a per-pixel disparity map, essentially a depth field baked into a single exposure, without a second lens or a stereo rig.
This article is about what that means for conversion workflows: specifically, how image data from an EOS R-series body can become a usable 3D model, what the pipeline actually looks like step by step, and where the real friction points are for anyone trying to move from a CR3 file to exportable geometry.
What Dual Pixel Data Actually Contains
Standard image sensors record one photon value per pixel (or per Bayer sub-pixel). A Dual Pixel sensor splits each photosite into two halves — left-facing and right-facing — and reads them separately before combining them into the final output. The combined value is what goes into your JPEG or RAW file as luminance. The difference between the two half-pixel values is what the AF system uses to calculate phase shift and drive the lens motor.
That per-pixel phase difference is effectively a disparity. Disparity, in stereo vision geometry, encodes depth: a subject close to the camera creates a larger apparent offset between two viewpoints; a distant subject creates a smaller one. A Dual Pixel sensor’s two sub-aperture views are separated by roughly half the lens aperture rather than the several centimeters of a stereo baseline, so the disparity values are small — but they’re real, spatially dense, and tied directly to the optical geometry of the lens in use.
The challenge is that Canon’s standard CR3 RAW format doesn’t preserve the raw dual-pixel channel data for third-party use. The EOS R5 and related bodies do produce dual-pixel RAW files (DPRAW), a Canon-specific option that retains the left/right sub-frame data alongside the standard merged exposure. Those files are larger than standard CR3s, and the additional data has historically been intended for Canon’s own software to apply micro-adjustments to bokeh rendering and facial lighting. Canon’s Dual Pixel 3D research repurposes that same stored disparity information for depth reconstruction instead.
The Conversion Pipeline, Stage by Stage
Turning a DPRAW file into 3D geometry involves several distinct processing steps, and each one is a potential source of error or quality loss.
1. Sub-aperture extraction
The DPRAW file is read to separate the left-channel and right-channel images. These aren’t two full-resolution frames — each is a sub-aperture view with roughly half the effective aperture of the combined image. At wide apertures, the separation is meaningful; stop down to f/8 or f/11 and the baseline shrinks proportionally, reducing depth resolution.
2. Disparity estimation
A stereo matching algorithm compares corresponding patches in the left and right sub-aperture images to estimate per-pixel horizontal disparity. This is computationally similar to what any stereo depth estimation system does, except the baseline is very short and both images come from the same sensor plane. Patch-based matching methods, semi-global matching (SGM), and more recent learned disparity networks have all been applied to this problem. The output is a dense disparity map — a floating-point image where each pixel’s value encodes relative depth.
3. Depth map conversion
Disparity converts to physical depth using the lens’s focal length, the known sensor geometry, and the aperture setting recorded in EXIF metadata. The math is well-established from stereo photogrammetry: depth is proportional to focal length times the physical sub-aperture baseline, divided by disparity. Metadata accuracy matters here — if the EXIF focal length is approximate or the lens has uncorrected distortion, depth values will drift accordingly.
4. Point cloud generation
With a depth value assigned to each pixel, software back-projects each point into 3D space using the camera’s intrinsic parameters. The result is a dense colored point cloud — one point per pixel, positioned in XYZ coordinates derived from the depth estimate, colored by the pixel’s RGB value from the merged image. Dense point clouds from this method can run into the tens of millions of points for a full-resolution EOS R5 exposure.
5. Mesh reconstruction and cleanup
Point clouds are not the same as meshes. Converting a point cloud to a polygonal surface typically involves Poisson surface reconstruction or ball-pivoting algorithms. Thin structures, specular surfaces, and depth discontinuities at edges all create reconstruction artifacts: holes, floating fragments, or false surfaces. Depth estimation from a very short stereo baseline is also substantially noisier than LiDAR or structured light, so mesh smoothing is usually necessary — with the attendant risk of washing out fine geometry.
6. Format export
The final mesh can be exported in any standard geometry format: OBJ, PLY, GLTF, or STL depending on the downstream use. Texture coordinates are generated by reprojecting the original pixel colors back onto the mesh surface.
What Works Well and What Doesn’t
Subjects that suit this workflow tend to share a few properties: single-focus-plane or shallow-depth scenes (where the dual-pixel baseline provides meaningful disparity), matte surfaces (which don’t confuse patch-matching with specular highlights), and sufficient texture for the stereo matcher to find correspondences. A face, a textured ceramic object, or a plant photographed at f/2.8 works measurably better than a glass bottle, a reflective car panel, or a scene with deep foreground-to-background range.
Foliage, hair, and fine wire structures create persistent problems. The dual-pixel baseline is too short to resolve depth at sub-pixel detail in those areas, so thin structures either collapse to a single depth plane or produce noisy depth estimates that triangulate erratically in 3D space.
Lens choice also matters in a way it doesn’t for conventional 3D scanning. A longer focal length increases the effective magnification of the disparity at a given scene distance, which improves depth resolution but compresses the scene’s depth range simultaneously. A wider lens increases working depth range but degrades per-pixel depth precision. Shooting at the widest usable aperture is consistently advisable — the sub-aperture baseline scales linearly with the physical aperture diameter.
Practical Workflow Considerations
Anyone attempting this pipeline outside Canon’s own software environment will encounter the DPRAW data access problem first. The DPRAW format is documented in Canon’s Digital Photo Professional (DPP) application, which surfaces the dual-pixel data for its own bokeh and lighting tools but does not expose it as a raw disparity map for export. Third-party DPRAW support, including the libraries that underpin much of the open-source camera RAW ecosystem, has historically treated the dual-pixel channels as implementation details rather than user-facing data. Check the current state of whichever RAW library you plan to use — this is exactly the kind of support detail that changes between versions.
Assuming you can extract the sub-aperture channels, the stereo matching and reconstruction steps can be handled in open-source tools like OpenCV (for disparity estimation), MeshLab or Open3D (for point cloud processing), and Blender (for mesh refinement and texture baking). The pipeline is not a single-click operation. Calibrating the camera intrinsics for your specific lens and aperture — rather than relying on EXIF values alone — will measurably improve depth accuracy, particularly at close focus distances where lens distortion deviates most from the paraxial model.
For anyone already maintaining a structured archive of their EOS R files, it’s worth noting that enabling DPRAW mode at capture time is a prerequisite — you cannot reconstruct the dual-pixel sub-frames from a standard CR3 after the fact. Deciding to capture DPRAW adds file size and some write-speed overhead, so it’s a deliberate workflow choice rather than something to leave on by default for every shoot.
Where This Fits in a Broader Conversion Workflow
Dual Pixel 3D sits in a specific and fairly narrow niche. It is not a replacement for photogrammetry workflows that aggregate dozens of overlapping frames for high-accuracy surface reconstruction. It is not comparable to structured light scanning for objects where precise measurement matters. What it does offer is single-capture depth reconstruction from hardware that photographers already own, without a specialized scanning rig or multi-shot capture sequence.
The practical applications tend toward visualization — generating a rough 3D model of a subject for compositing, previsualization, or interactive display — rather than engineering or archival measurement. Portrait photographers experimenting with synthetic relighting or volumetric effects have a more natural use case here than product photographers who need dimensional accuracy.
If you’re working in Canon’s EOS R ecosystem and haven’t yet explored what dual-pixel RAW capture actually stores, start there: enable DPRAW in your camera’s RAW recording options, shoot a frame with a textured subject at a wide aperture, and examine what DPP can do with the sub-frame data. That’s the foundation the 3D reconstruction pipeline builds on, and understanding what’s actually in the file is more useful than treating it as a black box. Our Conversion Workflows coverage tracks related file-to-format pipelines as this area develops.