The ability to automatically build 3D digital twins of plants from images has countless applications in agriculture, environmental science, robotics, and other fields. However, current 3D reconstruction methods fail to recover complete shapes of plants due to heavy occlusion and complex geometries. In this work, we present a novel method for 3D modeling of agricultural crops based on optimizing a parametric model of plant morphology via inverse procedural modeling. Our method first estimates depth maps by fitting a neural radiance field and then optimizes a specialized loss to estimate morphological parameters that result in consistent depth renderings. The resulting 3D model is complete and biologically plausible. We validate our method on a dataset of real images of agricultural fields, and demonstrate that the reconstructed canopies can be used for a variety of monitoring and simulation applications.

Plants in crop fields typically consist of many overlapping leaves that heavily occlude each other, making 3D reconstruction challenging. Our method overcomes this issue and produces morphologically complete 3D crop canopy models through inverse procedural modeling. We validate its performance on a dataset of images collected in real agricultural fields. On the left are sample images from the dataset, and on the right are renderings of the procedurally generated 3D models produced by our method.

Using Helios, a state-of-the-art biophysical modeling framework, we can perform radiative transfer simulations on the crop canopies reconstructed by our method. This allows us to predict the net photosynthesis rate for every leaf mesh face over time given a set of environmental variables. We show visualizations of the per-face photosynthesis rate and the timeseries of the aggregate rate across the entire canopy (units: µmol CO₂m^-2s^-1), which can be compared with ground-truth values calculated using flux tower eddy-covariance data.

The results demonstrate our pipeline's potential for facilitating large-scale monitoring of crop productivity.