Stem Removal from Point Clouds for Leaf Phenotype Analysis

This subsection outlines a method for removing stem points from preprocessed point clouds using normal features. The primary motivations for stem removal are:

1. Our research focuses on studying leaf phenotypic features, not stem features.
2. Stem points can significantly hinder the identification and analysis of leaves in the point cloud.

Normals are crucial spatial features for unorganized point clouds. Points on a flat surface have smooth, unchanging normals. However, the normals of points on irregular local structures vary greatly. Leaves on most plants are relatively flat, with minimal variation in normal directions within the same leaf's local region. Stems, due to their cylindrical shape, exhibit significant normal variations. This difference allows us to distinguish leaf areas from the stem system in the point cloud by analyzing changes in normals within a local region.

The DoN (Difference of Normals) operator is a multiscale filtering technique applicable to massive 3D point clouds [42]. By calculating the normal difference of the same point at different scales, we can determine if the point originates from a flat surface.

A common approach for computing normals involves establishing a set containing points within a local neighborhood with a fixed radius for each point. Principal Component Analysis (PCA) is then performed on this set to estimate the point's normal. Formally, the DoN operator Δn for any point pi in a point cloud P is defined as follows:

where r1 and r2 (r1 is smaller) represent the two different support radii at point pi. n(pi, r) is the normal of pi with the support radius r. Δn(pi, r1, r2) represents the magnitude of the difference between the two normals under support radii r1 and r2.

Note that for each single plane, there exist two normal vectors in opposite directions, both being correct. To avoid ambiguity, the angle between the two normal vectors is calculated first to determine if they are on the same side before calculating DoN. If n(pi, r1) · n(pi, r2) < 0, then one normal vector needs to be reversed. The result of the equation takes the L2-norm because the magnitude of the difference is more stable than its direction in calculations.

DoN values tend to be small for points on flat regions and large for points on rugged surfaces. This property allows us to separate leaf regions from stem regions in a canopy point cloud by thresholding the DoN values.

After thresholding DoN outputs, some stem parts, particularly the ends near leaves, may persist in the point cloud. A Euclidean clustering with radius r3 is performed on the point cloud after DoN, and all isolated clusters containing fewer than 200 points are removed to further filter fragments of remaining stem parts.

There is a risk of some leaf points being falsely filtered as stem by the DoN threshold, resulting in false negatives. To mitigate false negatives, we conduct a neighborhood point search to fill back leaf points that were erroneously removed along with the stems. We traverse each point on the remaining point cloud, searching for points within a radius r4 on the removed plant portion using DoN. Once found, these points are filled back into the current point cloud, as they were incorrectly removed by the previous DoN. Figure 4 illustrates the complete stems removal process for Maranta arundinacea, yielding a canopy containing only leaves. After stem removal, the Maranta arundinacea canopy point cloud contains 637,353 points, and the recall rate of leaf points increases from 19.4% (after DoN) to 97.4% (final canopy). Although multiple parameters exist in this process, they are not difficult to tune, and we provide suggestions in Section III-B.

Leaf Segmentation

At this point, plant stems have been removed from the canopy point cloud, leaving a point cloud containing only leaves. Since the leaves are now detached from the connecting stem system, they can be more easily separated in 3D space. However, leaves that are connected and overlapping remain challenging to separate, especially in dense canopies.

The overlapping phenomenon among individual leaves appears complex due to the varied positions and angles at which leaves can overlap. However, the solution is fundamentally straightforward.

To separate overlapping leaves, we first identify the overlapping areas by thresholding the curvature of each point. Then, we remove ambiguous points with curvature values exceeding a specific threshold. The curvature of each point is calculated in three steps:

1. Search for the k-nearest neighbors of the point (k = 20).
2. Perform PCA on the neighborhood and calculate the three eigenvalues λ1 > λ2 > λ3.
3. Calculate the curvature as s = λ3 / (λ1 + λ2 + λ3). A smaller s value indicates a smoother neighborhood [57], making it less likely for the point to be located in an overlapping area.

Complete individual leaf segmentation is achieved using a region-growing algorithm that considers multiple features, including point distance, the normal difference in angle, and curvature difference between the seed point and the searching point [45]. Figure 5 showcases three common cases of leaf overlapping from the preprocessed point clouds, explaining the proposed segmentation method. The second row of Figure 5 magnifies the overlapping areas and uses dots and grids to display the real 3D orientations of the leaves. The third row shows the normalized curvature of each point rendered in different colors, highlighting the generally high curvature values of points in overlapping areas. After removing the high curvature points, all individual leaves are successfully segmented using the adopted region-growing technique, as shown in the fourth row of Figure 5.

The GPT (Generalized Poisson Surface Reconstruction) algorithm [59] is used to generate the leaf mesh. The leaf area is then equivalent to the sum of all meshed triangular areas. Figure 6(a) illustrates the GPT result for a single Maranta arundinacea leaf. To minimize estimation errors caused by limited imaging accuracy and noise [resulting in a very rugged triangle mesh as seen in the side view of Figure 6(b)], downsampling and smoothing are necessary before GPT (shown on the rightmost part of Figure 6). Despite its accuracy advantage over traditional methods, the estimated area using GPT is often larger than the ground truth, and the method is not applicable to plants with porous leaves (such as Monstera deliciosa).