pointtree.evaluation

Evaluation tools.

class pointtree.evaluation.TimeTracker[source]

Bases: object

A tracker that stores the execution times of different code sections.

reset()[source]

Deletes all tracked execution times.

save(desc: str, value: float)[source]

Save the execution time of a certain code section.

Parameters:

  • desc – Description of the tracked code. If a value has already been saved for the description, the values are summed.

  • value – Execution time of the tracked code.

to_pandas() → DataFrame[source]
Returns:

Tracked execution times as pandas.DataFrame with the columns "Description" and "Runtime".

class pointtree.evaluation.Timer(desc: str, time_tracker: TimeTracker)[source]

Bases: object

A context manager that tracks the execution time of the contained code.

Parameters:

  • desc – Description of the tracked code.

  • time_tracker – Time tracker in which the measured execution time is to be stored.

pointtree.evaluation.instance_segmentation_metrics(
matched_target_ids: ndarray,
matched_predicted_ids: ndarray,
segmentation_metrics: DataFrame,
invalid_tree_id: int = -1,
) → Dict[str, float][source]

Calculates instance segmentation metrics.

Parameters:

  • matched_target_ids – ID of the matching target instance for each predicted instance.

  • matched_predicted_ids – ID of the matching predicted instance for each target instance.

  • segmentation_metrics – Segmentation metrics for each pair of matched target and predicted instance.

  • invalid_tree_id – ID that is assigned to instances that could not be matched. Defaults to -1.

Returns:

A dictionary containing the following instance segmentation keys: "tp", "fp", "fn", "panoptic_quality", "detection_f1_score", "detection_precision", "detection_recall", "segmentation_m_iou", "segmentation_m_precision", "segmentation_m_recall".

pointtree.evaluation.match_instances(
target: ndarray,
prediction: ndarray,
invalid_tree_id: int = -1,
iou_treshold: float = 0.5,
) → Tuple[ndarray, ndarray, DataFrame][source]

Matches tree instances from the ground truth and the prediction based on their intersection over union (IoU).

Parameters:

  • target – Ground truth instance ID for each point.

  • prediction – Predicted instance ID for each point.

  • invalid_tree_id – Instance ID that is assigned to points not belonging to any tree instance. Defaults to -1.

  • iou_treshold – Minimum IoU score that a ground truth and a predicted tree instance must exceed in order to be matched. The value must be greater than 0.5 in order to achieve a unique matching. Defaults to 0.5.

Returns:

Tuple of two numpy arrays and one pandas.DataFrame. The first array contains the ID of the matching target instance for each predicted instance. Predicted instances that are not matched to a target instance are assigned invalid_tree_id. The second array contains the ID of the matching predicted instance for each target instance. Target instances that are not matched to a predicted instance are assigned invalid_tree_id. The DataFrame contains one row per matched instance. It contains five columns "Target", "Prediction", "IoU", "Precision", "Recall" that contain the target ID and the predicted ID that match, and the IoU, precision, and recall between the two instances.

Shape:

  • target: \((N)\)

  • prediction: \((N)\)

  • Output: The first array in the output tuple has shape \((P)\) and the second shape \((T)\).

where

\(N = \text{ number of tree points}\)
\(P = \text{ number of predicted tree instances}\)
\(T = \text{ number of target tree instances}\)
pointtree.evaluation.semantic_segmentation_metrics(
target: ndarray,
prediction: ndarray,
class_map: Dict[str, int],
aggregate_classes: Dict[str, List[int]] | None = None,
) → Dict[str, float][source]

Calculates semantic segmentation metrics.

Parameters:

  • target – Ground truth semantic class IDs for each point.

  • prediction – Predicted semantic class IDs for each point.

  • class_map – A dictionary mapping class names to numeric class IDs.

  • aggregate_classes – A dictionary with which aggregations of classes can be defined. The keys are the names of the aggregated classes and the values are lists of the IDs of the classes to be aggregated.

Returns:

A dictionary containing the following keys for each semantic class: "<class_name>_iou", "<class_name>_precision", "<class_name>_recall". For each aggregated class, the keys "<class_name>_iou_aggregated", "<class_name>_precision_aggregated", "<class_name>_recall_aggregated" are provided.