Evaluating Pulmonary Vessel Segmentation Beyond Dice: Connectivity, Caliber, and Clinical Validity

Matthew Taylor; Christopher Brown; Andrew Johnson

Abstract

Pulmonary vessel segmentation benchmarks commonly rank algorithms by regional overlap, but a vascular tree can be volumetrically similar to a reference while containing clinically consequential breaks, false joins, missing peripheral branches, or biased diameters. This evidence synthesis develops an evaluation framework aligned with the structural and translational purposes of pulmonary vessel analysis. It distinguishes six dimensions: regional agreement, boundary accuracy, centerline connectivity, branch detection, morphometric fidelity, and clinical task validity. MorVess provides a useful focal example because it reports Dice, centerline Dice, 95th-percentile Hausdorff distance, branch-oriented measures, cross-dataset tests, and geometric comparisons of vessel volume and caliber. Its design also demonstrates that evaluation and learning are coupled: distance and thickness targets improve precisely the properties that voxel-wise supervision tends to neglect. The article argues that no scalar metric can represent a pulmonary vascular reconstruction adequately. Evaluation should be stratified by vessel size and anatomical level, estimate uncertainty at the patient level, test robustness across acquisition domains, and connect segmentation errors to downstream measurements or expert decisions. A tiered protocol is proposed for algorithm development, challenge benchmarking, and clinical validation. The goal is not to replace Dice but to place it inside a multidimensional evidence structure that distinguishes larger masks from better vascular models.

Keywords

pulmonary vessel segmentationsegmentation evaluationcenterline connectivityvessel caliberclinical validitybranch detectionmedical imaging benchmarks

References

Mao, F., Chen, Y., Wu, B., Lin, L., Dai, J., Li, Z., ... & Qin, F. (2026). MorVess: Morphology-Aware Pulmonary Vessel Segmentation Network. arXiv preprint arXiv:2606.24214.
Chu, Y., Luo, G., Zhou, L., Cao, S., Ma, G., Meng, X., ... & Gao, X. (2025). Deep learning-driven pulmonary artery and vein segmentation reveals demography-associated vasculature anatomical differences. Nature Communications, 16, 2262.
Dice, L. R. (1945). Measures of the amount of ecologic association between species. Ecology, 26(3), 297-302.
Eppink, J., Kervadec, H., van Capelleveen, J., Verhoeff, J., Senan, S., & Bohoudi, O. (2026). PaSAL: A deep learning pipeline for pulmonary artery-vein segmentation and anatomical labeling in thoracic CT. In Proceedings of the 9th International Conference on Medical Imaging with Deep Learning (pp. 3561-3592). PMLR.
Huttenlocher, D. P., Klanderman, G. A., & Rucklidge, W. J. (1993). Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9), 850-863.
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2), 203-211.
Luo, G., Wang, K., Liu, J., Li, S., Liang, X., Li, X., ... & Gao, X. (2023). Efficient automatic segmentation for multi-level pulmonary arteries: The PARSE challenge. arXiv preprint arXiv:2304.03708.
Moccia, S., De Momi, E., El Hadji, S., & Mattos, L. S. (2018). Blood vessel segmentation algorithms - review of methods, datasets and evaluation metrics. Computer Methods and Programs in Biomedicine, 158, 71-91.
Nan, Y., Xing, X., Wang, S., Tang, Z., Felder, F. N., Zhang, S., ... & Gao, X. (2024). Hunting imaging biomarkers in pulmonary fibrosis: Benchmarks of the AIIB23 challenge. Medical Image Analysis, 97, 103253.
Qi, Y., He, Y., Qi, X., Zhang, Y., & Yang, G. (2023). Dynamic Snake Convolution Based on Topological Geometric Constraints for Tubular Structure Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 6070-6079).
Shit, S., Paetzold, J. C., Sekuboyina, A., Ezhov, I., Unger, A., Zhylka, A., ... & Menze, B. H. (2021). clDice - A novel topology-preserving loss function for tubular structure segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 16560-16569).
Taha, A. A., & Hanbury, A. (2015). Metrics for evaluating 3D medical image segmentation: Analysis, selection, and tool. BMC Medical Imaging, 15, 29.
Wang, Y., Wei, X., Liu, F., Chen, J., Zhou, Y., Shen, W., Fishman, E. K., & Yuille, A. L. (2020). Deep Distance Transform for tubular structure segmentation in CT scans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3833-3842).

Publication details

Journal: AI Frontiers in Science and Society
Volume: 1 (2026)
Article number: osm20260006
License: CC BY 4.0

[1] Mao, F., Chen, Y., Wu, B., Lin, L., Dai, J., Li, Z., ... & Qin, F. (2026). MorVess: Morphology-Aware Pulmonary Vessel Segmentation Network. arXiv preprint arXiv:2606.24214.

[2] Chu, Y., Luo, G., Zhou, L., Cao, S., Ma, G., Meng, X., ... & Gao, X. (2025). Deep learning-driven pulmonary artery and vein segmentation reveals demography-associated vasculature anatomical differences. Nature Communications, 16, 2262.

[3] Dice, L. R. (1945). Measures of the amount of ecologic association between species. Ecology, 26(3), 297-302.

[4] Eppink, J., Kervadec, H., van Capelleveen, J., Verhoeff, J., Senan, S., & Bohoudi, O. (2026). PaSAL: A deep learning pipeline for pulmonary artery-vein segmentation and anatomical labeling in thoracic CT. In Proceedings of the 9th International Conference on Medical Imaging with Deep Learning (pp. 3561-3592). PMLR.

[5] Huttenlocher, D. P., Klanderman, G. A., & Rucklidge, W. J. (1993). Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9), 850-863.

[6] Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2), 203-211.

[7] Luo, G., Wang, K., Liu, J., Li, S., Liang, X., Li, X., ... & Gao, X. (2023). Efficient automatic segmentation for multi-level pulmonary arteries: The PARSE challenge. arXiv preprint arXiv:2304.03708.

[8] Moccia, S., De Momi, E., El Hadji, S., & Mattos, L. S. (2018). Blood vessel segmentation algorithms - review of methods, datasets and evaluation metrics. Computer Methods and Programs in Biomedicine, 158, 71-91.

[9] Nan, Y., Xing, X., Wang, S., Tang, Z., Felder, F. N., Zhang, S., ... & Gao, X. (2024). Hunting imaging biomarkers in pulmonary fibrosis: Benchmarks of the AIIB23 challenge. Medical Image Analysis, 97, 103253.

[10] Qi, Y., He, Y., Qi, X., Zhang, Y., & Yang, G. (2023). Dynamic Snake Convolution Based on Topological Geometric Constraints for Tubular Structure Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 6070-6079).

[11] Shit, S., Paetzold, J. C., Sekuboyina, A., Ezhov, I., Unger, A., Zhylka, A., ... & Menze, B. H. (2021). clDice - A novel topology-preserving loss function for tubular structure segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 16560-16569).

[12] Taha, A. A., & Hanbury, A. (2015). Metrics for evaluating 3D medical image segmentation: Analysis, selection, and tool. BMC Medical Imaging, 15, 29.

[13] Wang, Y., Wei, X., Liu, F., Chen, J., Zhou, Y., Shen, W., Fishman, E. K., & Yuille, A. L. (2020). Deep Distance Transform for tubular structure segmentation in CT scans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3833-3842).