Artificial intelligence and machine learning

Research Article

Application of RF-DETR and Yolo26 Detectors for Brain Tumor Localization Using MRI Data

Vitaly Petrovich Fralenko

	Ailamazyan Program Systems Institute of RAS, Ves'kovo, Russia
	alarmod@pereslavl.ru

Abstract. The study focuses on the application of artificial intelligence methods for detecting non-rigid objects in medical images, using the example of brain tumor detection from magnetic resonance imaging data.

The first part presents an analytical review of modern approaches to non-rigid registration and tumor segmentation, as well as transformer-based and hybrid object detector architectures with deformable attention, multi-channel feature aggregation, and specialized loss functions designed for processing deformable structures with blurred boundaries, as well as an analysis of software implementations and licensing constraints of the corresponding frameworks.

The second part presents an experimental comparison of the RF-DETR and Yolo26 neural networks on a labeled brain tumor dataset for bounding box detection. Notably, this task represents a standard procedure for localizing the region of interest and does not require special consideration of structural deformability or boundary uncertainty, unlike segmentation or non-rigid registration tasks. Quality assessment was performed using mAP$0.50$, mAP$0.50$–$0.95$ , and F1-score metrics for IoU thresholds of $0.50$ and $0.50$–$0.95$ across confidence thresholds.

The obtained results showed that RF-DETR provides higher overall quality metrics and better speed compared to Yolo26, confirming that transformer-based detectors are a promising tool for automated analysis of brain MRI images. Based on analytical and experimental findings, recommendations were formulated for further development of RF-DETR in medical imaging tasks, aimed at improving the model's generalization ability, robustness to data distribution shifts, and clinical interpretability of results. (In Russian).

Keywords: brain tumor, neural network, transformer, analytical review, object detection, RF-DETR, Yolo26, medical imaging

MSC-2020 68-02; 68T45

Acknowledgments: The work was carried out within the framework of the State Budget Theme of the Ailamazyan Program Systems Institute of the Russian Academy of Sciences No. 125021302067-9 (completion deadline: 2025\–2027)

For citation: Vitaly P. Fralenko. Application of RF-DETR and Yolo26 Detectors for Brain Tumor Localization Using MRI Data. Program Systems: Theory and Applications, 2026, 17:2, pp. 263–293. (In Russ.). https://psta.psiras.ru/2026/2_263-293.

Full text of article (PDF): https://psta.psiras.ru/read/psta2026_2_263-293.pdf.

The article was submitted 30.04.2026; approved after reviewing 19.05.2026; accepted for publication 20.05.2026; published online 26.06.2026.