Visual inertial odometry enabled 3D ultrasound and photoacoustic imaging

There is an increasing need for 3D ultrasound and photoacoustic (USPA) imaging technology for real-time monitoring of dynamic changes in vasculature or molecular markers in various malignancies. Current 3D USPA systems utilize expensive 3D transducer arrays, mechanical arms or limited-range linear stages to reconstruct the 3D volume of the object being imaged. In this study, we developed, characterized, and demonstrated an economical, portable, and clinically translatable handheld device for 3D USPA imaging. An off-the-shelf, low-cost visual odometry system (the Intel RealSense T265 camera equipped with simultaneous localization and mapping technology) to track free hand movements during imaging was attached to the USPA transducer. Specifically, we integrated the T265 camera into a commercially available USPA imaging probe to acquire 3D images and compared it to the reconstructed 3D volume acquired using a linear stage (ground truth). We were able to reliably detect 500 µm step sizes with 90.46% accuracy. Various users evaluated the potential of handheld scanning, and the volume calculated from the motion-compensated image was not significantly different from the ground truth. Overall, our results, for the first time, established the use of an off-the-shelf and low-cost visual odometry system for freehand 3D USPA imaging that can be seamlessly integrated into several photoacoustic imaging systems for various clinical applications.