A novel augmented reality for hidden organs visualisation in surgery: enhanced super-pixel with sub sampling and variance adaptive algorithm

摘要

In recent years, Augmented Reality (AR) has gained more attention as an effective tool in medical surgeries. The potentials of using AR in the medical field can change conventional medical procedures. However, the technology still facing fundamental challenges, especially hidden organs, for example, the organs behind the bowel and liver. The surgeries in these areas lack accuracy in the visualization of the soft tissues behind the bowel and liver like the uterus and gall bladder. This research aims to improve the accuracy of visualisation and the processing time of the augmented video. The proposed system consists of an enhanced super-pixel algorithm with variance weight adaptation and subsampling method. The simulation studies show significant improvements in visualization accuracy and a reduction in processing time. The results show reduced visualisation error by 0.23 mm. It provides better accuracy of the video in terms of visualization error from 1.58 similar to 1.83 mm to 1.35 similar to 1.60 mm, and the processing time decreases from 50 similar to 58 ms/frames to 40 similar to 48 ms/frames. The proposed system focused on the pixel refinement for the 3d reconstruction of the soft tissue, which helps solve the issue of visualising the bowel and liver in an augmented video.