Colonoscopy is an endoscopic technique that allows a physician to inspect the inside of the human colon and to perform if deemed necessaryat the same time a number of diagnostic and therapeutic operations. In order to see the inside of the colon, a video signal of the internal mucosa of the colon is generated by a tiny video camera at the tip of the endoscope and displayed on a monitor for real-time analysis by the physician. We have captured and stored these videos in digital format and call these colonoscopy videos. Based on new algorithms for instrument detection and shot segmentation, we introduce new spatio-temporal analysis techniques to automatically identify an operation shota segment of visual data in a colonoscopy video that corresponds to a diagnostic or therapeutic operation. Our experiments on real colonoscopy videos demonstrate the effectiveness of the proposed approach. The proposed techniques and software are useful for 1) postprocedure review for causes of complications due to diagnostic or therapeutic operations; 2) establishment of an effective content-based retrieval system to facilitate endoscopic research and education; 3) development of a systematic approach to assess and improve the procedural skills of endoscopists.
Bibliographical noteFunding Information:
Manuscript received March 21, 2006; revised October 29, 2006. This work was supported in part by the National Science Foundation (NSF) under Grant IIS-0513777, Grant IIS-0513809, and Grant IIS-0513582 and in part by the Mayo Clinic. Asterisk indicates corresponding author. Y. Cao, D. Liu, and J. Wong are with the Department of Computer Science, Iowa State University, Ames, IA 50011-1040 USA. *W. Tavanapong is with the Department of Computer Science, Iowa State University, Ames, IA 50011-1040 USA (e-mail: firstname.lastname@example.org). J. Oh is with the Department of Computer Science and Engineering, University of North Texas, Denton, TX 76203 USA. P. C. de Groen is with the Mayo Clinic College of Medicine, Mayo Clinic, Rochester, MN 55905 USA. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TBME.2007.890734
- Instrument detection
- Medical video analysis
- Video segmentation