GRAC: Graph-Based Retrieval-Augmented Captioning for Surgical Scene Description
[Widzenie komputerowe] Autorzy: Rochak Dhakal, Chen Zhao, Soup Cobb i 3 więcej
Purpose: Surgical operative reports support procedural documentation, surgical training, performance assessment, and workflow review. However, extracting meaningful information often requires extensive manual review and clinical expertise. Automated surgical scene descriptions could reduce this burden by leveraging recurring surgical activities and prior knowledge. Methods: We propose GRAC (Graph-Based Retrieval-Augmented Captioning), a framework that represents surgical scenes as graphs capturing relationships among instruments, actions, and anatomical structures. Given an input scene graph, GRAC learns to retrieve relevant graph-caption pairs from training data. It compares instrument-tissue interactions between retrieved examples and the input scene, using comparison features to condition retrieved-caption representations during generation. Results: On the EndoVis-18 benchmark, GRAC …
- Kategoria arxiv
- cs.CV
- Kategoria
- Widzenie komputerowe