Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgery

Ming Hu^{1,2,3 *} · Zhengdi Yu *⁴ · Feilong Tang^1,2,3 · Kaiwen Chen⁵ · Yulong Li³ · Imran Razzak³ · Junjun He² · Tolga Birdal⁴ · Kaijing Zhou †⁵ · Zongyuan Ge †¹

¹Monash University · ²Shanghai AI Laboratory · ³MBZUAI · ⁴Imperial College London · ⁵Eye Hospital, Wenzhou Medical Univeristy

We introduce OphNet-3D, the first large-scale RGB-D dataset for dynamic 3D hand-instrument reconstruction in ophthalmic microsurgery, supported by an efficient multi-stage annotation pipeline, and propose novel architectures (H-Net and OH-Net) that significantly outperform existing methods in accurate hand and instrument reconstruction tasks.

News 🚩

• [2025-6-26] 🎉🎉🎉 OphNet-3D is accepted by NeurIPS 2025 as a Spotlight Paper!.
• [2025/5/26] Paper is now available. ⭐

TO DO

• Release dataset
• Release baseline experimental results and checkpoints

Expected in mid-November

Citation

@article{hu2025towards,
  title={Towards dynamic 3d reconstruction of hand-instrument interaction in ophthalmic surgery},
  author={Hu, Ming and Yu, Zhengdi and Tang, Feilong and Chen, Kaiwen and Li, Yulong and Razzak, Imran and He, Junjun and Birdal, Tolga and Zhou, Kaijing and Ge, Zongyuan},
  journal={arXiv preprint arXiv:2505.17677},
  year={2025}
}

Contact

For any questions, please contact ming.hu@monash.edu or z.yu23@imperial.ac.uk .