ISINet: An Instance-Based Approach for Surgical Instrument Segmentation

*Both authors contributed equally to this work
We study the task of semantic segmentation of surgical instruments in robotic-assisted surgery scenes. We propose the Instancebased Surgical Instrument Segmentation Network (ISINet), a method that addresses this task from an instance-based segmentation perspective. Our method includes a temporal consistency module that takes into account the previously overlooked and inherent temporal information of the problem.
We validate our approach on the existing benchmark for the task, the Endoscopic Vision 2017 Robotic Instrument Segmentation Dataset [2], and on the 2018 version of the dataset [1], whose annotations we extended for the fine-grained version of instrument segmentation. Our results show that ISINet significantly outperforms state-of-the-art methods, with our baseline version duplicating the Intersection over Union (IoU) of previous methods and our complete model triplicating the IoU.

Publications

ISINet: An Instance-Based Approach for Surgical Instrument Segmentation

C. González*, L. Bravo-Sánchez* and P. Arbeláez

23rd International Medical Image Computing and Computer Assisted Intervention (MICCAI), 2020

Results

Table 1. Comparison against the state-of-the-art on the EndoVis 2017 dataset.

METHOD

D

T


  • TernausNet [28]
  • MF-TAPNet [15]
  • ISINet (Ours)



  • ISINet (Ours)

  • CHALLENGE IOU


  • 35.27
  • 37.35
  • 53.55
  • 55.62

  • 66.27
  • 67.74
  • MEAN CLASS IOU


  • 10.17
  • 10.77
  • 26.92
  • 28.96

  • 36.48
  • 38.08
  • Table 2. Comparison against the state-of-the-art on the EndoVis 2018 dataset.

    METHOD

    D

    T


  • TernausNet [28]
  • MF-TAPNet [15]
  • ISINet (Ours)



  • ISINet (Ours)

  • CHALLENGE IOU


  • 46.22
  • 67.87
  • 72.99
  • 73.03

  • 77.19
  • 77.47
  • MEAN CLASS IOU


  • 14.19
  • 24.68
  • 40.16
  • 40.21

  • 44.58
  • 45.29
  • Qualitative results

    Figure 1. Each row depicts an example result for the task of instrument type segmentation on the EndoVis 2017 and 2018 datasets. The columns from left to right: image, annotation, segmentation of TernausNet [28], segmentation of MFTAPNet [15] and the segmentation of our method ISINet. The instrument colors represent instrument types.

    Citation

    @article{ISINet2020,
    title={ISINet: An Instance-Based Approach for Surgical Instrument Segmentation},
    author={Cristina González and Laura Bravo-Sánchez and Pablo Arbelaez},
    journal={arXiv preprint arXiv:2007.05533},
    year={2020}
    }

    References

  • [1] Allan, M., Shvets, A., Kurmann, T., Zhang, Z., Duggal, R., Su, Y.H., et al.:2017 robotic instrument segmentation challenge. arXiv preprint arXiv:1902.06426(2019)

  • [2] Allan, M., Shvets, A., Kurmann, T., Zhang, Z., Duggal, R., Su, Y.H., et al.: 2017 robotic instrument segmentation challenge. arXiv preprint arXiv:1902.06426 (2019)

  • [15] Jin, Y., Cheng, K., Dou, Q., Heng, P.A.: Incorporating temporal prior from motion flow for instrument segmentation in minimally invasive surgery video. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. pp. 440–448. Springer International Publishing, Cham (2019)

  • [28] Shvets, A.A., Rakhlin, A., Kalinin, A.A., Iglovikov, V.I.: Automatic instrument segmentation in robot-assisted surgery using deep learning
    • Address:
      Cra. 1 E No. 19A - 40. 111711, Bogotá, Colombia - Mario Laserna Building - School of Engineering - Universidad de Los Andes
    • Phone:
      [571] 332 4327, 332 4328, 332 4329