Exploring Attention in Video Instance Segmentation

The aim of this repository is to document the code and our work on our CS 541 DL term project at Worcester Polytechnic Institute (WPI), MA.

 

This work focuses on exploring attention mechanisms to improve performance of the baseline architecture, ObjProp, for Video Instance Segmentation. Two new modifications are introduced, an attention neck module for region proposal and weighing the inter-frame affinity for mask propagation. Moreover, the techniques of
sampling reference frames for mask propagation are experimented on. Multiple trials were
conducted with these variables and the resulting metrics were investigated. Although the approaches did not improve the performance of the state-of-the-art, they provide future directions which can lead to better attention-based architecture with refined performance.

​

The code and report are available at Github in the link below