이중수준 지식 증류를 통한 강건한 퓨샷 분류

초록

In this paper, we propose a few-shot learning method for robust image classification using knowledge distillation. Existing benchmark datasets for few-shot learning only consists of clean images. These datasets have the problem of not being able to reflect the real-world degradations such as noise and corruptions. In addition, there are still few studies on few-shot learning for ro bust image classification in corruption. Therefore, in this study, we first propose four novel data sets, Mini-ImageNet-C, CUB-200-C, CIFAR-FS-C and FGVC-Aircraft-C for evaluating the robust ness of few-shot learning algorithms. As the baseline few-shot learning method, we employ the most representative meta-learning approach, especially Model-Agnostic Meta-Learning (MAML). Afterwards, we incorporate knowledge distillation (KD) into MAML to distill corruption robustness from the large teacher model to the small student model where KD is performed in both inner-loop and outer-loop of MAML. Our ‘Bilevel KD’ allows the student models to achieve better performance while maintaining low memory usage. At meta-test stage, experiments showed that in all cases, our method performed significantly better than the baseline.

키워드