MetaCleaner: Learning to Hallucinate Clean Representations for Noisy-Labeled Visual Recognition

Links: [paper]

Introduction

• A significant amount of label noise tends to deteriorate the classification performance of CNN’s.

• Existing label noise correction approaches often require a confusion matrix, to build the connections between clean and noise labels; this is often difficult and labor-intensive.

• Some of the recent works assign a confidence score for each image and embed it in the corresponding training loss, but the problem with these approaches are:
  • Since the confidence score is independently estimated, it may ignore the relationship between images in a category.
  • Simple weighting may reduce data diversity by decreasing importance of hard clean images.

• Along with improving robustness to label noise, meta cleaner helps generalize the model capacity with rich data diversity.

• MetaCleaner can effectively exploit the relations between different images in a random subset of category and leverage the importance of images to hallucinate diversified clean representations for noise reduction.

Approach

• The proposed method is inspired from the remarkable ability of human system to extract vision concepts from noisy images.

• MetaCleaner mainly consists of two sub-modules.
  1. Noisy weighting: For confidence score estimation.
  2. Clean Hallucinating: For generating a clean representation of the noisy subset.
• Noisy Weighting:
  • Aims to estimate a confidence score ($\alpha$) about whether a sample ($\bf{v}_{i}$) is correctly labeled or not.
  • Network takes as input a concatenated feature vector obtained from a subset of images of one category and outputs the confidence scores.
  • Confidence scores are used to construct novel clean samples.
• Clean Hallucinating:
  • After obtaining the confidence scores of the noisy images in the subset, a clean representation is hallucinated by summarizing the noisy images with their weights.

                  $\bf{v}_{c} = \frac{\sum_i \alpha_i \bf{v}_i}{\sum_i \alpha_i}$

  • $\bf{v}_{c}$ represents the clean representation for the noisy subset $\bf{v}_{i}$ and $\alpha_{i}$ represents the confidence score for a particular sample.
  • Leverages the importance of different images in the subset and summarizes a clean representation of the corresponding category.
• Training and Testing:
  • MetaCleaner can be easily integrated into any deep classification architecture with mini-batch SGD training.
  • During training, meta cleaner is used a new layer before classifier.
  • For each of the categories, a subset of samples are selected, and a clean representation is hallucinated using the two sub-modules of MetaCleaner.
  • MetaCleaner layer is removed during the testing phase.

Experiments and Results

• The proposed approach is evaluated on two popular benchmarks for noisy-labeled visual recognition, namely, Food-101N (Lee at al. 2018) and Clothing1M (Xiao et al. 2015)
• For the noisy weighting module, a simple two-layer network is used (FC-FC-ReLU-Sigmoid)
• Paper also comes up with some ablation studies to investigate the importance of sub-modules.

References

1. Kuang-Huei Lee, Xiaodong He, Lei Zhang, and Linjun Yang. Cleannet: Transfer learning for scalable image classifier training with label noise. In CVPR, 2018.

2. Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. In CVPR, 2015.