Idioma:

Adaptive Batch Normalization for Training Data with Heterogeneous Features

Alsobhi, Wael ; Alafif, Tarik ; Abdel-Hakim, Alaa ; Zong, Weiwei

arXiv.org, 2022-11

Texto completo disponível

Citações Citado por

Título:
Adaptive Batch Normalization for Training Data with Heterogeneous Features
Autor: Alsobhi, Wael ; Alafif, Tarik ; Abdel-Hakim, Alaa ; Zong, Weiwei
Assuntos: Artificial neural networks ; Computer Science - Learning ; Training
É parte de: arXiv.org, 2022-11
Descrição: Batch Normalization (BN) is an important preprocessing step to many deep learning applications. Since it is a data-dependent process, for some homogeneous datasets it is a redundant or even a performance-degrading process. In this paper, we propose an early-stage feasibility assessment method for estimating the benefits of applying BN on the given data batches. The proposed method uses a novel threshold-based approach to classify the training data batches into two sets according to their need for normalization. The need for normalization is decided based on the feature heterogeneity of the considered batch. The proposed approach is a pre-training processing, which implies no training overhead. The evaluation results show that the proposed approach achieves better performance mostly in small batch sizes than the traditional BN using MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets. Additionally, the network stability is increased by reducing the occurrence of internal variable transformation.
Editor: Ithaca: Cornell University Library, arXiv.org
Idioma: Inglês

Links

Voltar para lista de resultados