skip to main content
Idioma:

BNAS: Efficient Neural Architecture Search Using Broad Scalable Architecture

Ding, Zixiang ; Chen, Yaran ; Li, Nannan ; Zhao, Dongbin ; Sun, Zhiquan ; Chen, C. L. Philip

IEEE transaction on neural networks and learning systems, 2022-09, Vol.33 (9), p.5004-5018

United States: IEEE

Texto completo disponível

Citações Citado por
  • Título:
    BNAS: Efficient Neural Architecture Search Using Broad Scalable Architecture
  • Autor: Ding, Zixiang ; Chen, Yaran ; Li, Nannan ; Zhao, Dongbin ; Sun, Zhiquan ; Chen, C. L. Philip
  • Assuntos: Broad convolutional neural network (BCNN) ; Computational modeling ; Computer architecture ; Convolution ; Data models ; Graphics processing units ; image classification ; neural architecture search (NAS) ; reinforcement learning (RL) ; Topology ; Training
  • É parte de: IEEE transaction on neural networks and learning systems, 2022-09, Vol.33 (9), p.5004-5018
  • Notas: ObjectType-Article-1
    SourceType-Scholarly Journals-1
    ObjectType-Feature-2
    content type line 23
  • Descrição: Efficient neural architecture search (ENAS) achieves novel efficiency for learning architecture with high-performance via parameter sharing and reinforcement learning (RL). In the phase of architecture search, ENAS employs deep scalable architecture as search space whose training process consumes most of the search cost. Moreover, time-consuming model training is proportional to the depth of deep scalable architecture. Through experiments using ENAS on CIFAR-10, we find that layer reduction of scalable architecture is an effective way to accelerate the search process of ENAS but suffers from a prohibitive performance drop in the phase of architecture estimation. In this article, we propose a broad neural architecture search (BNAS) where we elaborately design broad scalable architecture dubbed broad convolutional neural network (BCNN) to solve the above issue. On the one hand, the proposed broad scalable architecture has fast training speed due to its shallow topology. Moreover, we also adopt RL and parameter sharing used in ENAS as the optimization strategy of BNAS. Hence, the proposed approach can achieve higher search efficiency. On the other hand, the broad scalable architecture extracts multi-scale features and enhancement representations, and feeds them into global average pooling (GAP) layer to yield more reasonable and comprehensive representations. Therefore, the performance of broad scalable architecture can be promised. In particular, we also develop two variants for BNAS that modify the topology of BCNN. In order to verify the effectiveness of BNAS, several experiments are performed and experimental results show that 1) BNAS delivers 0.19 days which is 2.37\times less expensive than ENAS who ranks the best in RL-based NAS approaches; 2) compared with small-size (0.5 million parameters) and medium-size (1.1 million parameters) models, the architecture learned by BNAS obtains state-of-the-art performance (3.58% and 3.24% test error) on CIFAR-10; and 3) the learned architecture achieves 25.3% top-1 error on ImageNet just using 3.9 million parameters.
  • Editor: United States: IEEE
  • Idioma: Inglês
Links
Voltar para lista de resultados

  • Realização: ABCD-USP USP   Logos de Redes Sociais: Freepik

Buscando em bases de dados remotas. Favor aguardar.