skip to main content
Idioma:
Primo Search
Clear Search Box
Search in: Busca Geral
Busca Avançada
Busca por Índices

DRRNets: Dynamic Recurrent Routing via Low-Rank Regularization in Recurrent Neural Networks

Shan, Dongjing ; Luo, Yong ; Zhang, Xiongwei ; Zhang, Chao

IEEE transaction on neural networks and learning systems, 2023-04, Vol.34 (4), p.2057-2067

United States: IEEE

Texto completo disponível

Citações Citado por
  • Título:
    DRRNets: Dynamic Recurrent Routing via Low-Rank Regularization in Recurrent Neural Networks
  • Autor: Shan, Dongjing ; Luo, Yong ; Zhang, Xiongwei ; Zhang, Chao
  • Assuntos: Computational modeling ; Logic gates ; Long-term memory ; low rank ; Memory management ; recurrent neural network (RNN) ; Recurrent neural networks ; Routing ; sparsity projection ; Task analysis ; temporal dependency ; Training ; vanishing gradients
  • É parte de: IEEE transaction on neural networks and learning systems, 2023-04, Vol.34 (4), p.2057-2067
  • Notas: ObjectType-Article-1
    SourceType-Scholarly Journals-1
    ObjectType-Feature-2
    content type line 23
  • Descrição: Recurrent neural networks (RNNs) continue to show outstanding performance in sequence learning tasks such as language modeling, but it remains difficult to train RNNs for long sequences. The main challenges lie in the complex dependencies, gradient vanishing or exploding, and low resource requirement in model deployment. In order to address these challenges, we propose dynamic recurrent routing neural networks (DRRNets), which can: 1) shorten the recurrent lengths by allocating recurrent routes dynamically for different dependencies and 2) reduce the number of parameters significantly by imposing low-rank constraints on the fully connected layers. A novel optimization algorithm via low-rank constraint and sparsity projection is developed to train the network. We verify the effectiveness of the proposed method by comparing it with multiple competitive approaches in several popular sequential learning tasks, such as language modeling and speaker recognition. The results in terms of different criteria demonstrate the superiority of our proposed method.
  • Editor: United States: IEEE
  • Idioma: Inglês
Links
Voltar para lista de resultados

  • Realização: ABCD-USP USP   Logos de Redes Sociais: Freepik

Buscando em bases de dados remotas. Favor aguardar.