skip to main content
Idioma:

Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

Wang, Bokun ; Yuan, Zhuoning ; Ying, Yiming ; Yang, Tianbao

2023-01

Texto completo disponível

Citações Citado por
  • Título:
    Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
  • Autor: Wang, Bokun ; Yuan, Zhuoning ; Ying, Yiming ; Yang, Tianbao
  • Assuntos: Computer Science - Learning
  • Descrição: In recent years, model-agnostic meta-learning (MAML) has become a popular research area. However, the stochastic optimization of MAML is still underdeveloped. Existing MAML algorithms rely on the ``episode'' idea by sampling a few tasks and data points to update the meta-model at each iteration. Nonetheless, these algorithms either fail to guarantee convergence with a constant mini-batch size or require processing a large number of tasks at every iteration, which is unsuitable for continual learning or cross-device federated learning where only a small number of tasks are available per iteration or per round. To address these issues, this paper proposes memory-based stochastic algorithms for MAML that converge with vanishing error. The proposed algorithms require sampling a constant number of tasks and data samples per iteration, making them suitable for the continual learning scenario. Moreover, we introduce a communication-efficient memory-based MAML algorithm for personalized federated learning in cross-device (with client sampling) and cross-silo (without client sampling) settings. Our theoretical analysis improves the optimization theory for MAML, and our empirical results corroborate our theoretical findings. Interested readers can access our code at \url{https://github.com/bokun-wang/moml}.
  • Data de criação/publicação: 2023-01
  • Idioma: Inglês
Links
Voltar para lista de resultados

  • Realização: ABCD-USP USP   Logos de Redes Sociais: Freepik

Buscando em bases de dados remotas. Favor aguardar.