제목: Towards brain-plausible general intelligence model: Perceiver 

발표자: 정진영 (경북대 인공지능학과 NeuroAI Lab)

날짜: 2023년 6월 29일 (목) 오후 4시  

초록: In this seminar, we will talk about attention mechanisms amidst the recent hype surrounding the field of artificial intelligence. Our focus will be on the Perceiver, a variant of the Transformer, developed by DeepMind. The Perceiver has exhibited remarkable general-purpose capabilities, being task- and modality-agnostic, while minimizing human architectural assumptions (inductive bias). Furthermore, we will introduce Sparceiver, our modification of the Perceiver that incorporates sparse latent activation, driven by the pursuit of brain plausibility. This modification has demonstrated faster convergence and higher performance in CIFAR-10 classification without pre-training or data augmentation schemes (Sparceiver = 74.3% vs. Perceiver = 64.6%). Additionally, we will discuss various strategies to further modify and enhance our proposed model to effectively address the challenges encountered when applying AI in real-world scenarios, such as Continual Learning or Out-of-Distribution tasks.