What is mixture-of-experts (MoE)?

Also known as 혼합 전문가(MoE) · Mixture-of-Experts · MoE(혼합전문가모델) · 전문가 혼합(MoE) · 전문가 혼합

Mixture-of-experts (MoE) is a model design that splits the model into specialized parts and switches on only the few needed for each request, so it takes less computation than its total size suggests.