What is Mixture-of-Experts?

Also known as MoE(혼합전문가모델)

An architecture that holds many smaller 'expert' sub-networks inside one model and activates only a few of them for each input. It lets a model have a large total parameter count while keeping the actual computation per request much smaller, so it can approach big-model performance at a lower cost.