What is speculative decoding?
Speculative decoding is a technique where a small, fast model guesses the next words and a large, accurate model checks the guesses in one go, so answers come out faster.
Terms seen alongside this one
- virtual machine 1× together
- reward hacking 1× together
- chain of thought 1× together
- reverse acqui-hire 1× together
Get it in your inbox every day
Daily headlines and summaries, a weekly synthesis every Sunday, and a monthly report at the start of each month. Sent at 8AM KST, which is the evening before in the US.
Free · no ads · one-click unsubscribe