I don't think it's a stall, two ways I would believe there is a stall:
- Does the epoch capability index progress show signs of plateauing? I consider this a good aggregate measure of diverse benchmarks into a single capability index. If we see things slowing down here thats a pretty direct and convincing piece of evidence for a stall.
- Do we see any signs that scaling laws are beginning to fail? That would be by far the most alarming to me, since I would interpret that to mean that the entire premise of this unprecedented capital allocation tsunami is broken.
Neither of these are true (for now). Progress is marching the same as it has for 4+ years now. It's still the same time to get a generation leap (I think like ~16-18 mo? Epoch has it) like GPT4->5. My theory is that people interpret plateauing because the releases are far more frequent now than they were in the past.
- Does the epoch capability index progress show signs of plateauing? I consider this a good aggregate measure of diverse benchmarks into a single capability index. If we see things slowing down here thats a pretty direct and convincing piece of evidence for a stall. - Do we see any signs that scaling laws are beginning to fail? That would be by far the most alarming to me, since I would interpret that to mean that the entire premise of this unprecedented capital allocation tsunami is broken.
Neither of these are true (for now). Progress is marching the same as it has for 4+ years now. It's still the same time to get a generation leap (I think like ~16-18 mo? Epoch has it) like GPT4->5. My theory is that people interpret plateauing because the releases are far more frequent now than they were in the past.