We have hit that wall almost 2 years ago with gpt-4. There was clearly no scaling as gpt-4 was already decently smart and if you got x2 smarter you’ll be more capable than anything on the market today. All models doing today (R1 and friends; and Claude) are trying to optimize this local maxima toward generating more useful responses (ie: code when it comes to Claude).
AI, at its current form, is a Deep Seek of compressed knowledge in a 30-50gb of interconnected data. I think we’ll look at this as trying to train networks on corpus of data and expecting them to have a hold of reality. Our brains are trained on “reality” which is not the “real” reality as your vision is limited to the visible spectrum. But if you want a network to behave like a human then maybe give him what a human see.
There is also the possibility that there is a physical limit to intelligence. I don’t see any elephants doing PhDs and the smartest of humans are just a small configuration away from insanity.
AI, at its current form, is a Deep Seek of compressed knowledge in a 30-50gb of interconnected data. I think we’ll look at this as trying to train networks on corpus of data and expecting them to have a hold of reality. Our brains are trained on “reality” which is not the “real” reality as your vision is limited to the visible spectrum. But if you want a network to behave like a human then maybe give him what a human see.
There is also the possibility that there is a physical limit to intelligence. I don’t see any elephants doing PhDs and the smartest of humans are just a small configuration away from insanity.