This just isn't true. Robotics barely worked before LLMs same goes with old symbolic means of computer vision recognition. You're feigning like this is old tech to downplay recent advances.
Not this, obviously. Also there are infinite token context windows coming down the pipes of frontier research. Look up continuous learning models like Sakana AI's Gödel Machine
Granted, there are limited workarounds to improve the performance of existing models; however, it's the model itself, not the method, that remembers the past and becomes unstable in higher-context scenarios. I am certain this issue will be solved, but it's nowhere near solved at the moment. There's an architecture change that needs to occur before that can be solved.
I'm letting you know that the architectural change that you're alluding to has arrived. Please read this: https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/ This won best paper at this year's NeurIPS conference. Its a legitimate step function change in the narrative. As methods like this get implemented, we're going to see an insanely fast past of improvement this year as systems become more capable of performing long term research on themselves.
hushpuppyextraordinaire
Got any more of that short term memory?
skinnermarinkykinkydink
I love how this post frames it as if it's some big revelation and now how all algorithms and robots have always worked before LLMs came on the scene.
TheGreatMani
This just isn't true. Robotics barely worked before LLMs same goes with old symbolic means of computer vision recognition. You're feigning like this is old tech to downplay recent advances.
jimhotep
The current gen of AI, from any company, goes insane when the context size is too large.
TheGreatMani
Not this, obviously. Also there are infinite token context windows coming down the pipes of frontier research. Look up continuous learning models like Sakana AI's Gödel Machine
jimhotep
Granted, there are limited workarounds to improve the performance of existing models; however, it's the model itself, not the method, that remembers the past and becomes unstable in higher-context scenarios. I am certain this issue will be solved, but it's nowhere near solved at the moment. There's an architecture change that needs to occur before that can be solved.
TheGreatMani
I'm letting you know that the architectural change that you're alluding to has arrived. Please read this: https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/ This won best paper at this year's NeurIPS conference. Its a legitimate step function change in the narrative. As methods like this get implemented, we're going to see an insanely fast past of improvement this year as systems become more capable of performing long term research on themselves.