Intelligence is Free, Now What? Data Systems for, of, and by Agents
government of the people, by the people, for the people ... The cost of AI is dropping rapidly. GPT-4-class capabilities cost roughly $30 per million tokens in early 2023; today the same runs under $1, and some providers are pushing costs below $0.10.
Key Takeaways
- - Abraham Lincoln, Gettysburg Address (1863) The cost of AI is dropping rapidly.
Across benchmarks, inference prices have fallen between 9x and 900x per year , with a median decline near 50x.
- Parameswaran -an Associate Professor of EECS and co-director of the EPIC Data Lab at UC Berkeley-together with his collaborators.
It is part landscape survey and part perspective, and several of the research directions discussed below (including agentic speculation, structured memory, and synthesizing custom data systems from scratch) draw on the authors' own ongoing work.
- What do data systems that reliably and efficiently run and manage agent swarms look like?
Agents are rapidly becoming capable of synthesizing entire data systems in one go-meaning we can rebuild custom systems for each new workload.
- With multiple agents each exploring portions of the hypothesis space, each user request could amount to 1000s of individual SQL queries.
- An agent-first data system can exploit such properties to help agents make progress faster.
Stats & Key Facts
- #GPT-4-class capabilities cost roughly $30 per million tokens in early 2023; today the same runs under $1, and some providers are pushing costs below $0.10.
- #GPT-4-class capabilities cost roughly $30 per million tokens in early 2023; today the same runs under $1 , and some providers are pushing costs below $0.10 .
- #For instance, on a text-to-SQL benchmark with multiple agents attempting each task, only 10-20% of the sub-plans are distinct.
- #Thus, 80-90% of sub-queries perform duplicate work.
government of the people, by the people, for the people ... - Abraham Lincoln, Gettysburg Address (1863) The cost of AI is dropping rapidly. GPT-4-class capabilities cost roughly $30 per million tokens in early 2023; today the same runs under $1 , and some providers are pushing costs below $0.10 .
Across benchmarks, inference prices have fallen between 9x and 900x per year , with a median decline near 50x. Even frontier models are getting dramatically cheaper each generation, with open-source models following closely behind. And crucially, even if "Nobel-Prize-winning genius-level" intelligence isn't here yet, the intelligence that suffices for the vast majority of knowledge work is here today, and getting cheaper by the month.
At this rate, we are soon entering the era of virtually free intelligence -the kind that is more than enough for everyday knowledge work. Disclosure: This post is a perspective led by Aditya G. Parameswaran -an Associate Professor of EECS and co-director of the EPIC Data Lab at UC Berkeley-together with his collaborators.
For more details please read the original article at Berkeley BAIR.
Continue Learning
Comments
Comments appear only after moderation. Your email identifies your submission to the moderator and is never displayed here.
No approved comments yet.