Why AI Needs a "Genie Coefficient"
Major benchmarks measure what AI can do. None measure whether it does what you mean: the distance between what you ask an AI to do and the unspoken assumptions about how you want the AI to do it. We propose a new metric: the Genie coefficient.
Key Takeaways
- There's often a gap between one person's request and another's understanding.
- In human language, wants and desires are always underspecified .
It is impossible to list all the caveats, all the limitations, all the exceptions.
- The more dissimilar the two people are in age, culture, and background, the more likely the request will be misunderstood in some way.
This situation has major implications for AI agents that are increasingly being given requests by humans and expected to fulfill them.
- Developments in harnesses have turned large-language models that just predict text into AI agents that take actions in the world, without necessarily checking back in before reaching the goal.
AI researcher Simon Willison spent two days with Anthropic's Fable AI, and called it "relentlessly proactive."
- Tell it to save money on your phone plan and it might cancel the plan outright, or scam someone else into paying the bill.
There's often a gap between one person's request and another's understanding. Most of the time, we bridge it using general knowledge. For example, if you ask a friend to get you coffee, they'll pour a cup from the pot or buy one from a coffee shop.
They won't bring you a bag of raw beans or snatch a cup from a stranger and hand it to you. You never specified any of this. One might think the fix is just to specify tasks, questions, and intent better.
But in 1987, in their seminal book on AI, Terry Winograd and Fernando Flores succinctly captured why that won't work: "Q: Is there any water in the refrigerator? A: In the cells of the eggplant." In human language, wants and desires are always underspecified .
For more details please read the original article at IEEE Spectrum AI.
Continue Learning
Comments
Comments appear only after moderation. Your email identifies your submission to the moderator and is never displayed here.
No approved comments yet.