AI and compute
OpenAI released an analysis examining the computational power used in major artificial intelligence training runs since 2012. The study revealed that compute usage expanded exponentially with a 3.4-month doubling time, outstripping standard hardware trends like Moore's Law. Because computational gains drive progress in the field, sustained growth implies future systems will far exceed contemporary capabilities.
Key Takeaways
- In May 2018, OpenAI published an analysis tracking the amount of compute used in the largest artificial intelligence training runs since 2012.
The results demonstrated an exponential increase characterized by a 3.4-month doubling time.
- By comparison, Moore's Law followed a 2-year doubling period, which would have yielded only a 7x increase over that timeframe rather than the observed growth of more than 300,000x.
Improvements in compute represent a key component of artificial intelligence progress.
- Because computational scale has expanded at this dramatic pace, monitoring hardware deployment offers insight into how quickly model capabilities evolve.
If this scaling trajectory continues, practitioners should anticipate and prepare for the broader implications of AI systems that operate far outside current limits.
- OpenAI released an analysis tracking compute usage in major artificial intelligence training runs starting from 2012.
The computational power applied to top AI training runs grew exponentially with a 3.4-month doubling time.
- Rapidly scaling compute suggests future AI models will operate far outside today's performance boundaries.
Stats & Key Facts
- #By comparison, Moore's Law followed a 2-year doubling period, which would have yielded only a 7x increase over that timeframe rather than the observed growth of more than 300,000x.
- #Total compute expanded by more than 300,000x, compared to the 7x increase expected under Moore's Law.
In May 2018, OpenAI published an analysis tracking the amount of compute used in the largest artificial intelligence training runs since 2012. The results demonstrated an exponential increase characterized by a 3.4-month doubling time. By comparison, Moore's Law followed a 2-year doubling period, which would have yielded only a 7x increase over that timeframe rather than the observed growth of more than 300,000x.
Improvements in compute represent a key component of artificial intelligence progress. Because computational scale has expanded at this dramatic pace, monitoring hardware deployment offers insight into how quickly model capabilities evolve. If this scaling trajectory continues, practitioners should anticipate and prepare for the broader implications of AI systems that operate far outside current limits.
OpenAI released an analysis tracking compute usage in major artificial intelligence training runs starting from 2012. The computational power applied to top AI training runs grew exponentially with a 3.4-month doubling time. Total compute expanded by more than 300,000x, compared to the 7x increase expected under Moore's Law.
For more details please read the original article at OpenAI.
Continue Learning
Comments
Comments appear only after moderation. Your email identifies your submission to the moderator and is never displayed here.
No approved comments yet.