The Secretive World of AGI Development

I have been hanging out with AI engineers at the watercooler, and hear them candidly caution about open-source models and the risks of Artificial General Intelligence. But how bad can it be?