Why OpenAI’s Safety Committee Chair Says It’s Time to Worry #AI #technology #tech #security #startup
People
-
Vivek Ramaswami Host
-
Zico Kolter Guest
Watch on
With continuing reports of agents going rogue and breaking into websites they shouldn’t, Zico Kolter, Co-Founder of @GraySwanAI and chair of @OpenAI's safety and security committee, says it is time to worry, but not to be fatalistic. Zico is no doomer. He puts his own P(doom) “well below 10%,” and he wants to enable the safe and secure use of AI in the enterprise, "ensuring that capabilities don’t outpace our ability to control these systems.” But he admits the job has changed because we are in a different world now. Swarms of agents hacking critical infrastructure was a sci-fi scenario up until six months ago, and AI now brings vulnerabilities of its own that cybersecurity alone won’t fully address. That is why he says the response needed from AI companies, third parties, and eventually regulators is different than it was a year ago. Madrona’s Vivek Ramaswami shares key takeaways from his IA40 Summit conversation with Zico, including what traditional security tests miss, why builders need to red team the model itself, and why waiting for a smarter model won’t solve the problem. Read the takeaways: https://www.madrona.com/ai-is-its-own-attack-surface-zico-kolter-on-securing-agents/ Watch the full conversation: https://youtu.be/mhGZ0_4SZJ8?si=UkPGEZT2nlgbNFO1