Varun Chitre finds Claude Code produces real causes and dangerous unsupported guesses in device RCAs, announces DeviceRCA-Bench
Varun Chitre reports a months-long evaluation of Claude Code for root-cause analysis (RCA) on device failures and finds it produces real causes alongside unsupported guesses that can be dangerous if treated as conclusions. He’s building DeviceRCA-Bench to grade AI RCAs against bugs with known fixes and urges anyone acting on AI RCAs to ask what the model checked that could have proven its answer wrong.