Jeff Dean, Google · Y Combinator
Build where the model scores zero, not twenty
Jeff Dean built MapReduce, BigTable, TensorFlow, the TPU and Gemini, so his rule for picking an AI problem is worth taking literally. Run today's general models against your domain and read the score. If they are completely failing, he says, that is probably a good sign. If they are kind of able to do some of it but not very well, that is not a great sign, because the capability is already present and more data or a bigger model finishes it.
His instruction is exact. Look for something where the model succeeds 0% or 1% of the time, not 20%.
Steal it
Score today's frontier models on your core task. If they land anywhere near 20 percent, pick a harder problem.