I am incredibly optimistic on AI, especially for drug discovery. However, the belief that AI drugs will be more likely to pass phase 2 tests than others is either based on the assumption that AI will mostly be used to attack safe indications (me too drugs for targets that already have therapies operating on the same mechanism), or on some belief that perhaps in a distant future the transfer to human biology is better understood by the AI but not documented or shared with humans. Although I think the latter will come true, eventually, people right now tend to forget that because of the extreme cost of human clinical trials, the preclinical candidate has undergone every possible test that regulatory agencies, consultants, the internal company knowledge have suggested. So the failure on a new drug is only related to the “surprise” differences in humans vs all the tests in animals, cells, or in vitro setups that were previously done. Right now AI can help you get to the clinical candidate much faster and with fewer resources than before, but I believe that the translation to success rate improvement has some assumption that is probably not correct in the context of a fixed/novel target. I am open to the possibility that I miss something fundamental and would love feedback.
It's also the case that preclinical candidates developed this way are even harder to translate into human trials, because regulators and drug developers don't know how to build a safe and informative trial given the novelty of the methods that produced the candidate. So you essentially end up redoing a lot of the work that the AI allowed you to skip in order to do the work that remains. Perhaps a future innovation in "explainable" AI could help here, but that such a technology would itself need to be clinically validated in order to solve this problem.
Source: spent 9 years at a AI drug discovery startup. We advanced 8 clinical candidates and every one of them ran into some variant of this issue.