Having anything in robots.txt reflected in law is such a bad idea, that I suggest we, the Internet standards community, do our best to make it not happen.  (I do understand the temptation to think we should make law, but others might disagree 🙂 )   So let’s not add any AI features into this standard.

https://mailarchive.ietf.org/arch/msg/ai-control/iy38WylitCEjq76ZogVLcriHeOQ/