As far as I know there is connection between bounded rationality and AI. Where AI do not know things they should know they hallucinate just like humans. I have read a piece somewhere i don't remember it now. And its like failing of the agent is already decided but we don't know the question to ask. Maybe jailbreaking is related to AI with less spaces to search for like a constrained AI which has no knowledge of every possible states will be easilly jailbroken. Bounded rationality can work here. It can be said that ai wont achieve the full rationality as the answer to every question is computational not possible(new variables arise as we keep moving in time and no amount of past information can predict the future states). Now the analogy here is, we can take AI as a bounded rational agent in spaces of intelligent agents. With each epoch the resources will increase and the rationality of the agent will keep moving on forward , but it will be suboptimal to the that space of time(the ai will get better, we will find more vulnerabilities, it will improve itself but it wont be able to achieve or be fully rational agent). Why it wont be fully rational? It can also be said that training a model with 1T parameters will always realise a same model . If answer is all math, the rationaly lies in math/software. Then AI is bounded by software/math. This is just my thought!! Maybe i am babbling nonsense , but i wanted to write this. I wanna know what you think?
for me the important point is that even a superhuman agent - like a future AI - will have limitations. As you say, information is uncertain, or limited, or changes. This means that even a superhuman agent will have properties of bounded rationality - they will make assumptions, take short cuts. So the study of cognitive biases and how rationality operates in the real world in human decision makers will remain relevant. I am not sure if that talks to your concern here, but that is my take!
Hey this one is nice read, can you write a piece on bounded rationality and AI? Thanks
i definitely take requests, but what about bounded rationality and AI are you thinking about:?
As far as I know there is connection between bounded rationality and AI. Where AI do not know things they should know they hallucinate just like humans. I have read a piece somewhere i don't remember it now. And its like failing of the agent is already decided but we don't know the question to ask. Maybe jailbreaking is related to AI with less spaces to search for like a constrained AI which has no knowledge of every possible states will be easilly jailbroken. Bounded rationality can work here. It can be said that ai wont achieve the full rationality as the answer to every question is computational not possible(new variables arise as we keep moving in time and no amount of past information can predict the future states). Now the analogy here is, we can take AI as a bounded rational agent in spaces of intelligent agents. With each epoch the resources will increase and the rationality of the agent will keep moving on forward , but it will be suboptimal to the that space of time(the ai will get better, we will find more vulnerabilities, it will improve itself but it wont be able to achieve or be fully rational agent). Why it wont be fully rational? It can also be said that training a model with 1T parameters will always realise a same model . If answer is all math, the rationaly lies in math/software. Then AI is bounded by software/math. This is just my thought!! Maybe i am babbling nonsense , but i wanted to write this. I wanna know what you think?
for me the important point is that even a superhuman agent - like a future AI - will have limitations. As you say, information is uncertain, or limited, or changes. This means that even a superhuman agent will have properties of bounded rationality - they will make assumptions, take short cuts. So the study of cognitive biases and how rationality operates in the real world in human decision makers will remain relevant. I am not sure if that talks to your concern here, but that is my take!