Decision-Making under Bounded Rationality and Model Uncertainty: an Information-Theoretic Approach

Abstract

Artificial intelligence research and high computational power have recently led to break- throughs in solving high-dimensional reinforcement learning and sequential decision-making problems. The foundations of these advances rely on the classical theory of choice under uncer- tainty, the so-called Subjective Expected Utility (SEU) theory. However, SEU theory assumes two important unrealistic scenarios. First, it disregards computational limitations when mak- ing decisions by assuming perfectly rational agents i.e. agents with unlimited computational resources. Importantly, humans and artificial agents are bounded rational, or equivalently, they suffer from precision and computational limitations. Second, SEU theory assumes that the internal models employed for computation can be fully trusted and that they do not suffer from model uncertainty. However, any model of the environment is inherently incorrect and thus it should not be fully trusted. Therefore, humans and artificial agents are indeed subject to model uncertainty. This thesis consists of an experimental and a theoretical part. On the experimental side, I aimed to explain human sensorimotor behavior with information-theoretic models of bounded rationality and model uncertainty. In particular, we designed three experiments where we expose human subjects to decision-making scenarios involving model uncertainty. We dis- cover that human decision-making behavior can be explained by information-theoretic models that manifest as risk-sensitive and ambiguity-sensitive models. On the theoretical part, we developed a novel planning algorithm for sequential decision-making that accounts for both, information-processing constraints and model uncertainty. Finally, we examined and extended bounded rational models of decision-making under precision and time limitations whose we drew analogies with non-equilibrium thermodynamics. This non-equilibrium thermodynam- ical point of view allowed to connect decision-making with concepts such as dissipation and time-reversibility, and to discover novel relations connecting equilibrium with non-equilibrium decision-making. In conclusion, information-theoretic models of decision-making might be the missing cor- nerstone towards unifying principles of decision-making able to explain complex behavior beyond classic expected-utility models.