| dc.description.abstract |
This thesis is composed of two broad strands of research. The first part of the thesis will discuss causality, with focus on a novel, measure-theoretic axiomatisation thereof, and the second part of the thesis will tackle some problems in regression, with focus on kernel methods and infinite-dimensional output spaces. Even though the two topics are very distinct in nature, we tackle them through a shared principle that places emphasis on theory.
Causality is a topic that has recently garnered much interest among the artificial intelligence research community, but it has always been a centrepiece of human intelligence. Humans have always perceived that, in addition to observing how events unfold around them, they can also make interventions on the world that potentially change the course of events. In other words, interventions on the world (not necessarily by the observers themselves) can cause events to occur, change their chances of occurring, or prevent them from occurring. This notion of intervention is viewed by many as the essence behind the concept of causality, and this is the view that we take in this thesis.
Mathematical modelling aims to describe the world in an abstract way, using mathematical concepts and language. Therefore, to describe the world with causality in mind, such that interventions can be modelled, the development of an axiomatic mathematical framework that can encode such information is a necessity. In this thesis, we take the view that such an axiomatic framework has been developed and established for the concept of uncertainty (or randomness, or stochasticity), namely probability theory, but we argue that, despite many competing propositions, most notably the structural causal models and the potential outcomes frameworks, a universally agreed, axiomatic framework that plays the role of probability spaces in the study of uncertainty does not yet exist for the study of causality. It is clear that, since interventions on the world do not, in general, cause the world to behave in a deterministic way, but there is ensuing uncertainty following most interventions as to how events will subsequently unfold, probability theory will play a fundamental role in any theory of causality. Based on this standpoint, we propose an axiomatic framework of causality, called causal spaces, that is built directly on probability spaces.
The second part of the thesis will be concerned with several aspects of kernel regression. Regression is a concept that is ubiquitous in statistics and machine learning, and has an endless list of applications in a wide range of domains, and regression techniques based on kernels have been some of the most popular and influential in statistics and machine learning research. It is natural, then, that it has also received much attention from theoreticians regarding its properties, guarantees and limitations. In this thesis, we make modest contributions to several aspects of them. First, we discuss kernel conditional mean embeddings, which have been known to researchers for over a decade. Our contribution lies in the fact that we view them as Bochner conditional expectations, as opposed to operators between reproducing kernel Hilbert spaces (RKHSs) as had been prevalently done in the literature, and hence, their estimation is precisely a regression problem in which the output space is an RKHS. The hypothesis space in which this regression is carried out is itself a (vector-valued) RKHS, and such a technique is widely known as kernel ridge regression.
Next, we propose a particular form of kernel ridge regression called U-statistic regression, and apply this and the previously studied conditional mean embeddings to the study of conditional distributional treatment effect in the potential outcomes framework, which is widely used in the domains of medicine or social sciences. The thesis then takes a more theoretical turn to study learning-theoretic and empirical process-theoretic aspects of regression with infinite-dimensional output spaces, which can naturally occur if the outputs are themselves functions, and of which kernel conditional mean embeddings are a particular case. We extend the existing theory of empirical processes, an indispensable tool in statistical learning theory but that was previously only developed for classes of real-valued functions, to take into account classes of (possibly infinite-dimensional) vector-valued functions; in particular, we propose bounds on the metric entropy of classes of smooth vector-valued functions. We also take a look at the special case of vector-valued kernel ridge regression and prove a consistency result, based not on empirical process theory, but on the powerful integral operator techniques that are popular in the analysis of kernel ridge regression. |
en |
