Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence
Belavkin, Roman V. 
ORCID: https://orcid.org/0000-0002-2356-1447
(2018)
Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence.
Information Geometry and Its Applications.
In: IGAIA IV 2016: Information Geometry and Its Applications, 12-17 June 2016, Liblice, Czech Republic.
ISBN 9783319977973.
ISSN 2194-1009
[Conference or Workshop Item]
(doi:10.1007/978-3-319-97798-0_15)
|
PDF
- Final accepted version (with author's formatting)
Download (208kB) | Preview |
Abstract
We discuss a relation between the Kantorovich-Wasserstein (KW) metric and the Kullback-Leibler (KL) divergence. The former is defined using the optimal transport problem (OTP) in the Kantorovich formulation. The latter is used to define entropy and mutual information, which appear in variational problems to find optimal channel (OCP) from the rate distortion and the value of information theories. We show that OTP is equivalent to OCP with one additional constraint fixing the output measure, and therefore OCP with constraints on the KL-divergence gives a lower bound on the KW-metric. The dual formulation of OTP allows us to explore the relation between the KL-divergence and the KW-metric using decomposition of the former based on the law of cosines. This way we show the link between two divergences using the variational and geometric principles.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Additional Information: | Paper published as:
Belavkin R.V. (2018) Relation Between the Kantorovich–Wasserstein Metric and the Kullback–Leibler Divergence. In: Ay N., Gibilisco P., Matúš F. (eds) Information Geometry and Its Applications. IGAIA IV 2016. Springer Proceedings in Mathematics & Statistics, vol 252. Springer, Cham |
| Research Areas: | A. > School of Science and Technology > Computer Science > Artificial Intelligence group |
| Item ID: | 25970 |
| Notes on copyright: | The final authenticated version is available online at https://doi.org/10.1007/978-3-319-97798-0_15 |
| Useful Links: | |
| Depositing User: | Roman Belavkin |
| Date Deposited: | 11 Jan 2019 16:32 |
| Last Modified: | 27 Jun 2022 02:43 |
| URI: | https://eprints.mdx.ac.uk/id/eprint/25970 |
Actions (login required)
 |
View Item |
Statistics
Additional statistics are available via IRStats2.
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)