A Constructive Density-Ratio Approach to Mutual Information Estimation: experiments in feature selection

Authors

  • Igor Braga University of São Paulo

DOI:

https://doi.org/10.5753/jidm.2014.1527

Keywords:

classification, density-ratio estimation, feature selection, mutual information estimation

Abstract

Mutual Information (MI) estimation is an important component of several data mining tasks (e.g. feature selection). In classification settings,  MI estimation essentially depends on the estimation of the ratio of two probability densities. Using a recently developed method of density-ratio estimation, which is constructive in nature, new estimators for MI can be derived. In this article, we consider one such new estimator - VMI - and compare it experimentally to previously proposed MI estimators. The first batch of experiments is conducted solely on mutual information estimation, and shows that VMI compares favorably to previous estimators. The second batch of experiments applies MI estimation to feature selection in classification tasks, evidencing that VMI leads to better feature selection performance. Combining the results of both experimental batches, we conclude that the development of improved density-ratio estimators can positively impact MI estimation and feature selection.

Downloads

Download data is not yet available.

Author Biography

Igor Braga, University of São Paulo

PhD candidate at the Institute of Mathematics and Computer Science of the University of São Paulo.

Downloads

Published

2014-07-18

How to Cite

Braga, I. (2014). A Constructive Density-Ratio Approach to Mutual Information Estimation: experiments in feature selection. Journal of Information and Data Management, 5(1), 134. https://doi.org/10.5753/jidm.2014.1527

Issue

Section

KDMiLe 2013