Conditional mutual information API

condmutualinfo([measure::CMI], est::CMIEstimator, x, y, z) → cmi::Real

Estimate a conditional mutual information (CMI) of some kind (specified by measure), between x and y, given z, using the given dedicated ConditionalMutualInformationEstimator, which may be discrete, continuous or mixed.

Estimators

condmutualinfo([measure::CMI], est::ProbabilitiesEstimator, x, y, z) → cmi::Real ∈ [0, a)

Estimate the conditional mutual information (CMI) measure between x and y given z using a sum of entropy terms, without any bias correction, using the provided ProbabilitiesEstimator est. If measure is not given, then the default is CMIShannon().

With a ProbabilitiesEstimator, the returned cmi is guaranteed to be non-negative.

Estimators

condmutualinfo([measure::CMI], est::DifferentialEntropyEstimator, x, y, z) → cmi::Real

Estimate the mutual information between x and y conditioned on z, using the differential version of the given conditional mutual information (CMI) measure. The DifferentialEntropyEstimator est must must support multivariate data. No bias correction is performed. If measure is not given, then the default is CMIShannon().

Estimators

condmutualinfo([measure::CMI], est::MutualInformationEstimator, x, y, z) → cmi::Real

Estimate the mutual information between x and y conditioned on z, using the given conditional mutual information (CMI) measure, computed as a a difference of mutual information terms (just the chain rule of mutual information)

\[\hat{I}(X; Y | Z) = \hat{I}(X; Y, Z) - \hat{I}(X; Z).\]

The MutualInformationEstimator est may be continuous/differential, discrete or mixed. No bias correction in performed, except the bias correction that occurs for each individual mutual information term. If measure is not given, then the default is CMIShannon().

Estimators

ConditionalMutualInformation <: AssociationMeasure
CMI # alias

The supertype of all conditional mutual information measures. Concrete subtypes are

• CMIShannon
• CMIRenyiJizba
• CMIRenyiPoczos

ConditionalMutualInformationEstimator <: InformationEstimator
CMIEstimator # alias

The supertype of all conditional mutual information estimators.

Subtypes

• FPVP.
• PoczosSchneiderCMI.
• Rahimzamani.
• MesnerShalizi.

GaussianCMI <: MutualInformationEstimator
GaussianCMI(; normalize::Bool = false)

GaussianCMI is a parametric estimator for Shannon conditional mutual information (CMI) (Vejmelka and Paluš, 2008).

Description

GaussianCMI estimates Shannon CMI through a sum of two mutual information terms that each are estimated using GaussianMI (the normalize keyword is the same as for GaussianMI):

\[\hat{I}_{Gaussian}(X; Y | Z) = \hat{I}_{Gaussian}(X; Y, Z) - \hat{I}_{Gaussian}(X; Z)\]

FPVP <: ConditionalMutualInformationEstimator
FPVP(k = 1, w = 0)

The Frenzel-Pompe-Vejmelka-Paluš (or FPVP for short) estimator is used to estimate the differential conditional mutual information using a k-th nearest neighbor approach that is analogous to that of the KraskovStögbauerGrassberger1 mutual information estimator (Frenzel and Pompe (2007); Vejmelka and Paluš (2008)).

w is the Theiler window, which controls the number of temporal neighbors that are excluded during neighbor searches.

MesnerShalizi <: ConditionalMutualInformationEstimator
MesnerShalizi(k = 1, w = 0)

The MesnerShalizi estimator is an estimator for conditional mutual information for data that can be mixtures of discrete and continuous data MesnerShalizi2020.

PoczosSchneiderCMI <: ConditionalMutualInformationEstimator
PoczosSchneiderCMI(k = 1, w = 0)

The PoczosSchneiderCMI estimator computes various (differential) conditional mutual informations, using a k-th nearest neighbor approach (Póczos & Schneider, 2012)^{[Póczos2012]}.

Rahimzamani <: ConditionalMutualInformationEstimator
Rahimzamani(k = 1, w = 0)

The Rahimzamani estimator, short for Rahimzamani-Asnani-Viswanath-Kannan, is an estimator for Shannon conditional mutual information for data that can be mixtures of discrete and continuous data (Rahimzamani et al., 2018).

This is very similar to the GaoKannanOhViswanath mutual information estimator, but has been expanded to the conditional case.