Transfer entropy API

transferentropy([measure::TEShannon], est, s, t, [c])
transferentropy(measure::TERenyiJizba, est, s, t, [c])

Estimate the transfer entropy $TE^*(S \to T)$ or $TE^*(S \to T | C)$ if c is given, using the provided estimator est, where $*$ indicates the given measure. If measure is not given, then TEShannon(; base = 2) is the default.

Arguments

• measure: The transfer entropy measure, e.g. TEShannon or TERenyi, which dictates which formula is computed. Embedding parameters are stored in measure.embedding, and is represented by an EmbeddingTE instance. If calling transferentropy without giving measure, then the embedding is optimized by finding suitable delay embedding parameters using the "traditional" approach from DynamicalSystems.jl.
• s: The source timeseries.
• t: The target timeseries.
• c: Optional. A conditional timeseries.

Description

The Shannon transfer entropy is defined as $TE^S(S \to T | C) := I^S(T^+; S^- | T^-, C^-)$, where $I^S(T^+; S^- | T^-, C^-)$ is CMIShannon, and marginals for the CMI are constructed as described in EmbeddingTE. The definition is analogous for TERenyiJizba.

If s, t, and c are univariate timeseries, then the the marginal embedding variables $T^+$ (target future), $T^-$ (target present/past), $S^-$ (source present/past) and $C^-$ (present/past of conditioning variables) are constructed by first jointly embedding s, t and c with relevant delay embedding parameters, then subsetting relevant columns of the embedding.

Since estimates of $TE^*(S \to T)$ and $TE^*(S \to T | C)$ are just a special cases of conditional mutual information where input data are marginals of a particular form of delay embedding, any combination of variables, e.g. $S = (A, B)$, $T = (C, D)$, $C = (D, E, F)$ are valid inputs (given as StateSpaceSets). In practice, however, s, t and c are most often timeseries, and if s, t and c are StateSpaceSets, it is assumed that the data are pre-embedded and the embedding step is skipped.

Compatible estimators

transferentropy is just a simple wrapper around condmutualinfo that constructs an appropriate delay embedding from the input data before CMI is estimated. Consequently, any estimator that can be used for ConditionalMutualInformation is, in principle, also a valid transfer entropy estimator. TransferEntropyEstimators are the exception - they compute transfer entropy directly.

TransferEntropy <: AssociationMeasure

The supertype of all transfer entropy measures. Concrete subtypes are

• TEShannon
• TERenyiJizba

The supertype of all dedicated transfer entropy estimators.

Zhu1 <: TransferEntropyEstimator
Zhu1(k = 1, w = 0, base = MathConstants.e)

The Zhu1 transfer entropy estimator (Zhu et al., 2015).

Assumes that the input data have been normalized as described in (Zhu et al., 2015).

This estimator approximates probabilities within hyperrectangles surrounding each point xᵢ ∈ x using using k nearest neighbor searches. However, it also considers the number of neighbors falling on the borders of these hyperrectangles. This estimator is an extension to the entropy estimator in Singh et al. (2003).

w is the Theiler window, which determines if temporal neighbors are excluded during neighbor searches (defaults to 0, meaning that only the point itself is excluded when searching for neighbours).

Lindner <: TransferEntropyEstimator
Lindner(k = 1, w = 0, base = 2)

The Lindner transfer entropy estimator (Lindner et al., 2011), which is also used in the Trentool MATLAB toolbox, and is based on nearest neighbor searches.

w is the Theiler window, which determines if temporal neighbors are excluded during neighbor searches (defaults to 0, meaning that only the point itself is excluded when searching for neighbours).

Description

For a given points in the joint embedding space jᵢ, this estimator first computes the distance dᵢ from jᵢ to its k-th nearest neighbor. Then, for each point mₖ[i] in the k-th marginal space, it counts the number of points within radius dᵢ.

The transfer entropy is then computed as

\[TE(X \to Y) = \psi(k) + \dfrac{1}{N} \sum_{i}^n \left[ \sum_{k=1}^3 \left( \psi(m_k[i] + 1) \right) \right],\]

where the index k references the three marginal subspaces T, TTf and ST for which neighbor searches are performed.

SymbolicTransferEntropy <: TransferEntropyEstimator
SymbolicTransferEntropy(; m = 3, τ = 1, lt = ComplexityMeasures.isless_rand

A convenience estimator for symbolic transfer entropy (Staniek and Lehnertz, 2008).

Description

Symbolic transfer entropy consists of two simple steps. First, the input time series are embedded with embedding lag m and delay τ. The ordinal patterns of the embedding vectors are then encoded using SymbolicPermutation with marginal_encodings. This transforms the input time series into integer time series using OrdinalPatternEncoding.

Transfer entropy is then estimated as usual on the encoded timeseries with transferentropy and the CountOccurrences naive frequency estimator.

Hilbert(est;
    source::InstantaneousSignalProperty = Phase(),
    target::InstantaneousSignalProperty = Phase(),
    cond::InstantaneousSignalProperty = Phase())
) <: TransferDifferentialEntropyEstimator

Compute transfer entropy on instantaneous phases/amplitudes of relevant signals, which are obtained by first applying the Hilbert transform to each signal, then extracting the phases/amplitudes of the resulting complex numbers (Paluš, 2014). Original time series are thus transformed to instantaneous phase/amplitude time series. Transfer entropy is then estimated using the provided est on those phases/amplitudes (use e.g. VisitationFrequency, or SymbolicPermutation).

See also: Phase, Amplitude.

Phase <: InstantaneousSignalProperty

Indicates that the instantaneous phases of a signal should be used.

Amplitude <: InstantaneousSignalProperty

Indicates that the instantaneous amplitudes of a signal should be used.

optimize_marginals_te([scheme = OptimiseTraditional()], s, t, [c]) → EmbeddingTE

Optimize marginal embeddings for transfer entropy computation from source time series s to target time series t, conditioned on c if c is given, using the provided optimization scheme.

EmbeddingTE(; dS = 1, dT = 1, dTf = 1, dC = 1, τS = -1, τT = -1, ηTf = 1, τC = -1)
EmbeddingTE(opt::OptimiseTraditional, s, t, [c])

EmbeddingTE provide embedding parameters for transfer entropy analysis using either TEShannon, TERenyi, or in general any subtype of TransferEntropy, which in turns dictates the embedding used with transferentropy.

The second method finds parameters using the "traditional" optimised embedding techniques from DynamicalSystems.jl

Convention for generalized delay reconstruction

We use the following convention. Let $s(i)$ be time series for the source variable, $t(i)$ be the time series for the target variable and $c(i)$ the time series for the conditional variable. To compute transfer entropy, we need the following marginals:

\[\begin{aligned} T^{+} &= \{t(i+\eta^1), t(i+\eta^2), \ldots, (t(i+\eta^{d_{T^{+}}}) \} \\ T^{-} &= \{ (t(i+\tau^0_{T}), t(i+\tau^1_{T}), t(i+\tau^2_{T}), \ldots, t(t + \tau^{d_{T} - 1}_{T})) \} \\ S^{-} &= \{ (s(i+\tau^0_{S}), s(i+\tau^1_{S}), s(i+\tau^2_{S}), \ldots, s(t + \tau^{d_{S} - 1}_{S})) \} \\ C^{-} &= \{ (c(i+\tau^0_{C}), c(i+\tau^1_{C}), c(i+\tau^2_{C}), \ldots, c(t + \tau^{d_{C} - 1}_{C})) \} \end{aligned}\]

Depending on the application, the delay reconstruction lags $\tau^k_{T} \leq 0$, $\tau^k_{S} \leq 0$, and $\tau^k_{C} \leq 0$ may be equally spaced, or non-equally spaced. The same applied to the prediction lag(s), but typically only a only a single predictions lag $\eta^k$ is used (so that $d_{T^{+}} = 1$).

For transfer entropy, traditionally at least one $\tau^k_{T}$, one $\tau^k_{S}$ and one $\tau^k_{C}$ equals zero. This way, the $T^{-}$, $S^{-}$ and $C^{-}$ marginals always contains present/past states, while the $\mathcal T$ marginal contain future states relative to the other marginals. However, this is not a strict requirement, and modern approaches that searches for optimal embeddings can return embeddings without the intantaneous lag.

Combined, we get the generalized delay reconstruction $\mathbb{E} = (T^{+}_{(d_{T^{+}})}, T^{-}_{(d_{T})}, S^{-}_{(d_{S})}, C^{-}_{(d_{C})})$. Transfer entropy is then computed as

\[\begin{aligned} TE_{S \rightarrow T | C} = \int_{\mathbb{E}} P(T^{+}, T^-, S^-, C^-) \log_{b}{\left(\frac{P(T^{+} | T^-, S^-, C^-)}{P(T^{+} | T^-, C^-)}\right)}, \end{aligned}\]

or, if conditionals are not relevant,

\[\begin{aligned} TE_{S \rightarrow T} = \int_{\mathbb{E}} P(T^{+}, T^-, S^-) \log_{b}{\left(\frac{P(T^{+} | T^-, S^-)}{P(T^{+} | T^-)}\right)}, \end{aligned}\]

Here,

• $T^{+}$ denotes the $d_{T^{+}}$-dimensional set of vectors furnishing the future states of $T$ (almost always equal to 1 in practical applications),
• $T^{-}$ denotes the $d_{T}$-dimensional set of vectors furnishing the past and present states of $T$,
• $S^{-}$ denotes the $d_{S}$-dimensional set of vectors furnishing the past and present of $S$, and
• $C^{-}$ denotes the $d_{C}$-dimensional set of vectors furnishing the past and present of $C$.

Keyword arguments

• dS, dT, dC, dTf (f for future) are the dimensions of the $S^{-}$, $T^{-}$, $C^{-}$ and $T^{+}$ marginals. The parameters dS, dT, dC and dTf must each be a positive integer number.
• τS, τT, τC are the embedding lags for $S^{-}$, $T^{-}$, $C^{-}$. Each parameter are integers ∈ 𝒩⁰⁻, or a vector of integers ∈ 𝒩⁰⁻, so that $S^{-}$, $T^{-}$, $C^{-}$ always represents present/past values. If e.g. τT is an integer, then for the $T^-$ marginal is constructed using lags $\tau_{T} = \{0, \tau, 2\tau, \ldots, (d_{T}- 1)\tau_T \}$. If is a vector, e.g. τΤ = [-1, -5, -7], then the dimension dT must match the lags, and precisely those lags are used: $\tau_{T} = \{-1, -5, -7 \}$.
• The prediction lag(s) ηTf is a positive integer. Combined with the requirement that the other delay parameters are zero or negative, this ensures that we're always predicting from past/present to future. In typical applications, ηTf = 1 is used for transfer entropy.

Examples

Say we wanted to compute the Shannon transfer entropy $TE^S(S \to T) = I^S(T^+; S^- | T^-)$. Using some modern procedure for determining optimal embedding parameters using methods from DynamicalSystems.jl, we find that the optimal embedding of $T^{-}$ is three-dimensional and is given by the lags [0, -5, -8]. Using the same procedure, we find that the optimal embedding of $S^{-}$ is two-dimensional with lags $[-1, -8]$. We want to predicting a univariate version of the target variable one time step into the future (ηTf = 1). The total embedding is then the set of embedding vectors

$E_{TE} = \{ (T(i+1), S(i-1), S(i-8), T(i), T(i-5), T(i-8)) \}$. Translating this to code, we get:

using CausalityTools
julia> EmbeddingTE(dT=3, τT=[0, -5, -8], dS=2, τS=[-1, -4], ηTf=1)

# output
EmbeddingTE(dS=2, dT=3, dC=1, dTf=1, τS=[-1, -4], τT=[0, -5, -8], τC=-1, ηTf=1)