TimeseriesSurrogates.jl

surrogate(x, method::Surrogate) → s

Create a single surrogate timeseries s from x based on the given method. If you want to generate more than one surrogates from x, you should use surrogenerator.

surrogenerator(x, method::Surrogate) → sg::SurrogateGenerator

Initialize a generator that creates surrogates of x on demand, based on given method. This is efficient, because for most methods some things can be initialized and reused for every surrogate.

To generate a surrogate, call sg as a function with no arguments, e.g.:

sg = surrogenerator(x, method)
for i in 1:1000
    s = sg()
    # do stuff with s and or x
    result[i] = stuff
end

RandomShuffle() <: Surrogate

A random constrained surrogate, generated by shifting values around.

Random shuffle surrogates preserve the mean, variance and amplitude distribution of the original signal. Properties not preserved are any temporal information, such as the power spectrum and hence linear correlations.

The null hypothesis this method can test for is whether the data are uncorrelated noise, possibly measured via a nonlinear function. Specifically, random shuffle surrogate can test the null hypothesis that the original signal is produced by independent and identically distributed random variables[^Theiler1991, ^Lancaster2018].

Beware: random shuffle surrogates do not cover the case of correlated noise^{[Lancaster2018]}.

BlockShuffle(n::Int) <: Surrogate

A block shuffle surrogate constructed by dividing the time series into n blocks of roughly equal width at random indices (end blocks are wrapped around to the start of the time series).

Block shuffle surrogates roughly preserve short-range temporal properties in the time series (e.g. correlations at lags less than the block length), but break any long-term dynamical information (e.g. correlations beyond the block length).

Hence, these surrogates can be used to test any null hypothesis aimed at comparing short-range dynamical properties versus long-range dynamical properties of the signal.

RandomFourier(phases = true) <: Surrogate

A surrogate that randomizes the Fourier components of the signal in some manner. If phases==true, the phases are randomized, otherwise the amplitudes are randomized.

Random Fourier phase surrogates^{[Theiler1991]} preserve the autocorrelation function, or power spectrum, of the original signal. Random Fourier amplitude surrogates preserve the mean and autocorrelation function but do not preserve the variance of the original. Random amplitude surrogates are not common in the literature, but are provided for convenience.

Random phase surrogates can be used to test the null hypothesis that the original signal was produced by a linear Gaussian process ^{[Theiler1991]}.

TFTS(fϵ::Real)

A truncated Fourier transform surrogate^{[Nakamura2006]} (TFTS).

TFTS surrogates are generated by leaving some frequencies untouched when performing the phase shuffling step (as opposed to randomizing all frequencies, like for RandomFourier surrogates).

These surrogates were designed to deal with data with irregular fluctuations superimposed over long term trends (by preserving low frequencies)^{[Nakamura2006]}. Hence, TFTS surrogates can be used to test the null hypothesis that the signal is a stationary linear system generated the irregular fluctuations part of the signal^{[Nakamura2006]}.

Controlling the truncation of the spectrum

The truncation parameter fϵ ∈ [-1, 0) ∪ (0, 1] controls which parts of the spectrum are preserved.

• If fϵ > 0, then fϵ indicates the ratio of high frequency domain to the entire frequency domain. For example, fϵ = 0.5 preserves 50% of the frequency domain (randomizing the higher frequencies, leaving low frequencies intact).
• If fϵ < 0, then fϵ indicates ratio of low frequency domain to the entire frequency domain. For example, fϵ = -0.2 preserves 20% of the frequency domain (leaving higher frequencies intact, randomizing the lower frequencies).
• If fϵ ± 1, then all frequencies are randomized. The method is then equivalent to RandomFourier.

The appropriate value of fϵ strongly depends on the data and time series length, and must be manually determined^{[Nakamura2006]}, for example by comparing periodograms for the time series and the surrogates.

AAFT()

An amplitude-adjusted-fourier-transform surrogate^{[Theiler1991]}.

AAFT have the same linear correlation, or periodogram, and also preserves the amplitude distribution of the original data.

AAFT can be used to test the null hypothesis that the data come from a monotonic nonlinear transformation of a linear Gaussian process (also called integrated white noise)^{[Theiler1991]}.

TAAFT(fϵ)

An truncated version of the amplitude-adjusted-fourier-transform surrogate^{[Theiler1991][Nakamura2006]}.

The truncation parameter and phase randomization procedure is identical to TFTS, but here an additional step of rescaling back to the original data is performed. This preserves the amplitude distribution of the original data.

References

IAAFT(M = 100, tol = 1e-6, W = 75)

An iteratively adjusted amplitude-adjusted-fourier-transform surrogate^{[SchreiberSchmitz1996]}.

IAAFT surrogate have the same linear correlation, or periodogram, and also preserves the amplitude distribution of the original data, but are improved relative to AAFT through iterative adjustment (which runs for a maximum of M steps). During the iterative adjustment, the periodograms of the original signal and the surrogate are coarse-grained and the powers are averaged over W equal-width frequency bins. The iteration procedure ends when the relative deviation between the periodograms is less than tol (or when M is reached).

IAAFT, just as AAFT, can be used to test the null hypothesis that the data come from a monotonic nonlinear transformation of a linear Gaussian process.

PseudoPeriodic(d, τ, ρ, shift=true) <: Surrogate

Create surrogates suitable for pseudo-periodic signals. They retain the periodic structure of the signal, while inter-cycle dynamics that are either deterministic or correlated noise are destroyed (for appropriate ρ choice). Therefore these surrogates are suitable to test the null hypothesis that the signal is periodic with uncorrelated noise^[Small2001].

Arguments d, τ, ρ are as in the paper, the embedding dimension, delay time and noise radius. The method works by performing a delay coordinates ambedding via the library DynamicalSystems.jl. See its documentation for choosing appropriate values for d, τ. For ρ, we have implemented the method proposed in the paper in the function noiseradius.

The argument shift is not discussed in the paper, but it is possible to adjust the algorithm so that there is little phase shift between the periodic component of the original and surrogate data.

WLS(surromethod::Surrogate = IAAFT(), 
    rescale::Bool = true,
    wt::Wavelets.WT.OrthoWaveletClass = Wavelets.WT.Daubechies{16}())

A wavelet surrogate generated by taking the maximal overlap discrete wavelet transform (MODWT) of the signal, shuffling detail coefficients at each dyadic scale using the provided surromethod, then taking the inverse transform to obtain a surrogate.

Coefficient shuffling method

In contrast to the original implementation where IAAFT is used, you may choose to use any surrogate method from this package to perform the randomization of the detail coefficients at each dyadic scale. Note: The iterative procedure after the rank ordering step (step [v] in ^{[Keylock2006]}) is not performed in this implementation.

If surromethod == IAAFT(), the wavelet surrogates preserves the local mean and variance structure of the signal, but randomises nonlinear properties of the signal (i.e. Hurst exponents)^{[Keylock2006]}. These surrogates can therefore be used to test for changes in nonlinear properties of the original signal.

In contrast to IAAFT surrogates, the IAAFT-wavelet surrogates also preserves nonstationarity. Using other surromethods does not necessarily preserve nonstationarity.

To deal with nonstationary signals, Keylock (2006) recommends using a wavelet with a high number of vanishing moments. Thus, the default is to use a Daubechies wavelet with 16 vanishing moments.

Rescaling

If rescale == true, then surrogate values are mapped onto the values of the original time series, as in the AAFT algorithm. If rescale == false, surrogate values are not constrained to the original time series values.

noiseradius(x::AbstractVector, d::Int, τ, ρs, n = 1) → ρ

Use the proposed* algorithm of^[Small2001] to estimate optimal ρ value for PseudoPeriodic surrogates, where ρs is a vector of possible ρ values.

*The paper is ambiguous about exactly what to calculate. Here we count how many times we have pairs of length-2 that are identical in x and its surrogate, but are not also part of pairs of length-3.

This function directly returns the arg-maximum of the evaluated distribution of these counts versus ρ, use TimeseriesSurrogates._noiseradius with same arguments to get the actual distribution. n means to repeat τhe evaluation n times, which increases accuracy.