pyFTS.benchmarks package

Submodules

pyFTS.benchmarks.Measures module

pyFTS module for common benchmark metrics

pyFTS.benchmarks.Measures.BoxLjungStatistic(data, h)

Q Statistic for Ljung–Box test

Parameters:

• data –
• h –

Returns:

pyFTS.benchmarks.Measures.BoxPierceStatistic(data, h)

Q Statistic for Box-Pierce test

Parameters:

• data –
• h –

Returns:

pyFTS.benchmarks.Measures.TheilsInequality(targets, forecasts)

Theil’s Inequality Coefficient

Parameters:

• targets –
• forecasts –

Returns:

pyFTS.benchmarks.Measures.UStatistic(targets, forecasts)

Theil’s U Statistic

Parameters:

• targets –
• forecasts –

Returns:

pyFTS.benchmarks.Measures.acf(data, k)

Autocorrelation function estimative

Parameters:

• data –
• k –

Returns:

pyFTS.benchmarks.Measures.brier_score(targets, densities)

Brier (1950). “Verification of Forecasts Expressed in Terms of Probability”. Monthly Weather Review. 78: 1–3.

pyFTS.benchmarks.Measures.coverage(targets, forecasts)

Percent of target values that fall inside forecasted interval

pyFTS.benchmarks.Measures.crps(targets, densities)

Continuous Ranked Probability Score

Parameters:

• targets – a list with the target values
• densities – a list with pyFTS.probabil objectsistic.ProbabilityDistribution

Returns:

float

pyFTS.benchmarks.Measures.get_distribution_statistics(data, model, **kwargs)

Get CRPS statistic and time for a forecasting model

Parameters:

• data – test data
• model – FTS model with probabilistic forecasting capability
• kwargs –

Returns:

a list with the CRPS and execution time

pyFTS.benchmarks.Measures.get_interval_statistics(data, model, **kwargs)

Condensate all measures for point interval forecasters

Parameters:

• data – test data
• model – FTS model with interval forecasting capability
• kwargs –

Returns:

a list with the sharpness, resolution, coverage, .05 pinball mean,

.25 pinball mean, .75 pinball mean and .95 pinball mean.

pyFTS.benchmarks.Measures.get_point_statistics(data, model, **kwargs)

Condensate all measures for point forecasters

Parameters:

• data – test data
• model – FTS model with point forecasting capability
• kwargs –

Returns:

a list with the RMSE, SMAPE and U Statistic

pyFTS.benchmarks.Measures.heavyside(bin, target)

pyFTS.benchmarks.Measures.heavyside_cdf(bins, targets)

pyFTS.benchmarks.Measures.mape(targets, forecasts)

Mean Average Percentual Error

Parameters:

• targets –
• forecasts –

Returns:

pyFTS.benchmarks.Measures.mape_interval(targets, forecasts)

pyFTS.benchmarks.Measures.pinball(tau, target, forecast)

Pinball loss function. Measure the distance of forecast to the tau-quantile of the target

Parameters:

• tau – quantile value in the range (0,1)
• target –
• forecast –

Returns:

float, distance of forecast to the tau-quantile of the target

pyFTS.benchmarks.Measures.pinball_mean(tau, targets, forecasts)

Mean pinball loss value of the forecast for a given tau-quantile of the targets

Parameters:

• tau – quantile value in the range (0,1)
• targets – list of target values
• forecasts – list of prediction intervals

Returns:

float, the pinball loss mean for tau quantile

pyFTS.benchmarks.Measures.pmf_to_cdf(density)

pyFTS.benchmarks.Measures.resolution(forecasts)

Resolution - Standard deviation of the intervals

pyFTS.benchmarks.Measures.rmse(targets, forecasts)

Root Mean Squared Error

Parameters:

• targets –
• forecasts –

Returns:

pyFTS.benchmarks.Measures.rmse_interval(targets, forecasts)

Root Mean Squared Error

Parameters:

• targets –
• forecasts –

Returns:

pyFTS.benchmarks.Measures.sharpness(forecasts)

Sharpness - Mean size of the intervals

pyFTS.benchmarks.Measures.smape(targets, forecasts, type=2)

Symmetric Mean Average Percentual Error

Parameters:

• targets –
• forecasts –
• type –

Returns:

pyFTS.benchmarks.Measures.winkler_mean(tau, targets, forecasts)

Mean Winkler score value of the forecast for a given tau-quantile of the targets

Parameters:

• tau – quantile value in the range (0,1)
• targets – list of target values
• forecasts – list of prediction intervals

Returns:

float, the Winkler score mean for tau quantile

pyFTS.benchmarks.Measures.winkler_score(tau, target, forecast)

18. 12. Winkler, A Decision-Theoretic Approach to Interval Estimation, J. Am. Stat. Assoc. 67 (337) (1972) 187–191. doi:10.2307/2284720.

pyFTS.benchmarks.ResidualAnalysis module

pyFTS.benchmarks.Util module

pyFTS.benchmarks.arima module

pyFTS.benchmarks.benchmarks module

pyFTS.benchmarks.distributed_benchmarks module

pyFTS.benchmarks.knn module

pyFTS.benchmarks.naive module

pyFTS.benchmarks.parallel_benchmarks module

pyFTS.benchmarks.quantreg module

Module contents

pyFTS module for benchmarking the FTS models