pyFTS.benchmarks package

Module contents

pyFTS module for benchmarking the FTS models

Submodules

pyFTS.benchmarks.benchmarks module

pyFTS.benchmarks.Measures module

pyFTS module for common benchmark metrics

pyFTS.benchmarks.Measures.BoxLjungStatistic(data, h)[source]

Q Statistic for Ljung–Box test

Parameters:
  • data
  • h
Returns:

pyFTS.benchmarks.Measures.BoxPierceStatistic(data, h)[source]

Q Statistic for Box-Pierce test

Parameters:
  • data
  • h
Returns:

pyFTS.benchmarks.Measures.TheilsInequality(targets, forecasts)[source]

Theil’s Inequality Coefficient

Parameters:
  • targets
  • forecasts
Returns:

pyFTS.benchmarks.Measures.UStatistic(targets, forecasts)[source]

Theil’s U Statistic

Parameters:
  • targets
  • forecasts
Returns:

pyFTS.benchmarks.Measures.acf(data, k)[source]

Autocorrelation function estimative

Parameters:
  • data
  • k
Returns:

pyFTS.benchmarks.Measures.brier_score(targets, densities)[source]

Brier (1950). “Verification of Forecasts Expressed in Terms of Probability”. Monthly Weather Review. 78: 1–3.

pyFTS.benchmarks.Measures.coverage(targets, forecasts)[source]

Percent of target values that fall inside forecasted interval

pyFTS.benchmarks.Measures.crps(targets, densities)[source]

Continuous Ranked Probability Score

Parameters:
  • targets – a list with the target values
  • densities – a list with pyFTS.probabil objectsistic.ProbabilityDistribution
Returns:

float

pyFTS.benchmarks.Measures.get_distribution_statistics(data, model, **kwargs)[source]

Get CRPS statistic and time for a forecasting model

Parameters:
  • data – test data
  • model – FTS model with probabilistic forecasting capability
  • kwargs
Returns:

a list with the CRPS and execution time

pyFTS.benchmarks.Measures.get_interval_statistics(data, model, **kwargs)[source]

Condensate all measures for point interval forecasters

Parameters:
  • data – test data
  • model – FTS model with interval forecasting capability
  • kwargs
Returns:

a list with the sharpness, resolution, coverage, .05 pinball mean,

.25 pinball mean, .75 pinball mean and .95 pinball mean.

pyFTS.benchmarks.Measures.get_point_statistics(data, model, **kwargs)[source]

Condensate all measures for point forecasters

Parameters:
  • data – test data
  • model – FTS model with point forecasting capability
  • kwargs
Returns:

a list with the RMSE, SMAPE and U Statistic

pyFTS.benchmarks.Measures.heavyside(bin, target)[source]
pyFTS.benchmarks.Measures.heavyside_cdf(bins, targets)[source]
pyFTS.benchmarks.Measures.mape(targets, forecasts)[source]

Mean Average Percentual Error

Parameters:
  • targets
  • forecasts
Returns:

pyFTS.benchmarks.Measures.mape_interval(targets, forecasts)[source]
pyFTS.benchmarks.Measures.pinball(tau, target, forecast)[source]

Pinball loss function. Measure the distance of forecast to the tau-quantile of the target

Parameters:
  • tau – quantile value in the range (0,1)
  • target
  • forecast
Returns:

float, distance of forecast to the tau-quantile of the target

pyFTS.benchmarks.Measures.pinball_mean(tau, targets, forecasts)[source]

Mean pinball loss value of the forecast for a given tau-quantile of the targets

Parameters:
  • tau – quantile value in the range (0,1)
  • targets – list of target values
  • forecasts – list of prediction intervals
Returns:

float, the pinball loss mean for tau quantile

pyFTS.benchmarks.Measures.pmf_to_cdf(density)[source]
pyFTS.benchmarks.Measures.resolution(forecasts)[source]

Resolution - Standard deviation of the intervals

pyFTS.benchmarks.Measures.rmse(targets, forecasts)[source]

Root Mean Squared Error

Parameters:
  • targets
  • forecasts
Returns:

pyFTS.benchmarks.Measures.rmse_interval(targets, forecasts)[source]

Root Mean Squared Error

Parameters:
  • targets
  • forecasts
Returns:

pyFTS.benchmarks.Measures.sharpness(forecasts)[source]

Sharpness - Mean size of the intervals

pyFTS.benchmarks.Measures.smape(targets, forecasts, type=2)[source]

Symmetric Mean Average Percentual Error

Parameters:
  • targets
  • forecasts
  • type
Returns:

pyFTS.benchmarks.Measures.winkler_mean(tau, targets, forecasts)[source]

Mean Winkler score value of the forecast for a given tau-quantile of the targets

Parameters:
  • tau – quantile value in the range (0,1)
  • targets – list of target values
  • forecasts – list of prediction intervals
Returns:

float, the Winkler score mean for tau quantile

pyFTS.benchmarks.Measures.winkler_score(tau, target, forecast)[source]
    1. Winkler, A Decision-Theoretic Approach to Interval Estimation, J. Am. Stat. Assoc. 67 (337) (1972) 187–191. doi:10.2307/2284720.

pyFTS.benchmarks.ResidualAnalysis module

pyFTS.benchmarks.Util module

pyFTS.benchmarks.arima module

pyFTS.benchmarks.knn module

pyFTS.benchmarks.naive module

pyFTS.benchmarks.quantreg module