Package jsl.utilities.statistic

Class BatchStatistic

  • All Implemented Interfaces:
    java.lang.Comparable<AbstractStatistic>, GetNameIfc, IdentityIfc, ArraySaverIfc, CollectorIfc, GetCSVStatisticIfc, StatisticAccessorIfc
    public class BatchStatistic
extends AbstractStatistic
    This class automates the batching of observations that may be dependent. It computes the batch means of the batches and reports statistics across the batches. Suppose we have observations, Y(1), Y(2), Y(3), ... Y(n). This class specifies the minimum number of batches, the minimum number of observations per batch, and a maximum batch multiple. The defaults are 20, 16, and 2, respectively. This implies that the maximum number of batches will be 40 = (min number of batches times the maximum batch multiple). The class computes the average of each batch, which are called the batch means. Once the minimum number of observations are observed, a batch is formed. As more and more observations are collected, more and more batches are formed until the maximum number of batches is reached. Then the batches are re-batched down so that there are 20 batches (the minimum number of batches). This re-batching essentially doubles the batch size and halves the number of batches. In other words, each sequential pair of batches are combined into one batch by averaging their batch means. The purpose of this batching process is to break up correlation structure within the data. Confidence intervals and summary statistics can be reported across the batch means under the assumption that the batch means are independent. The lag-1 correlation of the batch means is available as well as the Von-Neumann test statistic for independence of the batch means.

    • Field Detail

      • MIN_NUM_BATCHES

        public static final int MIN_NUM_BATCHES
        the default minimum number of batches
        See Also:
        Constant Field Values

      • MIN_NUM_OBS_PER_BATCH

        public static final int MIN_NUM_OBS_PER_BATCH
        the default minimum number of observations per batch
        See Also:
        Constant Field Values

      • MAX_BATCH_MULTIPLE

        public static final int MAX_BATCH_MULTIPLE
        the default multiplier that determines the maximum number of batches
        See Also:
        Constant Field Values

    • Constructor Detail

      • BatchStatistic

        public BatchStatistic()
        Creates a BatchStatistic with defaults: The minimum number of batches = 20 The minimum number of observations per batch = 16 The maximum number of batches = 40, i.e. max batch multiple is 2

      • BatchStatistic

        public BatchStatistic​(java.lang.String name)
        Creates a BatchStatistic with defaults: The minimum number of batches = 20 The minimum number of observations per batch = 16 The maximum number of batches = 40, i.e. max batch multiple is 2
        Parameters:
        name - The name for the statistic

      • BatchStatistic

        public BatchStatistic​(java.lang.String name,
                      double[] values)
        Creates a BatchStatistic with defaults: The minimum number of batches = 20 The minimum number of observations per batch = 16 The maximum number of batches = 40, i.e. max batch multiple is 2
        Parameters:
        name - The name for the statistic
        values - An array of values to collect on

      • BatchStatistic

        public BatchStatistic​(double[] values)
        Creates a BatchStatistic with defaults: The minimum number of batches = 20 The minimum number of observations per batch = 16 The maximum number of batches = 40, i.e. max batch multiple is 2
        Parameters:
        values - An array of values to collect on

      • BatchStatistic

        public BatchStatistic​(int minNumBatches,
                      int minBatchSize,
                      int maxNBMultiple)
        Creates a BatchStatistic
        Parameters:
        minNumBatches - The minimum number of batches, must be > = 2
        minBatchSize - The minimum number of observations per batch, must be >= 2
        maxNBMultiple - The maximum number of batches as a multiple of the minimum number of batches. For example, if minNumBatches = 20 and maxNBMultiple = 2 then the maximum number of batches allowed will be 40. maxNBMultiple must be 2 or more.

      • BatchStatistic

        public BatchStatistic​(int minNumBatches,
                      int minBatchSize,
                      int maxNBMultiple,
                      double[] values)
        Creates a BatchStatistic For example, if minNumBatches = 20 and maxNBMultiple = 2 then the maximum number of batches allowed will be 40. maxNBMultiple must be 2 or more.
        Parameters:
        minNumBatches - The minimum number of batches, must be >= 2
        minBatchSize - The minimum number of observations per batch, must be >= 2
        maxNBMultiple - The maximum number of batches as a multiple of the minimum number of batches.
        values - An array of values to collect on

      • BatchStatistic

        public BatchStatistic​(int minNumBatches,
                      int minBatchSize,
                      int maxNBMultiple,
                      java.lang.String name)
        Creates a BatchStatistic For example, if minNumBatches = 20 and maxNBMultiple = 2 then the maximum number of batches allowed will be 40. maxNBMultiple must be 2 or more.
        Parameters:
        minNumBatches - The minimum number of batches, must be >= 2
        minBatchSize - The minimum number of observations per batch, must be >= 2
        maxNBMultiple - The maximum number of batches as a multiple of the minimum number of batches.
        name - A String representing the name of the statistic

      • BatchStatistic

        public BatchStatistic​(int minNumBatches,
                      int minBatchSize,
                      int maxNBMultiple,
                      java.lang.String name,
                      double[] values)
        Creates a BatchStatistic with the given name For example, if minNumBatches = 20 and maxNBMultiple = 2 then the maximum number of batches allowed will be 40. maxNBMultiple must be 2 or more.
        Parameters:
        minNumBatches - The minimum number of batches, must be >= 2
        minBatchSize - The minimum number of observations per batch, must be >= 2
        maxNBMultiple - The maximum number of batches as a multiple of the minimum number of batches.
        name - A String representing the name of the statistic
        values - An array of values to collect on

    • Method Detail

      • getMinNumberOfBatches

        public final int getMinNumberOfBatches()
        Returns the minimum number of Batches that are needed
        Returns:
        the minimum number of Batches that are needed

      • getMaxNumberOfBatchesMultiple

        public final int getMaxNumberOfBatchesMultiple()
        Returns the maximum number of batches that is declared
        Returns:
        the maximum number of batches that is declared

      • getMinBatchSize

        public final int getMinBatchSize()
        Returns the minimum number of observations needed in a batch
        Returns:
        the minimum number of observations needed in a batch

      • setConfidenceLevel

        public void setConfidenceLevel​(double alpha)
        Description copied from class: AbstractStatistic
        Sets the confidence level for the statistic
        Overrides:
        setConfidenceLevel in class AbstractStatistic
        Parameters:
        alpha - must be in (0, 1)

      • reset

        public final void reset()
        Description copied from interface: CollectorIfc
        Resets the collector as if no observations had been collected.

      • checkMean

        public final boolean checkMean​(double mean)
        Checks if the supplied value falls within getAverage() +/- getHalfWidth()
        Parameters:
        mean - the mean to check
        Returns:
        true if the supplied value falls within getAverage() +/- getHalfWidth()

      • collect

        public final void collect​(double value)
        Description copied from interface: CollectorIfc
        Collect on the supplied value
        Parameters:
        value - a double representing the observation

      • rebatchToNumberOfBatches

        public final Statistic rebatchToNumberOfBatches​(int numBatches)
        Returns a StatisticAccessorIfc which has collected statistics after re-batching the batch means to the supplied number of batches
        Parameters:
        numBatches - the desired number of batches, must be > =2
        Returns:
        A reference to a Statistic

      • rebatchToNumberOfBatches

        public final Statistic rebatchToNumberOfBatches​(int numBatches,
                                                boolean save)
        Returns a StatisticAccessorIfc which has collected statistics after re-batching the batch means to the supplied number of batches
        Parameters:
        numBatches - the desired number of batches, must be >=2
        save - if true the returned Statistic has its save data option turned on
        Returns:
        A reference to a Statistic

      • getBatchMeanArrayCopy

        public final double[] getBatchMeanArrayCopy()
        Returns a copy of the batch means array. Zero index is the first batch mean
        Returns:
        An array holding the batch means

      • getAverage

        public final double getAverage()
        Gets the average over the batches.
        Returns:
        A double representing the average or Double.NaN if no observations.

      • getCount

        public final double getCount()
        Gets the count of the number of batches.
        Returns:
        A double representing the count

      • getSum

        public final double getSum()
        Gets the sum of the batch means
        Returns:
        A double representing the sum

      • getDeviationSumOfSquares

        public final double getDeviationSumOfSquares()
        Gets the sum of squares of the deviations from the average. This is the numerator in the classic sample variance formula
        Returns:
        A double representing the sum of squares of the deviations from the average

      • getVariance

        public final double getVariance()
        Gets the sample variance of the batches.
        Returns:
        A double representing the sample variance or Double.NaN if 1 or less batch.

      • getStandardDeviation

        public final double getStandardDeviation()
        Gets the sample standard deviation of the batches. Simply the square root of getVariance()
        Returns:
        A double representing the generate standard deviation or Double.NaN if 1 or less observations.

      • getHalfWidth

        public final double getHalfWidth​(double level)
        Gets the confidence interval half-width for the batches. Simply the generate standard error times the confidence coefficient
        Parameters:
        level - the confidence interval level
        Returns:
        A double representing the half-width or Double.NaN if < 1 observation

      • getMin

        public final double getMin()
        Gets the minimum of the batch means.
        Returns:
        A double representing the minimum

      • getMax

        public final double getMax()
        Gets the maximum of the batch means.
        Returns:
        A double representing the maximum

      • getLastValue

        public final double getLastValue()
        Gets the last observed batch mean
        Returns:
        A double representing the last batch mean

      • getKurtosis

        public final double getKurtosis()
        Gets the kurtosis of the batch means
        Returns:
        A double representing the kurtosis

      • getSkewness

        public final double getSkewness()
        Gets the skewness of the batch means
        Returns:
        A double representing the skewness

      • getStandardError

        public final double getStandardError()
        Gets the standard error of the batch means. Simply the generate standard deviation divided by the square root of the number of batches
        Returns:
        A double representing the standard error or Double.NaN if < 1 batches

      • getLag1Correlation

        public final double getLag1Correlation()
        Gets the lag-1 generate correlation of the unweighted batch means. Note: See Box, Jenkins, Reinsel, Time Series Analysis, 3rd edition, Prentice-Hall, pg 31
        Returns:
        A double representing the generate correlation or Double.NaN if <=2 batches

      • getLag1Covariance

        public final double getLag1Covariance()
        Gets the lag-1 generate covariance of the unweighted batch means. Note: See Box, Jenkins, Reinsel, Time Series Analysis, 3rd edition, Prentice-Hall, pg 31
        Returns:
        A double representing the generate correlation or Double.NaN if <=2 batches

      • getVonNeumannLag1TestStatistic

        public final double getVonNeumannLag1TestStatistic()
        Gets the Von Neumann Lag 1 test statistic for checking the hypothesis that the batches are uncorrelated Note: See Handbook of Simulation, Jerry Banks editor, McGraw-Hill, pg 253.
        Returns:
        A double representing the Von Neumann test statistic

      • getVonNeumannLag1TestStatisticPValue

        public final double getVonNeumannLag1TestStatisticPValue()
        Description copied from interface: StatisticAccessorIfc
        Returns the asymptotic p-value for the Von Nueumann Lag-1 Test Statistic:

        Normal.stdNormalComplementaryCDF(getVonNeumannLag1TestStatistic());

        Returns:
        the p-value

      • getLeadingDigitRule

        public final int getLeadingDigitRule​(double a)
        Description copied from interface: StatisticAccessorIfc
        Computes the right most meaningful digit according to (int)Math.floor(Math.log10(a*getStandardError())) See doi 10.1287.opre.1080.0529 by Song and Schmeiser
        Parameters:
        a - the std error multiplier
        Returns:
        the meaningful digit

      • toString

        public final java.lang.String toString()
        Description copied from interface: StatisticAccessorIfc
        Returns a String representation of the Statistic
        Specified by:
        toString in interface StatisticAccessorIfc
        Overrides:
        toString in class java.lang.Object
        Returns:
        A String with basic summary statistics

      • getMaxNumBatches

        public final int getMaxNumBatches()
        Returns:
        the max number of batches permitted

      • getNumRebatches

        public final int getNumRebatches()
        Returns:
        the number of re-batches performed

      • getAmountLeftUnbatched

        public final double getAmountLeftUnbatched()
        Returns:
        the amount of data that did not fit into a full batch

      • getTotalNumberOfObservations

        public final double getTotalNumberOfObservations()
        Gets the total number of observations observed
        Returns:
        a double representing the total number of observations

      • getCurrentBatchStatistic

        public final StatisticAccessorIfc getCurrentBatchStatistic()
        Returns a reference to the StatisticAccessorIfc that is tabulating the current batch
        Returns:
        a reference to the StatisticAccessorIfc that is tabulating the current batch

      • getCurrentBatchSize

        public final int getCurrentBatchSize()
        Returns the current number of observations per batch. This value varies during the execution. After all data have been collected, this method returns the number of observations per batch
        Returns:
        Returns the current Batch Size.