Package InversionTest :: Module MonteCarloSampler :: Class MCBStoppingRuleTest
[hide private]
[frames] | no frames]

Class MCBStoppingRuleTest

         object --+        
                  |        
MonteCarloEstimator --+    
                      |    
       MCHypothesisTest --+
                          |
                         MCBStoppingRuleTest
Known Subclasses:

Hypothesis test based on the MCB stopping rule (Kim 2010) Terminates when it hits bounds or when the maximal samples have been hit. Estimates within approximately 0.001 at the bounds for alpha=0.95 or alpha=0.99.

Upper Bounds: r_1 = pValue*(maxSamples+1) c1*((m*p*(1-p))**0.5) + n*p

Lower Bounds: r_0 = (1-pValue)*(maxSamples+1)) c2*((m*p*(1-p))**0.5) + n*p

For clarity in code, c1 is referred to as upperBuffer and c2 is referred to as lowerBuffer. The default upper and lower buffer sizes are taken from Kim (2010)

Instance Methods [hide private]
 
__init__(self, sample1, sample2, testStatistic, alternative='two.sided', maxSamples=10000, pValue=0.05, upperBuffer=2.241, lowerBuffer=-2.241)
Initialize the estimator
str or None
_calculateBoundHit(self, value, tieCount, p, m, n, alternative)
Calculate which bound has been hit, if any
 
_startEstimate(self)
Start the Monte Carlo estimate
 
_updateBoundHit(self)
Update which bound has been hit, if any
 
addSamples(self, n, finalize=True)
Add a maximum of n samples to the estimate.
str
getBoundHit(self, update=True)
Get the boundary that has been crossed, if any.
int
getLowerBound(self, p, m, n)
Return the active lower bound for values
int
getUpperBound(self, p, m, n)
Return the active upper bound value
bool
isBoundHit(self, update=True)
Check if upper or lower bound has been hit.

Inherited from MCHypothesisTest: getSampleFunction, getStatisticFunction

Inherited from MonteCarloEstimator: getEstimate, getNumSamples, updateEstimate

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Class Variables [hide private]
  DEFAULT_LOWER_BUFFER = -2.241
  DEFAULT_P_VALUE = 0.05
  DEFAULT_UPPER_BUFFER = 2.241
  INNER_BOUND_NAME = 'Inner'
  LOWER_BOUND_NAME = 'Lower'
  UPPER_BOUND_NAME = 'Upper'

Inherited from MCHypothesisTest: VERBOSE

Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__init__(self, sample1, sample2, testStatistic, alternative='two.sided', maxSamples=10000, pValue=0.05, upperBuffer=2.241, lowerBuffer=-2.241)
(Constructor)

 

Initialize the estimator

Parameters:
  • sample1 (list of object) - First sample of data points
  • sample2 (list of object) - Second sample of data points
  • testStatistic (callable) - Statistic function to run on each sample, in the form f(sample): return statistic
  • alternative (str) - Alternate hypothesis, from the set GREATER_THAN_HYPOTHESIS, LESS_THAN_HYPOTHESIS, TWO_SIDED_HYPOTHESIS
  • maxSamples (int) - Maximum number of samples to run for an estimate
  • pValue (float) - P-value (alpha) for the test, in (0,1) range
  • upperBuffer (float) - Buffer width for upper bound (c1). Should be positive. Higher values require more positives to terminate early.
  • lowerBuffer (float) - Buffer width for the lower bound (c2). Should be negative. Higher absolute values require more negatives to terminate early.
Overrides: object.__init__

_calculateBoundHit(self, value, tieCount, p, m, n, alternative)

 

Calculate which bound has been hit, if any

Parameters:
  • value (int) - The number of "hits" exceeding the observed value
  • tieCount (int) - The number of ties equal to the observed value
  • p (float) - The p-value being tested, in [0, 1]
  • m (int) - Max samples
  • n (int) - Number of samples
  • alternative (str) - Alternate hypothesis
Returns: str or None
Bound hit

_startEstimate(self)

 

Start the Monte Carlo estimate

Overrides: MonteCarloEstimator._startEstimate

addSamples(self, n, finalize=True)

 

Add a maximum of n samples to the estimate. If any termination bound hit, stop sampling and return.

Parameters:
  • n (int) - Maximum number of samples to add
  • finalize (bool) - If True, finalize estimate value after adding samples
Overrides: MonteCarloEstimator.addSamples

getBoundHit(self, update=True)

 

Get the boundary that has been crossed, if any.

Parameters:
  • update (bool) - If True, update the stored value before returning
Returns: str
Return UPPER_BOUND_NAME if crossed the upper bound, return LOWER_BOUND_NAME if crossed the lower bound, return None if neither crossed

getLowerBound(self, p, m, n)

 

Return the active lower bound for values

Parameters:
  • p (float) - Probability value for the test, in [0,1]
  • m (int) - Maximum samples for the test
  • n - Number of samples so far in the test
  • n - int
Returns: int
Lower bound value

getUpperBound(self, p, m, n)

 

Return the active upper bound value

Parameters:
  • p (float) - Probability value for the test, in [0,1]
  • m (int) - Maximum samples for the test
  • n - Number of samples so far in the test
  • n - int
Returns: int
Upper bound value

isBoundHit(self, update=True)

 

Check if upper or lower bound has been hit.

Parameters:
  • update (bool) - If True, update the stored value before returning
Returns: bool
True if either bound hit, else False