BaseStatistics (magpie API)

java.lang.Object
- magpie.statistics.performance.BaseStatistics

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Commandable, Options, Printable, Savable

Direct Known Subclasses:

ClassificationStatistics, RegressionStatistics
```
public abstract class BaseStatistics
extends java.lang.Object
implements java.io.Serializable, java.lang.Cloneable, Printable, Options, Commandable, Savable
```
This class stores basic statistics about model performance and contains methods to calculate each of these statistics.
Implementations of this case need to supply evaluate(magpie.data.Dataset), which actually performs the statistical calculations. Make sure to store the measurements and predicted class variables! It is also up to the implementation to provide some way of storing results and printing them using the toString() operation.
Usage: *No options to set*
Implemented Commands:
evaluate $<dataset> - Evaluate measured vs. predicted class of entries
dataset: Dataset object to be evaluated.
Implemented Print Commands:
stats - Print out all statistics
roc - Print out Receiver Operating Characteristic curve
baseline - Print statistics about the training data
Implemented Save Commands
data - Save predicted and measured class values used to compute statistics

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected double[]`	`Measured` Measured value of class variable
`int`	`NumberTested` Number of entries evaluated
`protected double[]`	`Predicted` Predicted value of class variable
`double[][]`	`ROC` Receiver operating characteristic curve
`double`	`ROC_AUC` Area under receiver operating characteristic curve normalized such that 1.0 is a perfect classifier and 0.0 is a perfectly-random classifier.

Constructor Summary

Constructors
Constructor and Description

BaseStatistics()

Constructors
Constructor and Description
`BaseStatistics()`

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`java.lang.String`	`about()` Prints a simple status message about this object
`java.lang.Object`	`clone()`
`protected abstract void`	`evaluate_protected(Dataset results)` Internal method for acutally computing results.
`void`	`evaluate(Dataset results)` Generates statistics about the performance on a model.
`protected void`	`getROCCurve(int[] measured, double[] score, int maxSteps)` Generate the receiver operating characteristic curve based on the measured and score variables for many instances.
`abstract java.util.Map<java.lang.String,java.lang.Double>`	`getStatistics()` Retrieve list of statistics stored in this class.
`java.util.Map<java.lang.String,java.lang.Double>`	`getStatisticsNoNaNs()` Get statistics, except for values that are NaNs or infinite
`abstract java.lang.String`	`printBaselineStats()` Print statistics about the measured class values.
`java.lang.String`	`printCommand(java.util.List<java.lang.String> Command)` Handles more complicated printing commands.
`java.lang.String`	`printDescription(boolean htmlFormat)` Print full name of object, and a simple description of the options.
`java.lang.String`	`printROCCurve()` Print the ROC Curve as a plottable string.
`java.lang.String`	`printUsage()` Print out required format for options.
`java.lang.Object`	`runCommand(java.util.List<java.lang.Object> Command)` Process some command described by a list of Objects.
`java.lang.String`	`saveCommand(java.lang.String Basename, java.lang.String Format)` Handles complicated saving commands.
`void`	`savePerformanceData(java.lang.String filename)` Write out measured and predicted class variables used to compute statistics.
`void`	`setOptions(java.util.List<java.lang.Object> Options)` Set any options for this object.

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - NumberTested
```
public int NumberTested
```
    Number of entries evaluated
  - ROC
```
public double[][] ROC
```
    Receiver operating characteristic curve
  - ROC_AUC
```
public double ROC_AUC
```
    Area under receiver operating characteristic curve normalized such that 1.0 is a perfect classifier and 0.0 is a perfectly-random classifier.
  - Measured
```
protected double[] Measured
```
    Measured value of class variable
  - Predicted
```
protected double[] Predicted
```
    Predicted value of class variable
- Constructor Detail
  - BaseStatistics
```
public BaseStatistics()
```
- Method Detail
  - setOptions
```
public void setOptions(java.util.List<java.lang.Object> Options)
                throws java.lang.Exception
```
    Description copied from interface: Options
    
    Set any options for this object. If parsing the options fails, return an Exception with the usage as its message.
    
    Specified by:
    
    setOptions in interface Options
    
    Parameters:
    
    Options - Array of options as Objects - can be null
    
    Throws:
    
    java.lang.Exception - if problem with inputs
  - printUsage
```
public java.lang.String printUsage()
```
    Description copied from interface: Options
    
    Print out required format for options. For consistency, should start with "Usage: ".
    
    Specified by:
    
    printUsage in interface Options
    
    Returns:
    
    Usage for this function
  - clone
```
public java.lang.Object clone()
                       throws java.lang.CloneNotSupportedException
```
    Overrides:
    
    clone in class java.lang.Object
    
    Throws:
    
    java.lang.CloneNotSupportedException
  - evaluate
```
public final void evaluate(Dataset results)
```
    Generates statistics about the performance on a model.
    
    Parameters:
    
    results - Dataset containing both measured and predicted classes.
  - evaluate_protected
```
protected abstract void evaluate_protected(Dataset results)
```
    Internal method for acutally computing results. evaluate(Dataset) contains some error-checking code
    
    Parameters:
    
    results - Dataset to be evaluated
  - getROCCurve
```
protected void getROCCurve(int[] measured,
                           double[] score,
                           int maxSteps)
```
    Generate the receiver operating characteristic curve based on the measured and score variables for many instances. Computes the ROC curve using the False Positive Rate for the x-axis and True Positive Rate for the y-axis. Assumes that the positive label is 0. Stores the result in ROC
    
    Parameters:
    
    measured - Measured class variable
    
    score - Classification score. Higher score means 'more likely to be positive class'
    
    maxSteps - Number of steps in curve
  - printROCCurve
```
public java.lang.String printROCCurve()
```
    Print the ROC Curve as a plottable string. Format for each row:
    
    [Cutoff] [False Positive Rate] [Sensitivity]
    
    Returns:
    
    ROC curve for model.
  - about
```
public java.lang.String about()
```
    Description copied from interface: Printable
    
    Prints a simple status message about this object
    
    Specified by:
    
    about in interface Printable
    
    Returns:
    
    Short status message
  - printDescription
```
public java.lang.String printDescription(boolean htmlFormat)
```
    Description copied from interface: Printable
    
    Print full name of object, and a simple description of the options. This description should print enough information to reproduce how this object was created, but not necessarily enough to run it (e.g., coefficients for the model).
    Example: For a model training a separate WekaRegression for intermetallics
    magpie.models.regression.SplitRegression
    Splitter: AllMetalsSplitter
    All Metals: magpie.models.regression.WekaRegression trees.REPTree
    Contains Nonmetal: magpie.regression.LASSORegreession -maxterms 2
    
    Specified by:
    
    printDescription in interface Printable
    
    Parameters:
    
    htmlFormat - Whether format for output to an HTML page (e.g., <div> to create indentation) or for printing to screen.
    
    Returns:
    
    String describing the model
    
    See Also:
    
    #printModel()
  - printBaselineStats
```
public abstract java.lang.String printBaselineStats()
```
    Print statistics about the measured class values. Useful as context for interpreting how good the performance statistics actually are
    
    Returns:
    
    String listing all of the measured baseline statistics
  - printCommand
```
public java.lang.String printCommand(java.util.List<java.lang.String> Command)
                              throws java.lang.Exception
```
    Description copied from interface: Printable
    
    Handles more complicated printing commands.
    
    Specified by:
    
    printCommand in interface Printable
    
    Parameters:
    
    Command - Command specifying what to print
    
    Returns:
    
    String formatted as requested
    
    Throws:
    
    java.lang.Exception - If command not understood
  - runCommand
```
public java.lang.Object runCommand(java.util.List<java.lang.Object> Command)
                            throws java.lang.Exception
```
    Description copied from interface: Commandable
    
    Process some command described by a list of Objects.
    
    Specified by:
    
    runCommand in interface Commandable
    
    Parameters:
    
    Command - Command as a list of objects
    
    Returns:
    
    Any output (null if no output generated by command)
    
    Throws:
    
    java.lang.Exception - If something goes wrong
  - saveCommand
```
public java.lang.String saveCommand(java.lang.String Basename,
                                    java.lang.String Format)
                             throws java.lang.Exception
```
    Description copied from interface: Savable
    
    Handles complicated saving commands.
    Dev Note: Make sure to add save format to Javadoc. See Dataset as an example. Required format:
    <save><p><b>format<b> - Description
    <br>Optional room to talk more about format </save>
    
    Specified by:
    
    saveCommand in interface Savable
    
    Parameters:
    
    Basename - Name of file without extension
    
    Format - Command specifying format in which to print
    
    Returns:
    
    Filename Path to output file
    
    Throws:
    
    java.lang.Exception - If command not understood
  - savePerformanceData
```
public void savePerformanceData(java.lang.String filename)
                         throws java.lang.Exception
```
    Write out measured and predicted class variables used to compute statistics. Prints into csv format.
    
    Parameters:
    
    filename - Name of output file
    
    Throws:
    
    java.lang.Exception
  - getStatistics
```
public abstract java.util.Map<java.lang.String,java.lang.Double> getStatistics()
```
    Retrieve list of statistics stored in this class.
    
    Returns:
    
    Map of statistic name to value.
  - getStatisticsNoNaNs
```
public java.util.Map<java.lang.String,java.lang.Double> getStatisticsNoNaNs()
```
    Get statistics, except for values that are NaNs or infinite
    
    Returns:

Class BaseStatistics

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

NumberTested

ROC

ROC_AUC

Measured

Predicted

Constructor Detail

BaseStatistics

Method Detail

setOptions

printUsage

clone

evaluate

evaluate_protected

getROCCurve

printROCCurve

about

printDescription

printBaselineStats

printCommand

runCommand

saveCommand

savePerformanceData

getStatistics

getStatisticsNoNaNs