magpie.utility.tools

Class PhaseDiagramExclusionValidator

java.lang.Object

magpie.utility.tools.PhaseDiagramExclusionValidator

All Implemented Interfaces:

Commandable, Options, Printable, Savable

public class PhaseDiagramExclusionValidator
extends java.lang.Object
implements Options, Commandable, Savable, Printable

Validate a model by using data each phase diagram as test set.

Provided a number of elements, iterates through all possible combinations of that number elements (i.e., all unique phase diagrams) and evaluates the predictive accuracy of a model by excluding all elements that contain all of those elements from the dataset, training a model on the remaining entries, and validates the performance of the model on the remaining data.

Uses the PhaseDiagramExclusionFilter to create training and test sets.

Implemented Commands

results = evaluate $<model> $>test data> - Evaluate the performance of a model by iteratively using entries from different phase diagrams as the test test.
model: Model to be tested
test data: Data to use when testing the model
results: Output: Dataset containing results from test

Implemented Save Commands

human - Print out results for each phase diagram from last evaluation in a human-readable format.

Implemented Print Commands

last [<command>] - Print out statistics generated during last model evaluation
command: Command to be passed to internal BaseStatistics object.

nsystems - Print number of systems evaluated in last test

Usage: <# elements>
# elements: Number of elements in each phase diagram

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<org.apache.commons.lang3.tuple.Pair<int[],BaseStatistics>>	LastResults Results from previous test.
protected BaseStatistics	LastStatistics Statistics from last test
protected int	NElements Number of elements to use.

Constructor Summary

Constructors

Constructor and Description
PhaseDiagramExclusionValidator()

Method Summary

Modifier and Type	Method and Description
java.lang.String	about() Prints a simple status message about this object
CompositionDataset	evaluateModel(BaseModel model, CompositionDataset data) Evaluate predictive accuracy of model by iteratively excluding elements from a certain phase diagram.
protected static java.lang.String[]	getElementNames(int[] names) Provided with a list of elements IDs, print out element names.
static java.util.List<java.lang.Integer>	getElements(CompositionDataset data) Provided a CompositionDataset, get list of all included elements.
java.lang.String	printCommand(java.util.List<java.lang.String> Command) Handles more complicated printing commands.
java.lang.String	printDescription(boolean htmlFormat) Print full name of object, and a simple description of the options.
java.lang.String	printLastResults() Print last results as formatted text.
java.lang.String	printUsage() Print out required format for options.
java.lang.Object	runCommand(java.util.List<java.lang.Object> Command) Process some command described by a list of Objects.
static Dataset	runExclusionTest(int[] elemsToExclude, BaseModel modelToTest, CompositionDataset data) Evaluate model by using excluding certain combinations of elements from a dataset, and using the excluded data as a test set.
java.lang.String	saveCommand(java.lang.String Basename, java.lang.String Format) Handles complicated saving commands.
void	setNElements(int nElements) Set the number of elements in phase diagrams being iterated through
void	setOptions(java.util.List<java.lang.Object> Options) Set any options for this object.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NElements

protected int NElements

Number of elements to use. Default = 2

LastResults

protected final java.util.List<org.apache.commons.lang3.tuple.Pair<int[],BaseStatistics>> LastResults

Results from previous test. Key is the validation results, value is the performance when those elements were used as the test set

LastStatistics

protected BaseStatistics LastStatistics

Statistics from last test

Constructor Detail

PhaseDiagramExclusionValidator

public PhaseDiagramExclusionValidator()

Method Detail

setOptions

public void setOptions(java.util.List<java.lang.Object> Options)
                throws java.lang.Exception

Description copied from interface: Options

Set any options for this object. If parsing the options fails, return an Exception with the usage as its message.

Specified by:

setOptions in interface Options

Parameters:

Options - Array of options as Objects - can be null

Throws:

java.lang.Exception - if problem with inputs

printUsage

public java.lang.String printUsage()

Description copied from interface: Options

Print out required format for options. For consistency, should start with "Usage: ".

Specified by:

printUsage in interface Options

Returns:

Usage for this function

runCommand

public java.lang.Object runCommand(java.util.List<java.lang.Object> Command)
                            throws java.lang.Exception

Description copied from interface: Commandable

Process some command described by a list of Objects.

Specified by:

runCommand in interface Commandable

Parameters:

Command - Command as a list of objects

Returns:

Any output (null if no output generated by command)

Throws:

java.lang.Exception - If something goes wrong

setNElements

public void setNElements(int nElements)

Set the number of elements in phase diagrams being iterated through

Parameters:

nElements - Desired size of phase diagram

evaluateModel

public CompositionDataset evaluateModel(BaseModel model,
                                        CompositionDataset data)

Evaluate predictive accuracy of model by iteratively excluding elements from a certain phase diagram.

Parameters:

model - Model to be validated

data - Data to use for validation

Returns:

Dataset containing properties predicted during validation

printLastResults

public java.lang.String printLastResults()

Print last results as formatted text. Style

Returns:

Last results in a human-readable format

saveCommand

public java.lang.String saveCommand(java.lang.String Basename,
                                    java.lang.String Format)
                             throws java.lang.Exception

Description copied from interface: Savable

Handles complicated saving commands.

Dev Note: Make sure to add save format to Javadoc. See Dataset as an example. Required format:

<save><p><b>format<b> - Description
<br>Optional room to talk more about format </save>

Specified by:

saveCommand in interface Savable

Parameters:

Basename - Name of file without extension

Format - Command specifying format in which to print

Returns:

Filename Path to output file

Throws:

java.lang.Exception - If command not understood

about

public java.lang.String about()

Description copied from interface: Printable

Prints a simple status message about this object

Specified by:

about in interface Printable

Returns:

Short status message

printCommand

public java.lang.String printCommand(java.util.List<java.lang.String> Command)
                              throws java.lang.Exception

Description copied from interface: Printable

Handles more complicated printing commands.

Specified by:

printCommand in interface Printable

Parameters:

Command - Command specifying what to print

Returns:

String formatted as requested

Throws:

java.lang.Exception - If command not understood

printDescription

public java.lang.String printDescription(boolean htmlFormat)

Description copied from interface: Printable

Print full name of object, and a simple description of the options. This description should print enough information to reproduce how this object was created, but not necessarily enough to run it (e.g., coefficients for the model).

Example: For a model training a separate WekaRegression for intermetallics

magpie.models.regression.SplitRegression

Splitter: AllMetalsSplitter
All Metals: magpie.models.regression.WekaRegression trees.REPTree
Contains Nonmetal: magpie.regression.LASSORegreession -maxterms 2

Specified by:

printDescription in interface Printable

Parameters:

htmlFormat - Whether format for output to an HTML page (e.g., <div> to create indentation) or for printing to screen.

Returns:

String describing the model

See Also:

#printModel()

getElementNames

protected static java.lang.String[] getElementNames(int[] names)

Provided with a list of elements IDs, print out element names.

Parameters:

names - Element IDs

Returns:

Element names

runExclusionTest

public static Dataset runExclusionTest(int[] elemsToExclude,
                                       BaseModel modelToTest,
                                       CompositionDataset data)

Evaluate model by using excluding certain combinations of elements from a dataset, and using the excluded data as a test set.

Parameters:

elemsToExclude - List of elements (by ID = Z - 1) defining

modelToTest - Model to be bested

data - Data to use as test data

Returns:

Predictions from test sets

getElements

public static java.util.List<java.lang.Integer> getElements(CompositionDataset data)

Provided a CompositionDataset, get list of all included elements.

Parameters:

data - Dataset to be analyzed

Returns:

List of all elements in dataset