magpie.utility.tools

Class BatchModelEvaluator

java.lang.Object

magpie.utility.tools.BatchModelEvaluator

All Implemented Interfaces:

Commandable, Options

public class BatchModelEvaluator
extends java.lang.Object
implements Commandable, Options

Generate attributes and simpleRun models in batches. Useful if a dataset contains too many entries to store all of their attributes in memory at one time.

Usage: $<model> <batch size>
model: Model to be evaluated
batch size: Number of entries to evaluate at one time

Implemented Commands

run $<data> - Evaluate a model in batch model
data: Dataset to be evaluated

add $<data> $<generator> [<include|exclude> $<filter>] - Generate entries, run them using a model, and add them to a dataset if they pass a filter
data: Dataset to be added to
generator: BaseEntryGenerator used to generate entries
include|exclude: Optional: Whether to include or exclude data based on filter
filter: BaseDatasetFilter used to determine whether entries should be added to data

Constructor Summary

Constructors

Constructor and Description
BatchModelEvaluator()

Method Summary

Modifier and Type	Method and Description
void	evaluate(Dataset data) Evaluate the model on a dataset in batch mode.
void	evaluate(Dataset data, BaseEntryGenerator generator, BaseDatasetFilter filter) Generate entries and store them in a dataset.
java.util.List<BaseEntry>	getSubList() Get list of entries to be evaluated
java.lang.String	printUsage() Print out required format for options.
protected void	processEntries(java.util.List<BaseEntry> entries, java.util.Map<java.lang.String,java.lang.Object> threadData) Operation to be performed on a batch of entries
protected java.util.List<BaseEntry>	runAndFilter(BaseModel model, Dataset data, BaseDatasetFilter filter) Create entries, evaluate them, filter out acceptable candidates
java.lang.Object	runCommand(java.util.List<java.lang.Object> Command) Process some command described by a list of Objects.
protected void	setAttributeComputer(Dataset dataset) Set dataset used to generate attributes
void	setBatchSize(int size) Define how many entries to evaluate at once.
protected void	setDataset(Dataset data) Set dataset to be evaluated.
protected void	setEntryIterator(java.util.Iterator<BaseEntry> iterator) Set iterator over entries to be acted on
void	setModel(BaseModel model) Define which model this class is used to evaluate
void	setOptions(java.util.List<java.lang.Object> Options) Set any options for this object.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BatchModelEvaluator

public BatchModelEvaluator()

Method Detail

setOptions

public void setOptions(java.util.List<java.lang.Object> Options)
                throws java.lang.Exception

Description copied from interface: Options

Set any options for this object. If parsing the options fails, return an Exception with the usage as its message.

Specified by:

setOptions in interface Options

Parameters:

Options - Array of options as Objects - can be null

Throws:

java.lang.Exception - if problem with inputs

printUsage

public java.lang.String printUsage()

Description copied from interface: Options

Print out required format for options. For consistency, should start with "Usage: ".

Specified by:

printUsage in interface Options

Returns:

Usage for this function

setModel

public void setModel(BaseModel model)

Define which model this class is used to evaluate

Parameters:

model - Model to be evaluated

setBatchSize

public void setBatchSize(int size)

Define how many entries to evaluate at once.

Parameters:

size - Desired batch size

setAttributeComputer

protected void setAttributeComputer(Dataset dataset)

Set dataset used to generate attributes

Parameters:

dataset - Dataset used to generate attributes

setEntryIterator

protected void setEntryIterator(java.util.Iterator<BaseEntry> iterator)

Set iterator over entries to be acted on

Parameters:

iterator - Iterator over entries being acted on

setDataset

protected void setDataset(Dataset data)

Set dataset to be evaluated. Sets both the attribute generator and the entry iterator

Parameters:

data - Dataset to be evaluated

evaluate

public void evaluate(Dataset data,
                     BaseEntryGenerator generator,
                     BaseDatasetFilter filter)
              throws java.lang.Exception

Generate entries and store them in a dataset. Optionally, filter data before adding it to dataset

Parameters:

data - Dataset to be added to

generator - Generator for creating entries

filter - Filter to be used before adding entries to data

Throws:

java.lang.Exception

evaluate

public void evaluate(Dataset data)
              throws java.lang.Exception

Evaluate the model on a dataset in batch mode.

Parameters:

data - Dataset to be evaluated

Throws:

java.lang.Exception

processEntries

protected void processEntries(java.util.List<BaseEntry> entries,
                              java.util.Map<java.lang.String,java.lang.Object> threadData)
                       throws java.lang.Exception

Operation to be performed on a batch of entries

Parameters:

entries - Entries to be processed

threadData - Copy of objects belonging to this thread

Throws:

java.lang.Exception

runAndFilter

protected java.util.List<BaseEntry> runAndFilter(BaseModel model,
                                                 Dataset data,
                                                 BaseDatasetFilter filter)
                                          throws java.lang.Exception

Create entries, evaluate them, filter out acceptable candidates

Parameters:

model - Model to be run

data - Empty dataset used to compute attributes

filter - Filter to be used to select best attributes. null to not filter results. Can be null

Returns:

Entries that passed the filter

Throws:

java.lang.Exception

getSubList

public java.util.List<BaseEntry> getSubList()

Get list of entries to be evaluated

Returns:

List of entries. If none remaining, list will be empty

runCommand

public java.lang.Object runCommand(java.util.List<java.lang.Object> Command)
                            throws java.lang.Exception

Description copied from interface: Commandable

Process some command described by a list of Objects.

Specified by:

runCommand in interface Commandable

Parameters:

Command - Command as a list of objects

Returns:

Any output (null if no output generated by command)

Throws:

java.lang.Exception - If something goes wrong