BatchAttributeGenerator (magpie API)

java.lang.Object
- magpie.utility.tools.BatchAttributeGenerator

All Implemented Interfaces:

Commandable, Options
```
public class BatchAttributeGenerator
extends java.lang.Object
implements Commandable, Options
```
Compute attributes and save them to a file in batches. This prevents needing to store all attributes in memory at a single time. Also supports computing attributes in parallel.
Note: Order of entries is not guaranteed to be preserved for parallel evaluation.
How to Use This Class
There are two major use cases for this class
1. Entries already stored in Dataset:
  - Java Interface: Call output(magpie.data.Dataset, java.io.File) or output(magpie.data.Dataset, java.io.OutputStream) from the to generate entries at write them to file
  - Text Interface: write <filename> $<data>, where "data" is a Dataset
2. Entries generated iteratively: Dynamically generate entries, and compute attributes. Entries will not be stored
  - Java Interface: Call output(magpie.data.Dataset, java.util.Iterator, java.io.OutputStream)
  - Text Interface: write <filename> $<data> $<generator>, where "data" is a template for computing attributes,and "generator" is an
Usage: <batch size> <output method> [<output options...>]
batch size: Number of entries to compute at once
output method: Name of BaseDatasetOutput method used for output
output options: Any options for the output method

Field Summary

Fields
Modifier and Type	Field and Description
`protected Dataset`	`AttributeComputer` Dataset used to generate attributes
`protected int`	`BatchSize` Number of entries to evaluate at a single time
`protected BaseDatasetOutput`	`Outputter` Tool used to output data

Constructor Summary

Constructors
Constructor and Description
`BatchAttributeGenerator()` Create an batch writer using the default settings: comma-delimited file, batch size of 10000 entries
`BatchAttributeGenerator(BaseDatasetOutput outputter, int batchSize)` Create a batch evaluator with a specified output method and batch size

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected java.util.List<BaseEntry>`	`getBatch()` Get a batch of entries.
`void`	`output(Dataset data, java.io.File file)` Compute attributes and write entries to a file
`void`	`output(Dataset template, java.util.Iterator<BaseEntry> iterator, java.io.OutputStream out)` Compute attributes and write entries to an output stream in batches
`void`	`output(Dataset data, java.io.OutputStream out)` Compute attributes and write entries to an output stream in batches
`java.lang.String`	`printUsage()` Print out required format for options.
`java.lang.Object`	`runCommand(java.util.List<java.lang.Object> Command)` Process some command described by a list of Objects.
`void`	`setBatchSize(int size)` Define maximum number of entries to compute at a single time
`protected void`	`setEntryIterator(java.util.Iterator<BaseEntry> iterator)` Set the iterator over entries being processed
`void`	`setOptions(java.util.List<java.lang.Object> Options)` Set any options for this object.
`void`	`setOutputter(BaseDatasetOutput output)` Define tool used to output data
`protected void`	`writeEntries(Dataset toWrite, java.io.OutputStream output)` Write entries to output stream

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - Outputter
```
protected BaseDatasetOutput Outputter
```
    Tool used to output data
  - BatchSize
```
protected int BatchSize
```
    Number of entries to evaluate at a single time
  - AttributeComputer
```
protected Dataset AttributeComputer
```
    Dataset used to generate attributes
- Constructor Detail
  - BatchAttributeGenerator
```
public BatchAttributeGenerator()
```
    Create an batch writer using the default settings: comma-delimited file, batch size of 10000 entries
  - BatchAttributeGenerator
```
public BatchAttributeGenerator(BaseDatasetOutput outputter,
                               int batchSize)
```
    Create a batch evaluator with a specified output method and batch size
    
    Parameters:
    
    outputter - Output method (default: DelimitedOutput)
    
    batchSize - Batch size (default: 10000)
- Method Detail
  - setOptions
```
public void setOptions(java.util.List<java.lang.Object> Options)
                throws java.lang.Exception
```
    Description copied from interface: Options
    
    Set any options for this object. If parsing the options fails, return an Exception with the usage as its message.
    
    Specified by:
    
    setOptions in interface Options
    
    Parameters:
    
    Options - Array of options as Objects - can be null
    
    Throws:
    
    java.lang.Exception - if problem with inputs
  - printUsage
```
public java.lang.String printUsage()
```
    Description copied from interface: Options
    
    Print out required format for options. For consistency, should start with "Usage: ".
    
    Specified by:
    
    printUsage in interface Options
    
    Returns:
    
    Usage for this function
  - setOutputter
```
public void setOutputter(BaseDatasetOutput output)
```
    Define tool used to output data
    
    Parameters:
    
    output - Output method
  - setBatchSize
```
public void setBatchSize(int size)
```
    Define maximum number of entries to compute at a single time
    
    Parameters:
    
    size - Desired batch size
  - setEntryIterator
```
protected void setEntryIterator(java.util.Iterator<BaseEntry> iterator)
```
    Set the iterator over entries being processed
    
    Parameters:
    
    iterator - Iterator
  - output
```
public void output(Dataset data,
                   java.io.File file)
            throws java.io.IOException
```
    Compute attributes and write entries to a file
    
    Parameters:
    
    data - Dataset to be written
    
    file - Path to output file
    
    Throws:
    
    java.io.IOException
  - output
```
public void output(Dataset data,
                   java.io.OutputStream out)
```
    Compute attributes and write entries to an output stream in batches
    
    Parameters:
    
    data - Dataset to be written
    
    out - Output stream for data
  - output
```
public void output(Dataset template,
                   java.util.Iterator<BaseEntry> iterator,
                   java.io.OutputStream out)
```
    Compute attributes and write entries to an output stream in batches
    
    Parameters:
    
    template - Dataset used to generate attributes
    
    iterator - Iterator over entries to be printed
    
    out - Output stream for data
  - getBatch
```
protected java.util.List<BaseEntry> getBatch()
```
    Get a batch of entries.
    
    Returns:
    
    List containing up to BatchSize entries. May be empty
  - writeEntries
```
protected void writeEntries(Dataset toWrite,
                            java.io.OutputStream output)
```
    Write entries to output stream
    
    Parameters:
    
    toWrite - Entries to be written
    
    output - Desired output stream
  - runCommand
```
public java.lang.Object runCommand(java.util.List<java.lang.Object> Command)
                            throws java.lang.Exception
```
    Description copied from interface: Commandable
    
    Process some command described by a list of Objects.
    
    Specified by:
    
    runCommand in interface Commandable
    
    Parameters:
    
    Command - Command as a list of objects
    
    Returns:
    
    Any output (null if no output generated by command)
    
    Throws:
    
    java.lang.Exception - If something goes wrong

Class BatchAttributeGenerator

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

Outputter

BatchSize

AttributeComputer

Constructor Detail

BatchAttributeGenerator

BatchAttributeGenerator

Method Detail

setOptions

printUsage

setOutputter

setBatchSize

setEntryIterator

output

output

output

getBatch

writeEntries

runCommand