com.raritantechnologies.verity.collection
Class FileSystemGateway

java.lang.Object
  com.raritantechnologies.verity.collection.FileSystemGateway

All Implemented Interfaces:

ICollectionGateway, java.lang.Runnable

public class FileSystemGateway

extends java.lang.Object

implements ICollectionGateway

Collection Gateway is responsible for collecting files. For XML files a XMLResultProcessor can be used to convert XML files into XMLResultSets.

XML Configuration Template:

  <CollectionGateway class="com.raritantechnologies.verity.collection.FileSystemGateway"
                        filePath="[ path to data files ]"
                        outputPath="[ (optional) directory to move processed data files to ]"
                        recurseSubdirectories="[ true|false(default) ]"
                        docTextField="[ result field to contain document text ]"
                        xmlDocs="[ true(default)|false ]"
                        filenameField="[ result field to contain filename ]"
                        startAt="[ file number to start with (0 by default) ]"
                        endAt="[ file number to end with (last file by default) ]"
                        multipleRecordsPerFile="true|false(default) ]"
                        recordTag="[ name of record tag (if multiple records per file is true ]"
                        startSegmentAt="[ optional string to start sub-file segment ]"
                        endSegmentAt="[ string to end sub-file segment ]"
                        resultSetSize="[ number of results in each result set ]" >

    <!-- Optional StringFilter applied to string before it is sent to XMLResultProcessor --> 
    <StringFilter class="[class of com.raritantechnologies.utils.filter.IStringFilter]" >

    </StringFilter>

    <!-- Optional FileName Comparator -->
    <FileNameComparator class="[ class of com.raritantechnologies.utils.comparators.IComparator ]" >

    </FileNameComparator>

    <!-- (Optional) One or more IStreamFilter's to handle non-text files -->
    <StreamFilter class="[ class of com.raritantechnologies.utils.filter.IStreamFilter ]"
                     extension="[ file extension to use this filter with ]" >

    </StreamFilter>

    <!-- Common Properties Map (for multiple records per file) -->
    <!-- Maps a property name with an XPath within the XML file -->
    <CommonFields>
      <Field ID="[ id of field ]" xPath="[ xpath within the XML file ]" />
      <!-- etc. . . -->
    </CommonFields>

    <XMLResultProcessor>
      <!-- configuration parameters for XMLResultProcessor -->
    </XMLResultProcessor>

    <!-- Mapping to FlatFileGateway(s) by fileName or file extension -->

  </CollectionGateway>
 

Developed by Raritan Technologies Inc..

Author:

Ted Sullivan

Constructor Summary

FileSystemGateway()

FileSystemGateway(java.lang.String rootPath, boolean recurseSubdirectories)

Method Summary

java.lang.String	getConfigurationXML()
IGatewayListener	getGatewayListener()
boolean	getRecurseSubdirectories()
int	getResultSetSize()
java.lang.String	getRootDirectory()
XMLResultProcessor	getXMLProcessor()
void	initialize(org.w3c.dom.Element gatewayElem, ISearchFieldMap sfMap) Initialize the ICollectionGateway from a Configuration XML element.
boolean	isPaused()
void	pause()
protected BasicResultSet	processResult(BasicResultSet resSet, java.lang.String text, java.lang.String fileName)
void	resume()
void	run()
void	run(ILoginInfo userInfo) Need to have loginInfo for sources which require login/pw access.
void	setDataAvailable(IResultSet rs)
void	setDataComplete()
void	setGatewayListener(IGatewayListener listener) Sets the IGatewayListener interface.
void	setInputParameters(OrderedMap inputParams) set input parameters...
void	setRecurseSubdirectories(boolean recurseSubdirectories)
void	setResultSetSize(int resultSetSize)
void	setRootDirectory(java.lang.String rootDirectory)
void	setXMLProcessor(XMLResultProcessor xmlProcessor)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileSystemGateway

public FileSystemGateway()

FileSystemGateway

public FileSystemGateway(java.lang.String rootPath,
                         boolean recurseSubdirectories)

Method Detail

initialize

public void initialize(org.w3c.dom.Element gatewayElem,
                       ISearchFieldMap sfMap)

Description copied from interface: ICollectionGateway

Initialize the ICollectionGateway from a Configuration XML element.

Specified by:

initialize in interface ICollectionGateway

setInputParameters

public void setInputParameters(OrderedMap inputParams)

Description copied from interface: ICollectionGateway

set input parameters...

Specified by:

setInputParameters in interface ICollectionGateway

run

public void run()

Specified by:

run in interface ICollectionGateway

processResult

protected BasicResultSet processResult(BasicResultSet resSet,
                                       java.lang.String text,
                                       java.lang.String fileName)

run

public void run(ILoginInfo userInfo)

Need to have loginInfo for sources which require login/pw access.

Specified by:

run in interface ICollectionGateway

setGatewayListener

public void setGatewayListener(IGatewayListener listener)

Description copied from interface: ICollectionGateway

Sets the IGatewayListener interface.

Specified by:

setGatewayListener in interface ICollectionGateway

setDataAvailable

public void setDataAvailable(IResultSet rs)

setDataComplete

public void setDataComplete()

pause

public void pause()

Specified by:

pause in interface ICollectionGateway

resume

public void resume()

Specified by:

resume in interface ICollectionGateway

isPaused

public boolean isPaused()

Specified by:

isPaused in interface ICollectionGateway

getGatewayListener

public IGatewayListener getGatewayListener()

setRootDirectory

public void setRootDirectory(java.lang.String rootDirectory)

getRootDirectory

public java.lang.String getRootDirectory()

setRecurseSubdirectories

public void setRecurseSubdirectories(boolean recurseSubdirectories)

getRecurseSubdirectories

public boolean getRecurseSubdirectories()

setXMLProcessor

public void setXMLProcessor(XMLResultProcessor xmlProcessor)

getXMLProcessor

public XMLResultProcessor getXMLProcessor()

setResultSetSize

public void setResultSetSize(int resultSetSize)

getResultSetSize

public int getResultSetSize()

getConfigurationXML

public java.lang.String getConfigurationXML()

Specified by:

getConfigurationXML in interface ICollectionGateway