Flume MorphlineSolrSink Configuration Options

The MorphlineSolrSink is a Flume sink used to ingest and index documents into Cloudera Search. For Cloudera Manager environments, the Flume agent is configured using procedures described in Configuring the Flume Agents. For unmanaged environments, you can use the standard configuration file flume.conf to configure Flume agents, including their sources, sinks, and channels. For more information about flume.conf, see the Flume User Guide.

Flume Morphline SolrSink supports the following configuration options (required options in bold):

Property Name Default Description
type   Must be set to the fully qualified class name (FQCN) org.apache.flume.sink.solr. morphline.MorphlineSolrSink.
channel   Specifies the channel to use for transferring records. For more information, see Flume Channels in the Flume User Guide.
morphlineFile   The location of the morphline configuration file.
  • For Cloudera Manager deployments, use: 
    <agentName>.sinks.<sinkName>.
morphlineFile=morphlines.conf
  • For unmanaged deployments, provide the relative or absolute path on the local filesystem to the morphline configuration file. For example, /etc/flume-ng/conf/ tutorialReadAvroContainer.conf
batchSize 100 The maximum number of Flume events per transaction. The transaction is committed when the specified batchSize or batchDurationMillis is reached, whichever comes first.
batchDurationMillis 1000 The maximum duration for a transaction, in milliseconds. The transaction is committed when the specified batchSize or batchDurationMillis is reached, whichever comes first.
indexerClass org.apache.flume.sink.solr. morphline.MorphlineSolrIndexer The FQCN of a class implementing org.apache.flume.sink.solr.morphline.SolrIndexer.
morphlineId null The name of the morphline to use when there is more than one morphline in a morphline configuration file.
This example shows a section for a MorphlineSolrSink named solrSink for an agent named agent: 
agent.sinks.solrSink.type = org.apache.flume.sink.solr.morphline.MorphlineSolrSink
agent.sinks.solrSink.channel = memoryChannel
agent.sinks.solrSink.batchSize = 100
agent.sinks.solrSink.batchDurationMillis = 1000
agent.sinks.solrSink.morphlineFile = /etc/flume-ng/conf/morphline.conf
agent.sinks.solrSink.morphlineId = morphline1