AbstractJDBCOutput (Punch Storm Spouts and Bolts 6.4.4 API)

java.lang.Object
- org.thales.punch.libraries.storm.api.BaseProcessingNode
- - org.thales.punch.libraries.storm.bolt.AbstractJDBCOutput

All Implemented Interfaces:

Serializable, org.apache.storm.task.IBolt, org.apache.storm.topology.IComponent, org.apache.storm.topology.IRichBolt

Direct Known Subclasses:

ClickHouseOutput, OmniSciOutput, PostgresqlOutput
```
public abstract class AbstractJDBCOutput
extends org.thales.punch.libraries.storm.api.BaseProcessingNode
```
AbstractJDBCBolt provides common JDBC configuration (host, username, port, etc...).
This class uses the storm jdbc connector and Hikari Jdbc Connection Pooling.
Connection to Jdbc database (refer to as datasource in HikariCP terminology) are stored in memory as long as the process is running.
This connection is shared amongst the topology process.
A specific number of Jdbc connection can be maintained alive. Please, refer to HikariCP official documentation for more in depth information.
Here is a list of additional Jdbc bolt that can be implemented easily: https://github.com/brettwooldridge/HikariCP, refer to datasources section.
Grab the datasource class accordingly to the Jdbc on which you want to interact.
```
 
        HirakiConfigMap.put("dataSourceClassName", "your_db_source_class");
 
 
```
On the other hand, some Jdbc connector are not listed by HirakiCP (no official datasource class), for instance Omnisci.
You will have to construct your own Jdbc String connector. Each url string connector may have their own convention.
Please, refer to the official documentation of the Jdbc you want to work on.
```
 
        HirakiConfigMap.put("dataSourceClassName", your_constructed_url_string);
 
 
```
For implementing a new Jdbc Insert Bolt, you should create a new class extending the AbstractJDBCBolt class.
Override the methods: execute, cleanup and prepare, if needed.
Don't forget to update your pom.xml by adding the official Jdbc connector you want to add
See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected List<org.apache.storm.tuple.Tuple>`	`ackFail`
`protected int`	`bulkSize`
`static String`	`COLUMN_NAME_WRAPPER_CHAR_SETTING` column_name_wrapper_char: This optional setting provides a DB-specific character that will be used around column names to allow for special characters in column names, like @
`static String`	`COLUMN_NAMES_SETTING` Settings
`static String`	`COLUMN_TYPE_CAST`
`static String`	`COLUMN_TYPE_FUNCTION`
`protected List<String[]>`	`columnNames`
`protected String`	`columnNameWrapper`
`protected String`	`columnsNamesSeparator`
`static String`	`CONVERT_BOOLEAN_TO_INT_SETTING` convert_boolean_to_int: if true, then the boolean value will be converted to 0 or 1 integer value at insert
`protected Boolean`	`convertBooleanToInt`
`protected String`	`database`
`protected long`	`expirationTimeout`
`protected List<Map<String,Object>>`	`hikariConfigMaps`
`protected List<String>`	`hosts`
`protected org.thales.punch.libraries.utils.api.RoundRobinIterator<JdbcHandle>`	`jdbcHandleIterator`
`protected long`	`lastUpdateTimestamp`
`protected String`	`password`
`protected String`	`poolName`
`protected int`	`poolSize`
`protected int`	`queryQueueTime`
`protected long`	`reconnectionTimeout`
`static String`	`SOURCE_TUPLE_FIELD_SETTING` source_tuple_field: This optional setting provides the name of an input tuple field that is expected to contain a map-like object, providing all fields, instead of having many fields in the storm stream.
`protected String`	`sourceTupleField`
`protected String`	`table`
`protected String`	`username`

Fields inherited from class org.thales.punch.libraries.storm.api.BaseProcessingNode
ackRate, collector, componentId, every, failRate, metricContext, nodeSettings, topologyContext, traversalTime

Constructor Summary

Constructors
Modifier Constructor and Description

protected AbstractJDBCOutput(org.thales.punch.libraries.storm.api.NodeSettings config, String backend)

Constructors
Modifier	Constructor and Description
`protected`	`AbstractJDBCOutput(org.thales.punch.libraries.storm.api.NodeSettings config, String backend)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected void`	`addTupleToQuery(org.apache.storm.tuple.Tuple input)` Add a tuple to INSERT query.
`void`	`cleanup()`
`protected void`	`failAll()`
`protected Object`	`formatColumnForAutoCast(Object value)`
`protected Object`	`formatColumnForFunction(Object value)`
`Map<String,Object>`	`getComponentConfiguration()`
`protected String`	`getDefaultColumnNameWrapperChar()`
`protected Boolean`	`getDefaultConvertBooleanToInt()`
`void`	`prepare(Map stormConf, org.apache.storm.task.TopologyContext context, org.apache.storm.task.OutputCollector collector)`
`void`	`process(org.apache.storm.tuple.Tuple input)`

Methods inherited from class org.thales.punch.libraries.storm.api.BaseProcessingNode
ack, declareOutputFields, enrichAndForwardMonitoringTuple, execute, fail, getNewPoint

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - COLUMN_NAMES_SETTING
```
public static final String COLUMN_NAMES_SETTING
```
    Settings
    column_names: This optional setting specifies a list of : where classifier is one of 'cast' (direct value) or 'function' If the setting is not provided, the first tuple is used to guess the columns, all with 'cast' classifier
    
    See Also:
    
    Constant Field Values
  - COLUMN_NAME_WRAPPER_CHAR_SETTING
```
public static final String COLUMN_NAME_WRAPPER_CHAR_SETTING
```
    column_name_wrapper_char: This optional setting provides a DB-specific character that will be used around column names to allow for special characters in column names, like @
    
    See Also:
    
    Constant Field Values
  - SOURCE_TUPLE_FIELD_SETTING
```
public static final String SOURCE_TUPLE_FIELD_SETTING
```
    source_tuple_field: This optional setting provides the name of an input tuple field that is expected to contain a map-like object, providing all fields, instead of having many fields in the storm stream. If you use a punchlet node as source, you need to use "field_publishing_strategy": "tuple" setting in this node, to avoid sending the tuple as json, because the JDBC nodes do not decode the "json" encoding of tuple.
    
    See Also:
    
    Constant Field Values
  - CONVERT_BOOLEAN_TO_INT_SETTING
```
public static final String CONVERT_BOOLEAN_TO_INT_SETTING
```
    convert_boolean_to_int: if true, then the boolean value will be converted to 0 or 1 integer value at insert
    
    See Also:
    
    Constant Field Values
  - COLUMN_TYPE_CAST
```
public static final String COLUMN_TYPE_CAST
```
    See Also:
    
    Constant Field Values
  - COLUMN_TYPE_FUNCTION
```
public static final String COLUMN_TYPE_FUNCTION
```
    See Also:
    
    Constant Field Values
  - hosts
```
protected final List<String> hosts
```
  - username
```
protected final String username
```
  - password
```
protected final String password
```
  - database
```
protected final String database
```
  - poolName
```
protected final String poolName
```
  - table
```
protected final String table
```
  - bulkSize
```
protected final int bulkSize
```
  - poolSize
```
protected final int poolSize
```
  - queryQueueTime
```
protected final int queryQueueTime
```
  - columnNames
```
protected List<String[]> columnNames
```
  - columnNameWrapper
```
protected final String columnNameWrapper
```
  - columnsNamesSeparator
```
protected final String columnsNamesSeparator
```
  - sourceTupleField
```
protected final String sourceTupleField
```
  - convertBooleanToInt
```
protected final Boolean convertBooleanToInt
```
  - expirationTimeout
```
protected final long expirationTimeout
```
  - lastUpdateTimestamp
```
protected long lastUpdateTimestamp
```
  - reconnectionTimeout
```
protected final long reconnectionTimeout
```
  - hikariConfigMaps
```
protected List<Map<String,Object>> hikariConfigMaps
```
  - ackFail
```
protected transient List<org.apache.storm.tuple.Tuple> ackFail
```
  - jdbcHandleIterator
```
protected transient org.thales.punch.libraries.utils.api.RoundRobinIterator<JdbcHandle> jdbcHandleIterator
```
- Constructor Detail
  - AbstractJDBCOutput
```
protected AbstractJDBCOutput(org.thales.punch.libraries.storm.api.NodeSettings config,
                             String backend)
                      throws org.thales.punch.exceptions.ConfigurationException
```
    Throws:
    
    org.thales.punch.exceptions.ConfigurationException
- Method Detail
  - getComponentConfiguration
```
public Map<String,Object> getComponentConfiguration()
```
    Specified by:
    
    getComponentConfiguration in interface org.apache.storm.topology.IComponent
    
    Overrides:
    
    getComponentConfiguration in class org.thales.punch.libraries.storm.api.BaseProcessingNode
  - prepare
```
public void prepare(Map stormConf,
                    org.apache.storm.task.TopologyContext context,
                    org.apache.storm.task.OutputCollector collector)
```
    Specified by:
    
    prepare in interface org.apache.storm.task.IBolt
    
    Overrides:
    
    prepare in class org.thales.punch.libraries.storm.api.BaseProcessingNode
  - process
```
public void process(org.apache.storm.tuple.Tuple input)
```
    Specified by:
    
    process in class org.thales.punch.libraries.storm.api.BaseProcessingNode
  - addTupleToQuery
```
protected void addTupleToQuery(org.apache.storm.tuple.Tuple input)
```
    Add a tuple to INSERT query. If bulk size is reached, flush this query
    
    Parameters:
    
    input - tuple to add
  - formatColumnForAutoCast
```
protected Object formatColumnForAutoCast(Object value)
```
  - formatColumnForFunction
```
protected Object formatColumnForFunction(Object value)
```
  - failAll
```
protected void failAll()
```
  - cleanup
```
public void cleanup()
```
    Specified by:
    
    cleanup in interface org.apache.storm.task.IBolt
    
    Overrides:
    
    cleanup in class org.thales.punch.libraries.storm.api.BaseProcessingNode
  - getDefaultColumnNameWrapperChar
```
protected String getDefaultColumnNameWrapperChar()
```
  - getDefaultConvertBooleanToInt
```
protected Boolean getDefaultConvertBooleanToInt()
```

Class AbstractJDBCOutput

Field Summary

Fields inherited from class org.thales.punch.libraries.storm.api.BaseProcessingNode

Constructor Summary

Method Summary

Methods inherited from class org.thales.punch.libraries.storm.api.BaseProcessingNode

Methods inherited from class java.lang.Object

Field Detail

COLUMN_NAMES_SETTING

COLUMN_NAME_WRAPPER_CHAR_SETTING

SOURCE_TUPLE_FIELD_SETTING

CONVERT_BOOLEAN_TO_INT_SETTING

COLUMN_TYPE_CAST

COLUMN_TYPE_FUNCTION

hosts

username

password

database

poolName

table

bulkSize

poolSize

queryQueueTime

columnNames

columnNameWrapper

columnsNamesSeparator

sourceTupleField

convertBooleanToInt

expirationTimeout

lastUpdateTimestamp

reconnectionTimeout

hikariConfigMaps

ackFail

jdbcHandleIterator

Constructor Detail

AbstractJDBCOutput

Method Detail

getComponentConfiguration

prepare

process

addTupleToQuery

formatColumnForAutoCast

formatColumnForFunction

failAll

cleanup

getDefaultColumnNameWrapperChar

getDefaultConvertBooleanToInt