Skip to content

HOWTO increase size of a storm topology

Why do that

When the size of logs suddenly increase or in specific case when a burst cross the topology, a java out of memory can occurs.

What to do

First of all, get the last topology configuration file. Do not forget to synchronize your platform configuration

1
2
$ cd $PUNCHPLATFORM_CONF_DIR
$ git pull

Then, find the following lines in your topology file:

1
2
3
4
5
"storm_settings" : {
    [...]
    "topology.worker.childopts" : "-server -Xms128m -Xmx128m",
    [...]
}

Update the topology.worker.childopts configuration, by adding some memory.

!!! warning "Check the memory available on storm cluster before increasing allocated memory."

Finally, restart the channel (or reload for processing channels) :

1
2
$ punchplatform-channel.sh --stop <tenant_name>/<channel_name>
$ punchplatform-channel.sh --start <tenant_name>/<channel_name>