Skip to content

File Batch Executor Output

The node file_batch_executor_output writes in a distributed way the dataset input in several files colocalized with the spark executors from a column containing string documents.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
{
    "type": "file_batch_executor_output",
    "component": "output",
    "settings": {
        "file_path": "file.txt",
        "column_input" : "input"
    },
    "subscribe": [
        {
            "component": "input",
            "stream": "data"
        }
    ]
}

Settings

  • file_path: String

    Location of the output file

  • column_input: String

    Name of the input column