Skip to content

Show

Overview

The show node prints useful information on whatever data it receives. In particular if it receives datasets, it will print out a quick view of datasets columns and the associated schema.

You can control the output format, regular spark-like dataset dumps, i.e.:

+---+-------+
| id|clicked|
+---+-------+
|  4|    1.6|
|  8|    3.9|
+---+----
root
|-- id: integer (nullable = false)
|-- clicked: double (nullable = false)

Runtime Compatibility

  • PySpark :
  • Spark :

Example

---
type: punchline
version: '6.0'
runtime: spark
dag:
- type: show
  component: show
  settings:
    title: mytitle
    json_format: false
    pretty: true
    truncate: true
    vertical: false
    num_rows: 10
    show_schema: true
  subscribe:
  - component: input
    stream: data

Parameters

Name Type mandatory Default value Description
json_format Boolean false false Output dataset as json
pretty Boolean false false use this when json_format is set to true
truncate Boolean false false truncate column size if values are too big
vertical Boolean false false Output dataset in vertical
num_rows Integer false 3 limit the number of rows to display
show_schema Boolean false true prints to stdout the dataset schema