Show

Overview

The show node prints useful information on whatever data it receives. In particular if it receives datasets, it will print out a quick view of datasets columns and the associated schema.

You can control the output format, regular spark-like dataset dumps, i.e.:

+---+-------+
| id|clicked|
+---+-------+
|  4|    1.6|
|  8|    3.9|
+---+----
root
|-- id: integer (nullable = false)
|-- clicked: double (nullable = false)

Runtime Compatibility

• PySpark : ✅
• Spark : ✅

Example

---
type: punchline
version: '6.0'
runtime: spark
dag:
- type: show
  component: show
  settings:
    title: mytitle
    json_format: false
    pretty: true
    truncate: true
    vertical: false
    num_rows: 10
    show_schema: true
  subscribe:
  - component: input
    stream: data

Parameters

Name	Type	mandatory	Default value	Description
json_format	Boolean	false	false	Output dataset as json
pretty	Boolean	false	false	use this when json_format is set to true
truncate	Boolean	false	false	truncate column size if values are too big
vertical	Boolean	false	false	Output dataset in vertical
num_rows	Integer	false	3	limit the number of rows to display
show_schema	Boolean	false	true	prints to stdout the dataset schema