In some scenario you would like to configure your spark punchline network setting. For instance, limitting the range of ports that spark can use...
Follow this documentation for your purpose.
Limiting the number of Ports availables for Spark applications¶
By default, spark is configured to bind it's spawned process on random ports, be it the driver, blockmanager, driver ui, and so on...
Because of security concerns, this behavior is unacceptable. In most cases, we want to restrict ports avaibles for an application to a finite range.
This can be achieved per Spark Punchline by playing with the below parameters:
spark.port.maxRetries takes an integer as value and defines how many available ports will be allowed for
spark.blockManager.port where all three of them takes a starting port integer as value.
Behind the scene, spark will try to bind on a defined starting port and increment it by one if the port tried port is already in use until reaching
initial defined port + spark.port.maxRetries.