Thursday, 6 December 2018

Spark App error: BindException: Can't assign requested address: Service 'sparkDriver'


When you run your Spark Application or Spark Application unit tests, we may get the following error: Can't assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'sparkDriver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.


To solve this problem, please do the following steps:
  • Know your host name
         $ hostname
  • Execute the following command 
         $ sudo hostname -s
  • Update "/etc/hosts" file with the following new entry
          $vi /etc/hosts
  • Finally export the following environment variable
          export SPARK_LOCAL_IP="" 

Thats it.

It solved my problem in running my Spark application unit tests.

No comments:

Post a Comment