Problem:
When you run your Spark Application or Spark Application unit tests, we may get the following error:
java.net.BindException: Can't assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'sparkDriver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.
Solution:
To solve this problem, please do the following steps:
- Know your host name
$ hostname
my_hostname
- Execute the following command
$ sudo hostname -s 127.0.0.1
- Update "/etc/hosts" file with the following new entry
$vi /etc/hosts
127.0.0.1 my_hostname
- Finally export the following environment variable
export SPARK_LOCAL_IP="127.0.0.1"
Thats it.
It solved my problem in running my Spark application unit tests.
No comments:
Post a Comment