![]() ![]() profile: echo "export SPARK_HOME=/opt/spark" > ~/.profileĮcho "export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin" > ~/.profileĮcho "export PYSPARK_PYTHON=/usr/bin/python3" > ~/.profile Use the echo command to add these three lines to. There are a few Spark home paths you need to add to the user profile. Configure Spark Environmentīefore starting a master server, you need to configure environment variables. If you mistype the name, you will get a message similar to: mv: cannot stat 'spark-3.0.1-bin-hadoop2.7': No such file or directory. The terminal returns no response if it successfully moves the directory. Use the mv command to do so: sudo mv spark-3.0.1-bin-hadoop2.7 /opt/spark ![]() The output shows the files that are being unpacked from the archive.įinally, move the unpacked directory spark-3.0.1-bin-hadoop2.7 to the opt/spark directory. Now, extract the saved archive using tar: tar xvf spark-* Remember to replace the Spark version number in the subsequent commands if you change the download URL. Note: If the URL does not work, please go to the Apache Spark download page to check for the latest version.
0 Comments
Leave a Reply. |