Wednesday, October 19, 2016

Set up the Apache SPARK keys and verifying your SPARK download

1. Go to this page, select your version of SPARK, and download it from a mirror
https://spark.apache.org/downloads.html

2. Download the MD5 and KEYS file associated with your version of SPARK
The links are in step 5 on the Apache SPARK download page.

https://www.apache.org/dist/spark/KEYS

Example MD5:
spark-2.0.1-bin-hadoop2.7.tgz.md5 


3. md5 the downloaded tar.gz file and compare the sum to the expected md5 value

4. Import the KEYS file and use it to verify the tar.gz file
https://www.apache.org/dyn/closer.cgi/spark



$ gpg --import KEYS.txt
gpg: key 15E06093: public key "Andrew Or " imported
gpg: key 82667DC1: public key "Xiangrui Meng (CODE SIGNING KEY) " imported
gpg: key 00799F7E: public key "Patrick Wendell " imported
gpg: key FC8ED089: public key "Patrick Wendell " imported
gpg: key 87FD1A97: public key "Tathagata Das (CODE SIGNING KEY) " imported
gpg: key 9E4FE3AF: public key "Patrick Wendell " imported
gpg: Total number processed: 6
gpg:               imported: 6  (RSA: 6)

VERIFY THE INTEGRITY OF THE FILES

It is essential that you verify the integrity of the downloaded file using the PGP signature (.asc file) or a hash (.md5 or .sha file). Please read Verifying Apache Software Foundation Releases for more information on why you should verify our releases.
The PGP signature can be verified using PGP or GPG. First download the KEYS as well as the asc signature file for the relevant distribution. Make sure you get these files from the main distribution site, rather than from a mirror. Then verify the signatures using
% gpg --import KEYS
% gpg --verify downloaded_file.asc downloaded_file
or
% pgpk -a KEYS
% pgpv downloaded_file.asc
or
% pgp -ka KEYS
% pgp downloaded_file.asc
Alternatively, you can verify the MD5 hash on the file. A unix program called md5 or md5sum is included in many unix distributions. It is also available as part of GNU Textutils. Windows users can get binary md5 programs from herehere , or here.

More help on verifying signatures: