VDB
Sign up
HIGH7.5

PYSEC-2019-114

Quick fix

PYSEC-2019-114 — pyspark: upgrade to the fixed version with the command below.

pip install --upgrade 'pyspark>=2.3.3'

Details

Prior to Spark 2.3.3, in certain situations Spark would write user data to local disk unencrypted, even if spark.io.encryption.enabled=true. This includes cached blocks that are fetched to disk (controlled by spark.maxRemoteBlockSizeFetchToMem); in SparkR, using parallelize; in Pyspark, using broadcast and parallelize; and use of python udfs.

Are you affected?

Enter the version of the package you're using.

Affected packages

PyPI/pyspark
Introduced in: 0Fixed in: 2.3.3
Fixpip install --upgrade 'pyspark>=2.3.3'

References