Working with columns in pyspark
Selecting and renaming dataframes in pyspark
This article with cover various ways of selecting columns in spark dataframe. For the demo, we are going to use Auto-mpg dataset available from kaggle.
spark
Pyspark Data Sources
Pyspark supported data sources
Spark support various data sources. Spark has some core data sources built into it while the others are available and maintained by other developers from the community. In this post, I am going to explain the core data sources supported by pyspark.
How Apache Spark works
Spark Architecture
Apache spark is a distributed compute engine used to process large volume/amount of data. In this article I am going to provide information on how it works behind the scenes.
