Understanding of HDFS principles Programming using the Spark framework Understanding in DevOps Understanding of Spark principles Experience in DevOps Programming in Scala Proficiency in SQL

data storage data analysis Zeppelin data collection Scala Airflow Hadoop Spark SQL Hive

YARN architecture in Hadoop Hadoop architecture Hadoop Distributed File System (HDFS) Linux Object-Oriented Programming (OOP) Apache Spark Scala language SQL queries Functional Programming (FP) MapReduce principle Design relational databases

subqueries PostgreSQL Apache Spark Hive QL analytical functions Hadoop database platform stored procedures/functions performance optimization code optimization for data storage architectures PySpark Greenplum database platform