Job Description

 Administer and maintain Hadoop clusters and associated technologies, including installation, configuration, and troubleshooting.

 Monitor Hadoop cluster performance and resource utilization, proactively identifying and resolving bottlenecks.

 Implement and enforce security policies and access controls for Hadoop clusters, ensuring compliance with regulatory standards.

 Plan and execute upgrades, patches, and migrations for Hadoop components, minimizing downtime and disruptions to business operations.

 Collaborate with data engineers and scientists to deploy and manage big data applications on Hadoop clusters.

 Automate routine administrative tasks using scripting languages and configuration management tools.

Technical expertise (Hadoop):

Proficiency in Hadoop ecosystem components such as HDFS, YARN, Hive, Spark, Kafka, etc.

Experience with Hadoop cluster management tools such as Cloudera Manager, Ambari, or Hortonworks Data Platform.

Knowledge of Hadoop security mechanisms, including Kerberos authentication, encryption, and access controls.

Understanding of Hadoop capacity planning, resource allocation, and performance optimization techniques.

Familiarity with Hadoop data ingestion tools and techniques for data ingestion from various sources.