HDP Administrator / Architect

Role Summary

The HDP Administrator / Architect will provide deep technical expertise in administering and modernizing Hortonworks (HDP) and Cloudera environments as part of enterprise-scale data modernization initiatives powered by NexusOne. This role is responsible for ensuring operational stability, performance, and compliance of existing Hadoop environments, while architecting and supporting the transition toward open-source, cloud-native components (e.g., Spark, Iceberg, Trino, Ranger, DataHub).

This individual must possess hands-on depth in the full Hortonworks stack — from cluster management and security to performance tuning and metadata governance — serving as the technical bridge between legacy and modern data ecosystems.

Core Responsibilities

• Administer, monitor, and optimize Hortonworks Data Platform (HDP) clusters, including HDFS, YARN, Hive, HBase, Oozie, Ranger, Atlas, and Ambari.
• Perform environment maintenance and upgrades — patching, version management, capacity planning, and backup/recovery.
• Ensure platform reliability, high availability, and adherence to enterprise security and compliance requirements.
• Collaborate with Data Engineering and Platform Architecture teams to design migration strategies from HDP/Cloudera to modern open-source frameworks.
• Lead the migration of data, metadata, and security policies from legacy components (e.g., Atlas/Ranger) to DataHub, Iceberg, and modern equivalents.
• Conduct root-cause analysis and resolve complex performance or stability issues across distributed systems.
• Implement and maintain Kerberos, LDAP/SSO integration, and role-based access control (RBAC) policies.
• Document architecture, configurations, and operational runbooks for cluster management and decommissioning.
• Train and mentor engineers in HDP administration, troubleshooting, and modernization best practices.

Must-Have Core Skills

• Deep Hortonworks Expertise: Proven hands-on administration experience with HDP 2.x / 3.x, including HDFS, YARN, Hive, HBase, Oozie, Ranger, Atlas, Ambari, and Kerberos.
• Cloudera Stack Knowledge: Understanding of Cloudera Manager and CDP transition paths.
• Security & Governance: Expertise in configuring and managing Ranger, Atlas, Kerberos, LDAP/SSO, and encryption frameworks.
• Performance Optimization: Skilled in resource tuning, query optimization, and capacity planning for large-scale clusters.
• Automation & Infrastructure: Experience with Ansible, Terraform, or equivalent for provisioning and configuration management.
• Monitoring & Observability: Hands-on with Grafana, Prometheus, Ambari Metrics, or equivalent platforms.
• Scripting & OS: Strong Linux administration and scripting (Bash, Python).

Ideal Background

• 5–10 years of experience as an HDP or Cloudera Administrator supporting production-grade clusters at scale.
• Deep operational experience with Hortonworks Data Platform (HDP), including upgrades, troubleshooting, and tuning.
• Prior involvement in data platform modernization or cloud migration programs.
• Knowledge of emerging open-source frameworks (Spark, Trino, Iceberg, Ranger, DataHub).
• Proven ability to collaborate across platform, security, and data teams to deliver mission-critical outcomes.

Success Criteria

• Sustained platform stability and high availability throughout modernization.
• Successful migration of data, metadata, and policies to NexusOne-aligned open-source components.
• Zero major incidents or regressions during transition or decommissioning.
• Clear documentation and knowledge transfer to engineering and operations teams.
• Recognition as a subject-matter expert in Hortonworks administration and modernization best practices.

We offer:

• We welcome new ideas and allow you to make an immediate impact on the team.
• Flexible Paid time off (PTO for any reason, including sick days (no specified limits) and flexible work schedule.
• Personal laptop.
• Health/Sport Budget.
• Fully remote.