ThirdEye Data Inc.

Compensation

Full Description

Responsibilities: • Kafka Cluster Management: • Install, configure, and maintain Confluent Kafka clusters in an on-premises environment. • Manage Kafka brokers, Zookeeper, Connect, Kafka Streams, and related services. • Monitor and tune the performance of Kafka clusters to meet operational SLAs. • Security & Compliance: • Implement and manage Kafka security including SSL/TLS, ACLs, role-based access control (RBAC). • Ensure that Kafka infrastructure complies with the organization's security policies and best practices. • Monitoring & Alerting: • Set up monitoring and alerting for Kafka clusters using tools like Prometheus, Grafana, or Confluent Control Center. • Analyze logs and metrics to proactively detect and resolve issues. • Automation (Good to have): • Develop automation scripts and frameworks to streamline Kafka operations, such as provisioning, configuration, scaling, and monitoring using tools like Ansible, Terraform, or custom Python scripts. • Automate Kafka cluster deployments, upgrades, and patch management. • Implement auto-scaling, failover, and disaster recovery solutions through automation. • Knowledge of CI/CD pipelines and infrastructure-as-code (IaC) concepts for Kafka deployments. • Maintenance & Upgrades: • Regularly perform maintenance and upgrades for Kafka components (brokers, Zookeeper, etc.). • Handle Kafka backup and disaster recovery procedures. • Troubleshooting & Support: • Provide operational support and incident management for Kafka-related issues. • Work closely with developers and data engineers to resolve application-level issues. • Documentation: • Maintain accurate documentation of Kafka architecture, configuration, and operational procedures. • Create runbooks for various Kafka administrative tasks. • Experience & Skills needed: • 3-5 years of hands-on experience managing Confluent Kafka clusters in production environments, preferably on-premises. • Solid understanding of distributed systems, high availability, and failover mechanisms. • Experience with Kafka Cluster Linking and cross-cluster replication. • Location/Work Timings/Travel • Remote Working position – preference for resource to be based in US. • Work Timings US CDT : • At least six hours coverage for US CDT – for resources based in India. • Travel required – At start of the engagement Job Type: Contract Pay: $50.00 - $55.00 per hour Work Location: Remote