CloudBlue Technologies

Compensation

Full Description

Google Cloud Data Migration Data Migration Team Lead Job Summary: Seeking a Google Cloud data engineer to design, build, and maintain scalable and efficient data processing systems on the Google Cloud platform. This engineer will be responsible for the entire data lifecycle, from ingestion and storage to processing, transformation, and analysis. Their work will enable client organizations to make data-driven decisions by providing clean, high-quality data to business intelligence tools, AI systems, analysts and data scientists. Required Skills & Qualifications: Google Cloud Platform Certified professional 8+ years of data engineering experience developing large data pipelines in very complex environments Very Strong SQL skills and ability to build very complex transformation data pipelines using custom ETL framework in Google BigQuery environment Very strong understanding of data migration methods and tooling, with hands-on experience in at least three (3) data migrations to Google Cloud Google Cloud Platform: Hands-on experience with key Google Cloud Platform data services is essential, including: o BigQuery: For data warehousing and analytics. o Cloud Dataflow: For building and managing data pipelines. o Cloud Storage: For storing large volumes of data. o Cloud Composer: For orchestrating workflows. o Cloud Pub/Sub: For real-time messaging and event ingestion. o DataProc: For running Apache Spark and other open-source frameworks. Programming languages: Strong proficiency in programming languages, most commonly Python, is mandatory. Experience with Java or Scala is also preferred. SQL expertise: Advanced SQL skills for data analysis, transformation, and optimization within BigQuery and other databases. ETL/ELT: Deep knowledge of Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes. Infrastructure as Code (IaC): Experience with tools like Terraform for deploying and managing cloud infrastructure. CI/CD: Familiarity with continuous integration and continuous deployment (CI/CD) pipelines using tools such as GitHub Actions or Jenkins. Data modeling: Understanding of data modeling, data warehousing, and data lake concepts