👋 Welcome to my portfolio

Sr Data Engineer
& AI Specialist

10+ years of experience building scalable data infrastructure on GCP, optimizing data pipelines, and implementing cutting-edge AI solutions.

Get In Touch View Projects
10+ Years Experience
500+ PB Data Migrated
80% Cost Reduction
☁️
GCP Expert
🤖
AI Engineer
⚡
Data Pipelines
🐍
Python Expert
About Me

Driving Innovation Through Data & AI

With 10 years of experience as a Data Engineer on Google Cloud Platform (GCP) and 2 years as an AI Engineer, I specialize in building scalable data infrastructure, optimizing pipelines, and implementing machine learning models.

I bring a systematic approach to problem-solving and excel at collaborating with cross-functional teams to drive project success. My expertise includes migrating 500 petabytes of data from BigQuery to BigLake, achieving an 80% cost reduction, and developing innovative tools that streamline data processing workflows.

🎓

Education

MCA from VTU University
BCA from Bangalore University

📍

Location

Bengaluru, India

🏆

Certifications

IISc PG Level Advanced
Google Professional Data Engineer

👨‍💻
Career Journey

Professional Experience

2019 - Present

Sr Data Engineer

News Corp

  • Led development of software to deploy pipelines for BI team, streamlining data processing
  • Migrated 500 petabytes from BigQuery to BigLake, reducing costs by 80%
  • Engineered solutions including API automation, AI SQL code review, and Data Wrangler
  • Introduced LNLC-compliant tools, optimizing tech stack and data management
  • Expertise in ETL utilizing Dataproc and Dataflow in GCP
2018 - 2019

Data Engineer

Euromonitor

  • Developed in-house ETL Python framework using Airflow and Kubernetes
  • Transformed data from multiple sources into standardized schema
  • Built RESTful APIs using Django Rest Framework
  • Led team in building user-friendly data access portal
2015 - 2018

Sr UI Developer

ANSR Source

  • Designed and deployed 50+ interactive simulations for clients
  • Led team in converting 2000 SPAs to WCAG 2.0 compliance
  • Created Python IDE simulation for client
  • Mentored new team members
Featured Work

Key Projects

🚀
11 Months

Newsflow Migration

Led migration of 800 pipelines from newsflow 1.0 to 2.0, transitioning from data warehouse to data mart architecture.

  • Migrated 500 PB from BigQuery to BigLake
  • Created LNLC tools reducing BAU task time
  • Decommissioned all old projects
GCP BigQuery BigLake
🔄
6 Months

API Automation

Developed robust API automation framework reducing pipeline development time from weeks to hours.

  • Automated 90% of API pipelines
  • Support for multiple authentication types
  • Integrated with Pandas for ETL processes
Python API Automation
🤖
2 Months

SQL Code Review AI

Implemented AI-powered SQL code feedback system for all PRs, enhancing code quality and deployment efficiency.

  • Integrated with Dataform & CircleCI
  • Compared Gemini 2.5-pro vs 2.5-flash
  • Audit tracking in BigQuery
AI Gemini SQL
🕷️
12 Months

Casper (Web Scraping)

Built end-to-end pipeline to replace 3rd party paid tool, handling data retrieval, processing, and delivery.

  • Python automated scraping scripts
  • Apache Airflow orchestration
  • Redis queue for intermediate storage
Python Airflow Redis
💻
6 Months

CSE 110 CPU Interactive

Created Python IDE simulator with visualization of conceptual machine state updates.

  • Execute arbitrary Python programs
  • Simulate recursive functions
  • Real-time state visualization
Python Simulation Education
Technical Expertise

Skills & Technologies

Cloud & Data Engineering

Google Cloud Platform
BigQuery
Dataflow
Dataproc
Apache Spark

AI & Machine Learning

Generative AI
Vertex AI
Agentic AI
Model Context Protocol

Programming & Frameworks

Python
Apache Airflow
JavaScript
HTML/CSS

DevOps & Infrastructure

Docker
Kubernetes (GKE)
Cloud Build
Cloud Run

Certifications

🎓

IISc PG Level Advanced Certification

Computational Data Science

☁️

Google Certified Professional

Data Engineer (Valid until 2025)

🐘

IIHT Hadoop

Big Data Technologies

Get In Touch

Let's Work Together

I'm always interested in hearing about new opportunities and collaborations

📧

Email

hemanthk0208@gmail.com
📱

Phone

+91 8095481467
💼

LinkedIn

linkedin.com/in/hemanth-k-592728102
📍

Location

Bengaluru, India

Available for Opportunities

I'm currently open to discussing new projects, innovative ideas, or opportunities to be part of your vision.

Send Email LinkedIn Profile