About

Data & AI engineer who builds and operates petabyte-scale data pipelines. At Amazon Music I built and operate the 24/7 data quality monitoring system that guards all 12B+ user events a day, an aggregated dataset platform with HyperLogLog data sketch capability, and a data pipeline and AI agent that triages ops tickets and generates personalized action items for every team. At Oracle Cloud I built the training pipelines and feature stores behind production ML models.

See more about me at srijanbhushan.com


Technical Tools

Big data: Spark · EMR · S3 · Parquet · Iceberg · Trino · Airflow · Kinesis · Glue · PyArrow · HyperLogLog sketches
Languages: Python · SQL · Scala · Java · TypeScript
Cloud & infra: AWS (S3, Glue, EMR, Redshift, Lambda, Step Functions, Bedrock) · CloudFormation · CI/CD · cost optimization · multiprocessing · multithreading · memory-efficient processing · Git
AI & LLM: Claude · OpenAI · open-source LLMs (Qwen via MLX) · LLM classification · Claude Code · Kiro
Other: Pandas · NumPy · Looker · QuickSight


Experience & Education

2022 – present
Data Engineer II
Amazon Music
  • Built and operated a 24/7 data quality monitoring system: petabyte-scale pipelines that process and monitor all 12B+ user events a day from user devices worldwide, detecting event volume drift and duplicate events and acting as the shield against critical data failures
  • Built an aggregated dataset platform serving analytics and monitoring systems across Amazon Music, with HyperLogLog sketches (including a Spark–Trino sketch compatibility library), compressing 12B+ daily events into 200M records at ~1% error and making data queries 21x faster and 14x cheaper
  • Built and operated a data pipeline and AI agent that triages ~10,000 ops tickets a week against a severity rubric I refined and generates personalized action items for all 100 teams, which cut tickets' mean resolution time from 30 days to 9 and lifted teams' high-severity assignment accuracy to 90%
  • Led Amazon Music's org-wide table format move from Hive to Apache Iceberg: championed the switch, built the first production tables, and drove adoption across 10+ datasets, cutting partition migration from 7 days to 1 and metadata updates from ~2 minutes to under 10 seconds
2017 – 2022
Sr. Data Scientist
Oracle Cloud
  • Owned training and inference pipelines for Oracle's Audience ranking ML service, including retraining and the production scoring path; improved performance 13% on average across quarters, driving more revenue
  • Built a versioned offline feature store and training pipeline for ad-targeting models covering 200M+ US profiles and 1B+ credit card transactions a day
2016
R&D Intern
Tesla
  • Vehicle diagnostics for Model S and X service
2015 – 2017
Master of Science
University of Washington
Big Data, Machine Learning, Data Science
2011 – 2015
Bachelor of Engineering
BMS College of Engineering
Signal Processing, Computer Networking, Electronics

Projects & Writing

shannongenerator.com

Synthetic data generator with scenario simulations across business models.

SO-101 robot arm

Teach-by-hand record/replay and webcam gesture control (MediaPipe) on a 6-DoF arm.

Running a small LLM on my Mac

7.6B open model on Apple silicon: cost, speed, what's inside the weights, where it fails.