About the Role

This role is a Data Engineer II position focused on building and maintaining near real-time data infrastructure for an LLM-based insights platform at Amazon's WW FBA. The engineer will design streaming pipelines, optimize embedding freshness, and implement global latency strategies to deliver low-latency, AI-driven analytics worldwide.

Key Responsibilities

• Design and implement streaming data pipelines to process high-volume, near real-time data from multiple sources
• Build and maintain infrastructure supporting large language models, including embedding generation, vector storage, and retrieval systems
• Develop and optimize a modern data lakehouse to support both batch and real-time analytics workloads
• Implement caching strategies, query optimization, and multi-region deployment to achieve sub-second response times
• Ensure data reliability, freshness, and compliance across the entire data pipeline

Required Skills & Qualifications

Must Have:

• 3+ years of data engineering experience
• 4+ years of SQL experience
• Experience with data modeling, warehousing and building ETL pipelines
• 5+ years in data processing and AWS services (Kinesis, MSK, Lambda, Glue)

Nice to Have:

• Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
• Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
• Familiarity with global latency strategies (multi-region deployments, edge caching)