About the Role

This role involves designing, building, and maintaining Spark-based infrastructure to process large datasets for AGI research. You'll work closely with researchers to develop data workflows and tools for multimodal datasets, ensuring scalability and efficiency.

Key Responsibilities

• Develop and maintain reliable infrastructure for large-scale data extraction and transformation
• Work closely with researchers to create tooling for emerging data-related needs
• Manage project prioritization, deliverables, timelines, and stakeholder communication
• Illuminate trade-offs, educate the team on best practices, and influence technical strategy
• Operate in a dynamic environment to deliver high quality software

Required Skills & Qualifications

Must Have:

• 5+ years of non-internship professional software development experience
• 5+ years of programming with at least one software programming language experience, or Bachelor's degree in computer science or equivalent
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience

Nice to Have:

• Experience as a mentor, tech lead or leading an engineering team
• Experience with full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations
• Experience contributing to the architecture and design (architecture, design patterns, reliability and scaling) of new and current systems