Does this position allow remote work?

Yes, fully remote work is allowed for this job.

What type of contract is offered for this position?

Type of contract for this position: {contract_type}

Job summary

Permanent contract

Fully-remote

Salary: $208.6K to 429.5K

Experience: > 5 years

Skills & expertise

Collaboration and teamwork

Python

yesterday

Interested in this job?

Questions and answers about the job

As a Senior Staff Software Engineer on the AI Platform team, you will shape and scale Pinterest’s AI/ML evaluation platform into a unified, reliable system adopted company-wide
You will identify modernization opportunities, elevate the roadmap, and execute technical strategy of a durable evaluation system for GenAI and ML workloads at Pinterest, partnering across engineering, research, product, and safety to improve quality, launch decisions, and trust in AI-powered experiences
Identify modernization opportunities and scale the roadmap for a company-wide evaluation system in partnership with technical leadership
Define cross-team technical strategy, align partner teams, and drive broad adoption of shared evaluation capabilities
Make durable architectural decisions that improve the user experience and reliability of Pinterest’s AI/ML evaluation platform
Establish evaluation methodologies and standards that balance automated metrics, expert judgment, human feedback, safety, and business outcomes
Lead complex cross-team workstreams, resolve technical tradeoffs and risks, and deliver impact

Deep familiarity with GenAI and RecSys evaluation libraries and frameworks, AutoEvals, OpenEvals, Ragas, etc
3+ years of deep, hands-on experience designing and operating evaluation systems for GenAI and complex ML systems, including benchmark design, golden datasets, automated metrics, human evaluation, model-based grading, and failure analysis
Strong programming skills in Python coupled with a solid grasp of distributed systems principles, and comfortable iterating with AI tools on coding and design
Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience
Strong judgment around GenAI risks, including robustness, bias, privacy, security, and the limitations of automated evaluation signals
5+ years of experience building scalable platforms for distributed experimentation, data and model versioning, orchestration, observability, release management, and production monitoring

These companies are also recruiting for the position of “Software & Web Development”.