Upwind is a next-generation Cloud Security Platform that leverages runtime context to identify and prioritize critical risks, providing precise insights and efficient cloud security management. Unlike traditional tools, Upwind uses runtime data proactively for risk prioritization and posture insights, ensuring teams focus on what truly matters. With industry-leading efficiency and eBPF-powered sensors, Upwind delivers comprehensive capabilities, including agentless cloud posture discovery, real-time threat protection, and integrated API security. From misconfigurations to malware defense, Upwind ensures end-to-end, cost-effective cloud infrastructure protection.At Upwind, you'll have the opportunity to work on some of the most challenging problems in applied AI and security, turning cutting-edge research and LLM capabilities into products that operate in real customer environments.We're looking for an AI Engineering Lead to help drive the quality, intelligence, and continuous improvement of Upwind's AI and agentic systems.This is a hands-on AI engineering lead role focused on data, evaluation, and agentic workflows. You'll build the systems and methodologies that allow us to understand how our agents perform, identify where they fail, and continuously improve their reasoning, tool usage, and outcomes.You'll work closely with AI Researchers, Engineers, and Product teams to turn experiments into reliable, high-performing AI systems — from defining evaluation datasets and metrics to improving agentic workflows and deploying them at scale.What You’ll Build:Agentic Workflows: Design and continuously improve multi-step agent workflows, including reasoning, tool usage, planning, state management, and recovery strategies.AI Evaluation: Build evaluation frameworks, datasets, metrics, and automated pipelines to measure the quality and reliability of LLMs and agents.AI Data Infrastructure: Build the data pipelines and feedback loops required to collect, curate, analyze, and learn from real-world agent interactionsAI Observability: Build visibility into agent behavior and performance, including trajectories, tool calls, latency, failures, and outcome quality.Responsibilities:Own and evolve the evaluation strategy for Upwind's AI agents and LLM-powered features.Define meaningful metrics and automated evaluation pipelines for measuring agent quality, accuracy, reliability, and task completion.Design and improve agentic workflows to make agents more capable, reliable, efficient, and predictable.Build feedback loops from production usage back into evaluation datasets and development workflows.Develop tooling for running experiments and comparing models, prompts, workflows, and agent configurations.Work closely with AI Researchers to turn research ideas and experiments into production-quality AI capabilities.Collaborate with backend engineers to integrate AI workflows and evaluation systems into our production platform.Establish best practices for testing and validating non-deterministic AI systems.Monitor AI systems in production and continuously identify opportunities for improvement.Take end-to-end ownership of projects- from defining the problem and building the first experiment to productionizing and measuring its impact.