All fields marked * are required
Role Summary
We are looking for a QA Lead (5–8 years experience) to drive quality engineering for RAIA, our AI Assurance and Evaluation platform. The ideal candidate should have strong experience in test strategy, test planning, test automation, and modern web application testing, along with exposure to AI/LLM-based applications. This role will be responsible for establishing QA processes, building automation frameworks, and ensuring the reliability, security, and performance of both traditional application features and AI-driven capabilities.
Key Responsibilities
Define and own the overall QA Strategy, Test Plan, and Automation Roadmap for the platform.
Establish testing processes across the SDLC including functional, integration, regression, performance, security, and UAT testing.
Design and implement automated test suites using Playwright.
Create and maintain test cases, test scenarios, and test data for UI, API, and backend services.
Collaborate closely with Product Managers, Developers, and Architects to ensure quality is built into the development lifecycle.
Lead release validation activities and drive defect triage and resolution.
Define quality metrics, test coverage goals, and release readiness criteria.
Mentor QA engineers and promote QA best practices across the team.
Evaluate and recommend testing tools, frameworks, and CI/CD quality gates.
AI Testing Responsibilities
Design and execute test strategies for AI-powered features, agentic workflows, and LLM integrations.
Validate AI evaluation outputs for quality, consistency, safety, and reliability.
Define test datasets, evaluation scenarios, and acceptance criteria for AI use cases.
Perform prompt, workflow, and integration testing for AI agents and evaluation pipelines.
Collaborate with engineering teams to establish repeatable AI testing and regression validation processes.
Required Skills
5–8 years of experience in Software Quality Assurance and Test Automation.
Strong hands-on experience with Playwright for UI automation.
Experience testing REST APIs using tools such as Postman, Swagger, or equivalent frameworks.
Experience creating Test Strategy, Test Plan, and Automation Strategy documents.
Strong understanding of SDLC, STLC, Agile, and CI/CD practices.
Experience with test management and defect tracking tools (e.g., Jira, Azure DevOps).
Good understanding of database testing and SQL.
Experience defining quality metrics and release readiness criteria.
Preferred Skills
Hands-on experience using AI-assisted development tools such as Claude, GitHub Copilot, Cursor, or equivalent AI coding assistants to accelerate test case creation, automation scripting, test data generation, and QA productivity.
Ability to effectively leverage AI tools for generating and reviewing automation scripts while ensuring code quality, maintainability, and accuracy through human validation.
Experience testing AI/LLM, Agentic AI, ML, or GenAI applications.
Exposure to AI evaluation concepts such as quality, safety, performance, hallucination, and response validation.
Experience with performance and load testing tools.
Familiarity with AWS cloud environments and modern SaaS applications.
Experience testing event-driven and microservices-based architectures.
What Success Looks Like
Established QA strategy and automation roadmap within the first 90 days.
High automation coverage for critical user journeys and APIs.
Consistent release quality with reduced production defects.
Repeatable testing framework for both platform and AI evaluation capabilities.
Clear quality metrics and test reporting adopted across engineering teams.
Required
Preferred