The role
Job description
Experience: 7–8 Years
Work Mode: Remote
Working Hours: 10:30 AM – 7:30 PM IST
Open Positions: 2
Joining: Immediate Joiners Only
Key Skills: LLM | Agentic AI | Test Automation | MCP Integration Testing | RAGAS
Job Description:
We are looking for an experienced Senior Software Engineer – QA with strong expertise in test automation, API testing, and AI testing practices. The ideal candidate should have hands-on experience validating LLM-based applications, Agentic AI workflows, MCP integrations, and RAG evaluation using RAGAS, along with a solid foundation in quality engineering.
Key Responsibilities:
- Analyse business requirements, technical specifications, and architecture documents to define comprehensive test strategies.
- Design and execute test plans, test scenarios, and test cases for functional and non-functional testing.
- Lead end-to-end testing activities, including functional, regression, integration, system, API, database, and end-to-end testing.
- Perform AI/LLM application testing, including response accuracy, relevance, hallucination detection, and reliability validation.
- Test Agentic AI workflows, tool-calling mechanisms, multi-step execution, and agent behaviour.
- Perform MCP integration testing to validate tool connectivity, communication, data exchange, and error handling.
- Use RAGAS to evaluate RAG applications, including retrieval quality, faithfulness, and answer relevance.
- Develop, maintain, and enhance automated test scripts and reusable automation frameworks.
- Perform API testing using Postman or REST Assured and validate backend integrations.
- Execute SQL-based database validation to ensure data integrity and consistency.
- Integrate automated tests into CI/CD pipelines and support continuous testing initiatives.
- Review test cases and automation scripts, manage defects, and collaborate with development teams for timely resolution.
- Prepare test execution reports, quality metrics, and release readiness assessments.
- Mentor junior QA engineers and drive improvements in QA processes, automation coverage, and testing efficiency.
- Support production validation, defect triage, and root cause analysis.
Mandatory Skills:
- 7–8 years of overall QA / Software Testing experience.
- Strong hands-on experience in test automation and framework development.
- Relevant practical experience in AI/LLM application testing.
- Experience with or strong hands-on exposure to Agentic AI and MCP integration testing.
- Experience with RAGAS for evaluating RAG/LLM applications.
- Strong API testing, SQL, integration testing, and regression testing skills.
- Familiarity with CI/CD pipelines and Agile methodologies.
Pay: ₹90,000.00 - ₹100,000.00 per year
Work Location: Remote