The role
Job description
The Role
We are looking for a hands-on AI Architect to lead the design and productionisation of agentic AI systems on ONDC.
You will own the architecture for LLM- and SLM-powered agents that operate across the network, drive the training, distillation and domain adaptation of small language models tuned for Indian commerce, and stand up the MLOps backbone that takes these systems from prototype to production at network scale.
This is a senior individual contributor role with significant scope: you will set architectural direction across multiple workstreams, mentor a growing engineering team, and partner with policy, product and Network Participants to ship AI capabilities responsibly and in the open. At ONDC, architects build — you will design, code and be in the room when something breaks in production.
What You'll Own
Design and build multi-agent systems for buyer- and seller-side use cases — intent capture, search and discovery, negotiation, order orchestration, and issue/dispute resolution — operating over ONDC's protocol APIs.
Define patterns for tool use, planning, memory, retrieval and guardrails; establish standards for agent observability, evaluation and safe rollout.
Architect retrieval-augmented systems over heterogeneous catalog, policy and transaction data, including hybrid lexical + semantic retrieval and multilingual indexing for Indian languages.
Lead end-to-end training pipelines for SLMs — data curation, tokenizer choices, supervised fine-tuning, DPO/RLHF and evaluation.
Drive knowledge distillation and quantisation strategies to ship cost-efficient models suitable for edge and low-latency network use cases.
Build domain adaptation playbooks for Indic languages and commerce-specific tasks, and establish rigorous offline/online evaluation harnesses.
Define the reference MLOps stack — feature and embedding stores, training orchestration, experiment tracking, model registry, CI/CD for models, and reproducible serving.
Stand up scalable inference for batch, real-time and streaming workloads across GPU and CPU fleets, owning latency, throughput and unit-economics targets.
Codify responsible AI practice — data lineage, PII handling, model cards and review gates — aligned with ONDC's open-network and DPI principles.
Author RFCs and reusable blueprints adopted across teams and Network Participants, and mentor senior engineers on evaluation rigour and production discipline.
What Success Looks Like
You become the owner of ONDC's agentic AI architecture — someone who can take an SLM from data curation to production inference, and whose RFCs and reference designs get adopted across teams and Network Participants.
What We're Looking For
7–10 years of software/ML engineering experience, with at least 3 years shipping production ML or LLM systems at meaningful scale.
Demonstrated depth in modern transformer architectures, training stacks (PyTorch, DeepSpeed/FSDP, Megatron, Hugging Face), and distributed training.
Strong grasp of agentic patterns — planning, tool use, RAG, evaluation — and associated frameworks (LangGraph, LlamaIndex, DSPy, or equivalent in-house).
Production MLOps experience: model serving, vector stores, feature stores, containers, Kubernetes, CI/CD, and cloud (AWS/GCP/Azure).
Excellent system design judgment; able to reason about correctness, scale, cost and failure modes simultaneously.
Clear written communication — comfortable writing RFCs and influencing without authority across cross-functional and external stakeholders.
A strong plus
Experience across Indic NLP | multilingual tokenisation | open-source AI/ML contributions | Beckn / UPI / ULI or other India DPI rails | commerce search and fraud/risk ML.