AMAP's evolution and practice of team-level AI development powered by Harness governance and SDD.
Hello everyone. I'm Wang Shuxin from AMAP's Large Model Application Platform. Today I'd like to share a topic titled "Farewell to 'Vibe Coding': The Evolution and Practice of Team-Level AI Development Powered by Harness Governance and SDD."
The story starts last September. Back then, I was invited to the Qoder sub-forum at the Yunqi Conference to share our team's experience in improving R&D efficiency with Qoder. It was a moment full of expectation—leveraging prompt engineering and context engineering, we achieved a 53% AI code-generation rate across the technical-design and development phases. At the time, that number showed us the enormous potential of AI programming.
After more than half a year of rapid AI progress, our team's code-generation rate can now reach 80%–90% and beyond. The number looks great—it has nearly doubled. But when we interviewed the team in depth and reviewed the PMO's data metrics, we discovered a puzzling fact: the efficiency gains were not obvious.
The code-generation rate went up, but project delivery cycles didn't shorten noticeably; AI wrote more code, yet developers' workloads didn't decrease. This forced us to stop and think seriously about the problem.
Let me first revisit the core content I shared back then. We identified three problems with AI Coding:
First, the free-improvisation problem. AI-generated code is often wildly unconstrained, because of insufficient business understanding and missing conventions. Ask it to build a feature, and it may give you three different implementations—each one "runs," but each may clash with your existing architecture.
Second, the reduced-efficiency problem. It sounds contradictory—isn't AI supposed to boost efficiency? But in real use, if your instructions aren't clear enough, you end up going back and forth across many rounds of dialogue. You say "change this," it changes it; you say "no, like this," it changes again. After several rounds, you might as well have written it yourself.
Third, the lost-key-information problem. Across multi-turn conversations, AI often "forgets" important constraints stated earlier. When the task granularity is too large, the architectural requirements mentioned at the start vanish by the end.
To address these issues, we proposed a systematic Qoder practice at the time: using Repo Wiki, Memory, and Rules to constrain AI's free improvisation; using prompt engineering to improve efficiency; and using context engineering and Quest mode to avoid losing key information.
At the end of that talk, I was full of expectations for the future of AI programming: in the future, developers would only need to define requirements and validate results, while the "grunt work" of documentation, coding, and testing would be handed to AI. AI would shift from a production tool into a new R&D infrastructure, and developers would evolve from coders into AI architects. That was the vision back then.
But half a year later, we were confused. Why didn't the higher code-generation rate translate into real efficiency gains? I spent a long time thinking about this, and eventually found three core reasons.
Let's look at the complete chain of a requirement from proposal to launch: product proposal, product-and-engineering review, solution design, development, code review, testing, integration, and launch. Every stage carries communication costs, waiting time, and the possibility of error.
The Mythical Man-Month has a famous thesis—there is no silver bullet. Why? Because software development is not just coding; it involves communication, collaboration, and decision-making. If you optimize the coding stage by 50%, but coding is only 30% of the whole chain, then the overall gain is just 15%. What's more, AI-generated code may bring more code-review time, more debugging time, and more rework.
This made me realize: real efficiency gains must connect the entire chain, not just optimize a single stage. Therefore, we need AI to cross stage boundaries and form a closed loop from requirement to deployment.
What is Vibe Coding? It's "programming by vibes"—casually tossing AI a few prompts and letting it generate thousands of lines of code in seconds. This approach might be fine for new projects or small scripts, but in legacy applications the risk is extremely high.
What characterizes legacy applications? They carry historical baggage, implicit dependencies, and business knowledge embedded in the code. If you let AI "vibe code," it may generate a solution that looks perfect but is completely incompatible with the existing system. Worse, these problems may only surface after going live.
We once ran into a case: AI-generated code changed the parameter order of a core interface, all unit tests passed, but after launch it caused errors in three downstream services. It took an entire day to track down. This drove home a lesson: in legacy applications, AI programming must move from "vibes" to "conventions," and there must be clear acceptance criteria.
That is exactly why we introduced SDD (Specification-Driven Development). The core idea of SDD is that before AI writes any code, we must first turn humans' vague ideas into clear, unambiguous, structured specifications, so that AI runs on a controllable track.
We've encountered requirements like this: a refactoring task involving a dozen front-end and back-end modules. You can't finish it in a single conversation—AI's context window is limited and its attention gets scattered. When the task is too large, AI loses sight of one thing while attending to another.
These three reasons point to the same conclusion: AI programming must upgrade from a "personal skill" to a "team-level engineering capability," and evolve from "vibe coding" into a "specification-driven, engineering-governed" R&D paradigm.
Having clarified the problems, we began looking for solutions. Our goal: let AI improve efficiency not only in the code-writing stage, but across the entire flow from requirement PRD to direct deployment.
We focused on two core ideas: SDD (Specification-Driven Development) and Harness (Harness Engineering).
The core idea of SDD is disruptive: specifications are no longer prose written for humans, but structured "intent code" that AI Agents can precisely understand and execute.
In traditional development, the PRD or design doc is merely a "guidebook," while code is the only "source of truth." This causes documents to quickly become outdated and disconnected from the code. SDD overturns this structure: the specification becomes the single source of truth. When requirements change, developers first modify the "specification," and AI tools then regenerate, validate, and update the underlying code accordingly.
The SDD workflow contains four stages:
First, Specify. The developer discusses with AI and produces a structured specification defining user stories, acceptance criteria, and system constraints. This is the "raw requirement" stage.
Second, Plan. Like a compiler, AI "compiles" the specification into a detailed technical plan and a task-breakdown list. This is the "technical document" stage.
Third, Implement. AI Agents execute the task list one by one, automatically generating high-quality code. This is the "software development" stage.
Fourth, Validate. Test cases are automatically generated from the specification and executed, ensuring the generated code fully conforms to the specification. This is the "functional and code-specification testing" stage.
If SDD solves the problem of "what to do," Harness solves the problem of "how to do it controllably."
The word "Harness" is vivid. Picture a wild horse—a large AI model has boundless power, but without a harness you can't even mount it and might get thrown off. The core of Harness Engineering is not to change the horse's genes (the model itself), but to design a precise control system for this wild horse.
A mature Harness system contains four core pillars:
First, context engineering. No longer simple RAG (Retrieval-Augmented Generation), but structured information feeding. Maintain a "single source of truth" so the Agent knows the project's directory structure, the current execution plan, and which documents are up to date.
Second, architectural constraints. This is the most hardcore part of Harness. Physical means force AI to obey the rules. For example, code in the UI layer is absolutely forbidden from directly accessing the database layer. If AI tries to violate the architectural layering, the code won't even pass syntax checks and is blocked before submission.
Third, feedback loops and entropy management. AI will inevitably make mistakes; the key is how to detect and correct them. Build an automated testing sandbox: the Agent writes code → tests run automatically → failure → error logs are read → the Agent self-corrects and retries. More importantly, human bug-fixing experience is solidified into new rules, ensuring AI never makes the same mistake twice.
Fourth, human oversight. Humans shift from "the ones who write code" to "reviewers" and "environment designers." Their responsibilities are to define complex business boundaries, handle the 5% of ambiguous logic AI can't judge, and optimize the Harness rules themselves.
From prompt engineering to context engineering to Harness Engineering, this is a paradigm shift: from "how to talk to AI," to "what AI should see," to "how AI runs in a controlled environment."
Based on these two core ideas, we began putting them into practice with Qoder.
Below, through a demo, I walk through the complete process of end-to-end development of a large requirement with Qoder.
In the demo, you can see the entire process is fully automated: starting from the requirement PRD, to Spec generation, to task breakdown, to code generation, to test validation, and finally to deployment. Throughout this process, the developer plays the role of requirement clarifier, specification reviewer, and result validator—not code writer.
Next, let me break down the whole practice in detail.
The foundation of the entire practice is the knowledge base. We organize knowledge in a three-layer structure: project layer, technology layer, and asset layer.
With the knowledge base in place, the next step is handling the requirement PRD. We use Qoder's Quest Spec mode to generate a standardized design.md document.
This process is not fully automated—it requires human intervention. This is the idea of HITL (Human-In-The-Loop).
Why HITL? Because requirement documents contain a lot of "tacit knowledge"—information product managers take for granted but that actually needs clarification. Take the simple feature of "user login": behind it there may be questions like which login methods to support, whether to remember login state, what password-strength requirements exist, how to handle login failures, and so on.
Through Spec mode, AI proactively asks questions, guiding developers to clarify this tacit knowledge and gradually complete a full Spec. A Spec includes:
Data model: which tables are involved, what the field definitions are, and what the relationships are.
Interface specification: API inputs and outputs, error codes, idempotency requirements, and so on.
Most importantly: acceptance criteria. This is the core of SDD. Acceptance criteria must be testable and unambiguous. For example, "the user is redirected to the home page after a successful login" is a vague description, whereas "after a successful login, the user is redirected to the home page within 3 seconds, and the home page displays the user's nickname" is a testable acceptance criterion.
With a complete Spec, AI has a clear "construction blueprint" and is no longer "vibe coding."
Once the Spec is ready, we enter the execution phase. We use Qoder's Experts Mode.
The core idea of Experts Mode is that different tasks are handled by Agents playing different roles—just like a real development team with front-end engineers, back-end engineers, test engineers, architects, and so on.
AI generates an execution plan from the Spec, breaking a large task into manageable subtasks. Each subtask has clear inputs, outputs, and acceptance criteria. Then, based on the task type, it is assigned to an Agent of the appropriate role.
The system comes with five built-in expert types, each with its own toolset, and it also supports custom expert types.
I also want to emphasize the shift in the user's role: the user is part of the coordination loop too. You can step in at any time while the Experts team is running, and the Experts Leader handles it in the next cycle—adjusting task direction or canceling tasks that are no longer needed. Your role has changed: you clarify intent with the Experts Leader, align on direction, review plans, and validate results—much like leading an experienced R&D squad.
After code generation and testing pass, we enter the deployment phase. Through the MCP (Model Context Protocol) tools provided by Aone (Alibaba's internal CI/CD platform), we hand the build artifacts to an Ops Agent for deployment.
Through MCP, the Ops Agent can trigger CI/CD pipelines, execute deployment scripts, query deployment status, and handle deployment exceptions. This connects the entire chain from requirement to deployment. Developers no longer need to manually operate various tools—they only make the key decisions under AI's guidance.
For projects that couple front end and back end, we also enabled some extended-tool Skills.
What are Skills? Skills are Qoder's capability-extension mechanism. Through Skills, AI can gain various additional abilities, for example:
Database-operation Skill: AI can directly query and modify the database to prepare and validate data.
With these Skills, AI can complete end-to-end development, testing, and validation—not just the generation of engineering code.
With the SDD-and-Harness solution, we connected the entire chain from requirement to deployment. More importantly, we achieved a paradigm shift from "vibe coding" to "specification-driven, engineering-governed" development. Developers are no longer passive code writers, but active requirement definers, specification reviewers, and result validators. AI is no longer an uncontrollable black box, but a reliable tool operating under Harness constraints.
Looking ahead, we believe three directions are worth exploring:
First, smarter Spec generation. Currently, Spec generation still requires considerable human intervention; in the future we hope to lower that cost through smarter, conversational requirement clarification.
Second, more powerful Agent Teams. The collaboration model of Agent Teams is still relatively simple today; in the future we hope to explore more complex modes, such as multi-round iteration and dynamic role assignment.
Third, more complete knowledge management. The knowledge base is the foundation of the entire system; in the future we hope to explore smarter mechanisms for knowledge extraction, updating, and reuse.
1. Identifying the Three Core Problems of AI Coding
The story starts last September. Back then, I was invited to the Qoder sub-forum at the Yunqi Conference to share our team's experience in improving R&D efficiency with Qoder. It was a moment full of expectation—leveraging prompt engineering and context engineering, we achieved a 53% AI code-generation rate across the technical-design and development phases. At the time, that number showed us the enormous potential of AI programming.
After more than half a year of rapid AI progress, our team's code-generation rate can now reach 80%–90% and beyond. The number looks great—it has nearly doubled. But when we interviewed the team in depth and reviewed the PMO's data metrics, we discovered a puzzling fact: the efficiency gains were not obvious.
The code-generation rate went up, but project delivery cycles didn't shorten noticeably; AI wrote more code, yet developers' workloads didn't decrease. This forced us to stop and think seriously about the problem.
Let me first revisit the core content I shared back then. We identified three problems with AI Coding:
First, the free-improvisation problem. AI-generated code is often wildly unconstrained, because of insufficient business understanding and missing conventions. Ask it to build a feature, and it may give you three different implementations—each one "runs," but each may clash with your existing architecture.
Second, the reduced-efficiency problem. It sounds contradictory—isn't AI supposed to boost efficiency? But in real use, if your instructions aren't clear enough, you end up going back and forth across many rounds of dialogue. You say "change this," it changes it; you say "no, like this," it changes again. After several rounds, you might as well have written it yourself.
Third, the lost-key-information problem. Across multi-turn conversations, AI often "forgets" important constraints stated earlier. When the task granularity is too large, the architectural requirements mentioned at the start vanish by the end.
To address these issues, we proposed a systematic Qoder practice at the time: using Repo Wiki, Memory, and Rules to constrain AI's free improvisation; using prompt engineering to improve efficiency; and using context engineering and Quest mode to avoid losing key information.
At the end of that talk, I was full of expectations for the future of AI programming: in the future, developers would only need to define requirements and validate results, while the "grunt work" of documentation, coding, and testing would be handed to AI. AI would shift from a production tool into a new R&D infrastructure, and developers would evolve from coders into AI architects. That was the vision back then.
2. Looking Beyond "Code-Generation Rate" to the Deeper Dilemma of "Efficiency"
But half a year later, we were confused. Why didn't the higher code-generation rate translate into real efficiency gains? I spent a long time thinking about this, and eventually found three core reasons.
Reason 1: R&D Is a Full-Chain Process, Not Just Writing Code
Let's look at the complete chain of a requirement from proposal to launch: product proposal, product-and-engineering review, solution design, development, code review, testing, integration, and launch. Every stage carries communication costs, waiting time, and the possibility of error.
The Mythical Man-Month has a famous thesis—there is no silver bullet. Why? Because software development is not just coding; it involves communication, collaboration, and decision-making. If you optimize the coding stage by 50%, but coding is only 30% of the whole chain, then the overall gain is just 15%. What's more, AI-generated code may bring more code-review time, more debugging time, and more rework.
This made me realize: real efficiency gains must connect the entire chain, not just optimize a single stage. Therefore, we need AI to cross stage boundaries and form a closed loop from requirement to deployment.
Reason 2: Vibe Coding on Legacy Applications Is Extremely Risky
What is Vibe Coding? It's "programming by vibes"—casually tossing AI a few prompts and letting it generate thousands of lines of code in seconds. This approach might be fine for new projects or small scripts, but in legacy applications the risk is extremely high.
What characterizes legacy applications? They carry historical baggage, implicit dependencies, and business knowledge embedded in the code. If you let AI "vibe code," it may generate a solution that looks perfect but is completely incompatible with the existing system. Worse, these problems may only surface after going live.
We once ran into a case: AI-generated code changed the parameter order of a core interface, all unit tests passed, but after launch it caused errors in three downstream services. It took an entire day to track down. This drove home a lesson: in legacy applications, AI programming must move from "vibes" to "conventions," and there must be clear acceptance criteria.
That is exactly why we introduced SDD (Specification-Driven Development). The core idea of SDD is that before AI writes any code, we must first turn humans' vague ideas into clear, unambiguous, structured specifications, so that AI runs on a controllable track.
Reason 3: Large Projects and Complex Requirements Exceed the Capability Boundary of a Single AI Conversation
We've encountered requirements like this: a refactoring task involving a dozen front-end and back-end modules. You can't finish it in a single conversation—AI's context window is limited and its attention gets scattered. When the task is too large, AI loses sight of one thing while attending to another.
These three reasons point to the same conclusion: AI programming must upgrade from a "personal skill" to a "team-level engineering capability," and evolve from "vibe coding" into a "specification-driven, engineering-governed" R&D paradigm.
3. The Solution: Introducing SDD and Harness
Having clarified the problems, we began looking for solutions. Our goal: let AI improve efficiency not only in the code-writing stage, but across the entire flow from requirement PRD to direct deployment.
We focused on two core ideas: SDD (Specification-Driven Development) and Harness (Harness Engineering).
SDD (Specification-Driven Development)
The core idea of SDD is disruptive: specifications are no longer prose written for humans, but structured "intent code" that AI Agents can precisely understand and execute.
In traditional development, the PRD or design doc is merely a "guidebook," while code is the only "source of truth." This causes documents to quickly become outdated and disconnected from the code. SDD overturns this structure: the specification becomes the single source of truth. When requirements change, developers first modify the "specification," and AI tools then regenerate, validate, and update the underlying code accordingly.
The SDD workflow contains four stages:
First, Specify. The developer discusses with AI and produces a structured specification defining user stories, acceptance criteria, and system constraints. This is the "raw requirement" stage.
Second, Plan. Like a compiler, AI "compiles" the specification into a detailed technical plan and a task-breakdown list. This is the "technical document" stage.
Third, Implement. AI Agents execute the task list one by one, automatically generating high-quality code. This is the "software development" stage.
Fourth, Validate. Test cases are automatically generated from the specification and executed, ensuring the generated code fully conforms to the specification. This is the "functional and code-specification testing" stage.
Harness Engineering
If SDD solves the problem of "what to do," Harness solves the problem of "how to do it controllably."
The word "Harness" is vivid. Picture a wild horse—a large AI model has boundless power, but without a harness you can't even mount it and might get thrown off. The core of Harness Engineering is not to change the horse's genes (the model itself), but to design a precise control system for this wild horse.
A mature Harness system contains four core pillars:
First, context engineering. No longer simple RAG (Retrieval-Augmented Generation), but structured information feeding. Maintain a "single source of truth" so the Agent knows the project's directory structure, the current execution plan, and which documents are up to date.
Second, architectural constraints. This is the most hardcore part of Harness. Physical means force AI to obey the rules. For example, code in the UI layer is absolutely forbidden from directly accessing the database layer. If AI tries to violate the architectural layering, the code won't even pass syntax checks and is blocked before submission.
Third, feedback loops and entropy management. AI will inevitably make mistakes; the key is how to detect and correct them. Build an automated testing sandbox: the Agent writes code → tests run automatically → failure → error logs are read → the Agent self-corrects and retries. More importantly, human bug-fixing experience is solidified into new rules, ensuring AI never makes the same mistake twice.
Fourth, human oversight. Humans shift from "the ones who write code" to "reviewers" and "environment designers." Their responsibilities are to define complex business boundaries, handle the 5% of ambiguous logic AI can't judge, and optimize the Harness rules themselves.
From prompt engineering to context engineering to Harness Engineering, this is a paradigm shift: from "how to talk to AI," to "what AI should see," to "how AI runs in a controlled environment."
Based on these two core ideas, we began putting them into practice with Qoder.
4. Full-Flow Automation in Practice
Below, through a demo, I walk through the complete process of end-to-end development of a large requirement with Qoder.
In the demo, you can see the entire process is fully automated: starting from the requirement PRD, to Spec generation, to task breakdown, to code generation, to test validation, and finally to deployment. Throughout this process, the developer plays the role of requirement clarifier, specification reviewer, and result validator—not code writer.
Next, let me break down the whole practice in detail.
Step 1: Design the Knowledge Base
The foundation of the entire practice is the knowledge base. We organize knowledge in a three-layer structure: project layer, technology layer, and asset layer.
- Project-layer knowledge includes the project overview, directory structure, architectural design, technology choices, and so on. These are the basis for AI to understand the project context. Following the idea of on-demand loading, we maintain a top-level README.md file as Qoder's "single source of truth"—if a piece of information isn't in the docs, it simply doesn't exist as far as Qoder is concerned.
- Technology-layer knowledge includes general technical knowledge, coding conventions, middleware, third-party library documentation, best practices, and common problem solutions. This knowledge is reusable across projects and reflects the team's technical accumulation.
- Asset-layer knowledge includes reusable code snippets, components, templates, historical requirement PRDs, technical solutions, archived test cases, and more. These are the "bricks" the team has accumulated over the years, and AI can use them directly to build new features.