AI Coder

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2023 Jun 11 5:19
Editor
Edited
Edited
2026 Feb 12 11:47

AI Coding Agent

The human role shifts from "coding" to environment design, specification writing, and feedback loop construction.
  • Project Folder (Repository) = System of Record: All knowledge must exist explicitly within the repo.
    • AGENTS.md should be kept short, with actual knowledge distributed in docs/ structure.
    • Documents are also mechanically verified with CI and lint.
  • Agent Legibility
    First: Code style prioritizes structures that agents can easily understand over human preferences.
    • Architecture layer structure: Types → Config → Repo → Service → Runtime → UI
    • Dependency direction enforced by lint. Rules are enforced by code, not documentation.
  • Agents directly handle UI, logs, and metrics
    • Chrome DevTools Protocol integration.
    • LogQL / PromQL access available, feedback loop must be automated

Problem for agent is
Language Model Context
Size

The advent of large language models could potentially reduce software development costs to nothing, sparking a rapid and diverse growth in software akin to the content boom or
Cambrian explosion
.
https://aavetis.github.io/ai-pr-watcher/
LLMs should be used in conjunction with other tools to prevent the human review process from becoming a bottleneck.
One approach to reinforcement learning involves generative and discriminative models, such as
GAN
. Typical high-level AI development follows this approach and requires automation. While images can be compared visually, it's much harder to evaluate text, code, and audio. Therefore, a good AI coding assistant should not just provide results, but should help by breaking tasks down into smaller, easily verifiable steps. In other words, the importance of verifiability aligns with
Verifiable Reward
, suggesting that larger units like code blocks or video clips should be gradually incorporated.
AI Coder Services
 
 
 
AI Coding Agents
 
 
AI Coder Models
 
 
 
 
 
 

Code Quality
leaderboard

LLM Leaderboard for Code Quality & Security | Sonar
Independent analysis of code generation quality, security, and maintainability for leading LLMs.
LLM Leaderboard for Code Quality & Security | Sonar

Current limitations

  • Stop Digging; Know Your Limits
  • Mise en Place
  • Scientific Debugging
  • The tail wagging the dog
  • Consistent formatting
  • Read the Docs
  • Use Static Types
AI Blindspots
Blindspots in LLMs I’ve noticed while AI coding. Sonnet family emphasis. Maybe I will eventually suggest Cursor rules for these problems.

Leaderboard

WebDev Arena
WebDev Arena: AI Battle to build the best website
WebDev Arena
Big Code Models Leaderboard - a Hugging Face Space by bigcode
Discover amazing ML apps made by the community
Big Code Models Leaderboard - a Hugging Face Space by bigcode
PR workflow integration
Git workflow
Resolving code review comments with ML
Resolving code review comments with ML
Designing tools for developers means designing for LLMs too
Most large language models (LLMs) aren't great at using less popular frameworks.
Using LLMs to help LLMs build Encore apps – Encore Blog
How we used LLMs to produce instructions for LLMs to build Encore applications.
Using LLMs to help LLMs build Encore apps – Encore Blog
MLE-STAR: A state-of-the-art machine learning engineering agent
Jinsung Yoon, Research Scientist, and Jaehyun Nam, Student Researcher, Google Cloud
MLE-STAR: A state-of-the-art machine learning engineering agent
 
 
 

 

Recommendations