Why Regular AI Agents Are Bad at Code Review and How Alibaba Solved This Problem

A familiar story: you set up Claude Code or Cursor for pull request review, and end up with a bunch of vague comments, mismatched line numbers, and completely ignored files. Reading such feedback gets old fast, and the tool gets shelved.

Alibaba engineers spent two years running their own internal AI code review assistant on massive codebases. Recently they open-sourced the project under the name OpenCodeReview. The repository instantly gathered over 10,000 stars, and there are good reasons for that.

Where Universal LLM Agents Stumble

The problem with most LLM wrappers lies in the architecture. When you ask a general-purpose language model to analyze a Git diff, it tries to be clever where precise calculation is required.

Here's what developers regularly encounter when trying to delegate review to generative agents:

• Line positions get misaligned. The model gets confused in long files and attaches comments to neighboring code blocks.
• Attention scatters. On large PRs, the agent gets tired, starts cutting corners on context, and simply skips individual files.
• Quality fluctuates. A small prompt tweak changes the review character beyond recognition.
• Token usage goes through the roof. The model shuttles the entire file context back and forth without real necessity.

Alibaba concluded that a purely linguistic approach doesn't work for such tasks. The tool needs strict engineering guardrails.

Determinism Plus Agent

OpenCodeReview is built on a hybrid approach. The authors split the process into two layers: hard engineering logic handles organization, while the LLM is responsible only for code decisions.

Hard algorithms take care of the routine:

• Precise file filtering. The algorithm immediately determines which changes need review and which to discard.
• Related file grouping. The tool automatically bundles dependent files like code and its localization into a single unit.
• Parallel sub-agents. Each bundle is processed separately, allowing the tool to easily digest massive pull requests.
• Line alignment. A separate module verifies exact comment coordinates before output.

The neural network connects only where flexibility is needed: selecting context, fetching file contents from the repository, and finding specific errors like thread safety issues, NPE, or SQL injections.

What the Tests Showed

The developers assembled a benchmark from 50 popular open source repositories, 200 real PRs in 10 programming languages, and asked senior engineers to label 15,005 real bugs.

The result was telling. On the same model, OpenCodeReview beats a regular agent in precision and overall F1 score, while using 9 times fewer tokens.

An interesting detail: OpenCodeReview's recall is lower than Claude Code's. And this was an intentional choice. The tool was deliberately tuned to reduce noisy and false comments, even at the cost of missing debatable minor issues.

How to Try It Out

The utility is written in Go and distributed via npm. Installation takes half a minute:

npm install -g @alibaba-group/open-code-review

After installation, the ocr command becomes available. Model provider setup is interactive:

ocr config provider
ocr config model

The tool supports any OpenAI-compatible API as well as Anthropic. After entering the key, the system immediately verifies the connection.

For everyday work, there are several basic scenarios.

Check current changes in the working directory:

ocr review

Compare two branches:

ocr review --from main --to feature-branch

Scan a directory without Git history (e.g., when auditing an external project):

ocr scan --path src/services

If you already use Cursor or Claude Code, there's no need to configure separate API keys for OpenCodeReview. The utility can work in ocr delegate mode: it handles file filtering and rule matching, while your current AI assistant does the actual review.

Additionally, the project offers a browser-based session viewer Session Viewer, OpenTelemetry integration for metrics collection, and ready-made support for GitHub Actions or GitLab CI pipelines.

Who Will Benefit

The project will be useful for teams tired of useless noise in automated reviews. It's a tool for those who care about precise comment-to-line attachment and clear API budget consumption. If you need a strict assistant for finding obvious vulnerabilities and errors before merging, OpenCodeReview definitely deserves a place in your local terminal.