Cursor and GitHub Copilot: A Repository Trial Worksheet
On this page 8 sections
Short answer
Evaluate Cursor and GitHub Copilot on accepted repository changes, including setup, failed runs and human review. Keep tasks, permissions and success criteria comparable. Record the product, model, version and billing unit used for each run. A faster-looking demo does not establish that either tool improves your team’s throughput.
This page is a trial worksheet, not a second general buying guide or a report of tests we performed. For product, pricing and privacy questions, start with the main Cursor versus GitHub Copilot comparison.
A published result is a reason to measure your own work
METR’s July 2025 study of experienced open-source developers reported that participants took longer with the early-2025 tools studied, despite expecting them to help. The reported slowdown was 19 percent in that setting.
This does not establish a current ranking of Cursor and Copilot. The study concerns particular developers, repositories, tasks and tools at a particular time. Later products and other working conditions can produce different results. It is a useful counterexample to assuming that positive impressions are a productivity measurement.
The worksheet below is an editorial proposal. It does not replicate that experiment, supply a statistically powered study design, or turn a small team’s results into a universal conclusion.
Decide what counts as accepted work
Choose a narrow set of authorized tasks before running either product. Examples include a regression with a failing test, a small feature with specified behavior, a refactor with compatibility requirements and a documentation correction that can be checked against the code.
Give each task an acceptance definition. A reviewer should be able to reject a patch for a concrete reason even if it compiles. Record the baseline commit, relevant tests and behavior expected from the completed change.
Do not use the same solved task twice and interpret the second run as independent. A developer who has seen the answer can carry that knowledge into another tool. Rotate task assignments or use comparable unsolved tasks, and disclose the limitations of the comparison.
Keep the evaluator’s ordinary workflow visible. If production work includes review, tests and a release check, the trial should not stop at the first generated diff.
Match the mode of work before comparing tools
Compare interactive editing with interactive editing, or delegated repository work with delegated repository work. Record where execution happens and what the agent can access.
Name the model rather than writing only the product brand. Save the product version, task instructions, enabled tools and relevant configuration. If the model is routed automatically and its identity is unavailable, mark it unknown instead of guessing.
Repository access should be authorized before the run. Use a scoped test environment without production credentials or customer data that the team is not permitted to transmit. A convenient real-world task does not create permission to expose that material.
Also record administrator settings that affect the work. A blocked command can be a successful control, not an agent failure. A patch generated with broader permissions is not an equivalent result unless the comparison explains the difference.
Keep one record per attempted task
Use this table as an empty worksheet. Its fields are not observed results.
| Field | What to record |
|---|---|
| Task and baseline | Task ID, repository revision and acceptance criteria |
| Product and model | Product version, model name or unknown, execution mode and date |
| Permissions | Allowed repositories, tools, data and network access |
| Time | Setup, active work, waiting, review and correction time |
| Attempts | Retries, abandoned runs and manual completion |
| Output | Patch or pull request, tests and reviewer decision |
| Cost | Provider-reported usage, applicable billing unit and attributable charges |
| Failure | Incorrect behavior, regressions, access violations or unresolved issues |
Measure elapsed time and human time separately if the distinction matters to the team. A delegated run may finish slowly while requiring little supervision, or finish quickly while creating substantial review work.
Preserve failed tasks in the denominator. Removing an abandoned run while retaining its successful retry understates the effort needed to obtain accepted work.
Compare the bill in the units the provider actually uses
Cursor’s model and pricing documentation describes usage pools and model-dependent consumption. Read the currently assigned plan and dashboard rather than converting every prompt into an assumed fixed price.
GitHub’s Copilot billing reference separates current model-pricing information from legacy request-based billing for eligible annual subscribers. Those legacy request allowances should not be applied to every user.
During the trial, retain the account’s actual unit and attributable usage. If a pooled plan prevents precise per-task attribution, report that limitation. Do not manufacture precision by dividing the full bill by the number of prompts.
Compare total task cost per accepted change, with setup shown separately. This proposed measure misses some effects: learning, maintainability and later defects may require a longer observation period.
Interpret a small trial conservatively
Report the tasks attempted, the tasks accepted and the main failure types. Keep the distribution visible if one difficult task dominates the total. A single average can hide that the tools helped routine edits but struggled with cross-module changes.
Where possible, have a reviewer assess the patch without being told which tool produced it. This can reduce one source of preference bias, although coding style or the task itself may still reveal the tool. Record whether that blinding was practical rather than claiming a fully controlled experiment.
Ask reviewers whether a failure came from the model, missing context, the environment or unclear instructions. The distinction can guide a better setup, but it should not erase the failed attempt from the record.
If the tools produce similar accepted results, the decision may depend on administration, workflow fit and cost. If the trial is inconclusive, record what additional observation would change the decision rather than declaring a winner for the sake of a comparison.
Before buying more seats, review an accepted patch with the person who will maintain it. Keep their assessment beside the task and usage records.
Revision note
On September 9, 2026, this article was narrowed from a second broad buying guide to a repository-trial worksheet. The earlier unsourced meeting scene, generalized adoption claims and universal cost assertions were removed. Source links and the limits of the proposed evaluation are explicit. No new hands-on benchmark is claimed, and the existing URL is preserved.