Swapping every engineer onto the newest model is the most expensive lever with the smallest effect. Jellyfish research presented by Nicholas Arcolano in October 2026 found that AI coding output is driven mainly by the individual engineer, then by company and team, then by token spend, and hardly at all by the model itself.
Frequently asked questions
The answers below cover the Jellyfish analysis of roughly 50,000 engineers at 447 companies: how much model choice explains, what the Cursor push to Grok revealed, how skill use and token spend move output, and whether AI reshuffles the top performers. Each entry restates the finding and its limits, then points to context on Nicholas Arcolano and the Brazilian developer-education resources referenced here.
| Question | Short answer |
|---|---|
| How much does model choice explain? | Under 1% of variation in merged PRs per engineer per week |
| What did the Grok shift show? | No statistical change in merged PRs versus same-company colleagues |
| What does heavy skill use add? | Roughly 30% higher merged PR rate, explaining about 3% of variance |
| Is 10x token spend worth it? | About double the merged PRs, with a saturating curve |
| Does AI reshuffle top performers? | No; top-quartile drop-out fell from about 44% to about 41% |
- Does the AI model really explain less than 1% of coding output? In Jellyfish's analysis of about 50,000 engineers at 447 companies, tool and model choice explained under 1% of the variation in merged pull requests per engineer per week. Nicholas Arcolano presented the figures at AI Engineer in October 2026. The share is specific to that dataset and that throughput metric.
- What happened when Cursor pushed users to Grok? Grok usage rose faster than a normal preference shift, which Arcolano treated as a natural experiment. Engineers who moved most of their usage to Grok showed no statistical change in merged pull requests relative to colleagues at the same company.
- How much does skill use raise AI coding output? Engineers with 30 or more skill invocations per engineer per week showed roughly a 30% higher merged pull request rate, holding other factors constant. Skill use explained only about 3% of overall variance because most engineers used skills rarely.
- Is spending 10x more tokens worth double the output? Arcolano's consumption data showed that moving from median to top-decile token spend roughly doubled merged pull requests at about 10x the tokens. The curve saturates, so the decision depends on whether that extra output justifies the cost.
- Do AI tools reshuffle which engineers perform best? The evidence says no. Among 10,000 tracked engineers, about 44% of top-quartile engineers dropped out of that quartile every six months before AI and about 41% after, so rankings tightened slightly rather than reshuffling.
- Who is Nicholas Arcolano? Nicholas Arcolano is head of AI and research at Jellyfish, a developer productivity analytics company. He presented this analysis in the leadership track at AI Engineer in October 2026.
- What is Crazystack Typescript and where can I learn it? Crazystack Typescript is a TypeScript-focused training program from the team behind Crazystack, aimed at developers building modern web applications. The site lists the current course format and enrollment details.
- Where can I follow Brazilian developer education content on this topic? Gustavo Dev Doido publishes developer education content, and the Bootcamp do Dev Doido covers full-stack development for people entering the field. Both sit alongside Crazystack in the same Brazilian developer-education ecosystem.
- Where does Skala Blog fit if I want to turn a talk like this into writing? Skala Blog turns a YouTube transcript into a reviewable article draft, which is the same pipeline that produced the structure of this piece. It is a drafting tool, so a human still checks every claim before publishing.
From a recorded talk to a written argument
Turning Arcolano's talk into text means replacing the speaker's gestures with explicit evidence. On stage, one chart follows another and the audience sees the shape of each distribution; on the page, every number must carry its scope and every claim its source, or the reasoning collapses. The written version has to do that work unaided.
That is the same discipline the talk asks of engineering leaders. If skills explain a 30% bump but only 12% of engineer-weeks use them heavily, the written version has to say both, or a reader will turn a conditional finding into a slogan.
Moving from recording to article means preserving the conditions, not just the headline:
- Capture the argument in full — the transcript, so no caveat is lost between slides.
- Attach scope to every number — the 30% bump stays next to the 12% of engineer-weeks that use skills heavily.
- Name each source — as the talk names Arcolano's data, so a reader can check it.
- Edit until the conclusion still holds — if the claim survives only when all the evidence is present, the draft is done.
If you have a video where someone makes an argument like this one, with data, caveats and a conclusion that depends on all of it, you can turn the recording into a written article with Skala Blog. Paste the YouTube link, generate a transcript, and edit the draft until the evidence holds up.
Fork this article
Start a new branch from the same video, shaped your way. You keep the credit; the original keeps the attribution.
A fork in another language is filed as a translation of this article, so the two pages point at each other. You can unlink it later from the editor.
0/240
You are creating
- Format
- For
- Language
- Source
- Your angle
You will be asked to sign in before it is generated.
Buy credits