Jamie Nicholls

I’ve been in enterprise IT long enough to be pretty sceptical of anything described as a paradigm shift. But the way AI is starting to fit into development workflows genuinely feels different, and not just because the code suggestions have gotten better.

The thing that’s caught my attention is agents.

Not AI that helps you write code faster, but AI that takes a task, figures out what needs to change, does it, and hands you a pull request. That’s a different thing. And I’ve been meaning to properly get my head around it for a while now.

I’ve been circling it, too. I built a Dynamics 365 wave-release summariser out of small, single-responsibility agents, and my spec-driven development experiment showed how much a detailed spec changes what AI coding tools produce. Both left me with the same question: what happens when you hand the agent the whole task, not just a step?

So I’m building a small project specifically to force myself to find out.

This is Part 1, covering what agentic development actually is, what I’m building, and what I want to find out. Part 2 will be the honest debrief once it’s done.

My Hypotheses

Before I start, I’ve made a few predictions about how this experiment will play out:

1. Agentic development will be significantly faster for repetitive implementation work.
2. The quality of the ticket will have a direct impact on the quality of the code produced.
3. Claude Code will be surprisingly effective at generating tests and boilerplate.
4. Most failures won’t come from coding mistakes, but from ambiguity in the requirements.
5. The bottleneck will shift from implementation to review.
6. A well-written CLAUDE.md will measurably improve the output — and I’ll test that with a control, not a feeling.

I’ve been around long enough to know predictions rarely survive first contact with reality, but that’s part of the point. Part 2 will score every one of these against what actually happened.

What Agentic Development Actually Is

If you’ve used GitHub Copilot, you know the drill. You start typing, it suggests the next line or a block of code, you accept it or you don’t. It’s genuinely useful but you’re still the one driving. The AI is a smart assistant sitting next to you.

Agentic development is not that.

An agent takes a task description, reads your codebase to understand the context, works out what needs to change, writes the code, runs the tests, and opens a pull request. You review it. If something’s wrong, you tell it and it fixes the PR.

The loop looks roughly like this:

The perceive step is reading everything it needs: a ticket description, your existing code, your project conventions. Plan is working out what to do. Act is doing it. Observe is checking if it worked. Iterate is what happens when it didn’t.

The key distinction from Copilot is that you’re not guiding it line by line. You’re giving it a task and reviewing the output — a genuinely different way of working, and it changes what you spend your time on.

What That Actually Means for the Developer

The framing I keep seeing online is “AI is going to replace developers”, which I think mostly misses the point, at least for now. What’s actually happening is quieter than that: the job description shifts.

Instead of spending most of your time writing implementation code for well-understood tasks, you spend it writing requirements clearly enough that an agent can act on them, reviewing code you didn’t write, and designing the conventions the agent works within. The work moves up a level — less time in the weeds of implementation, more time deciding what good looks like and whether the output meets it.

The reviewing part matters more than people give it credit for. If you can’t read a pull request critically and recognise when something’s off, it doesn’t matter how fast the agent is. You still need to understand the code — you’re just applying that knowledge at the review stage instead of the implementation stage. If anything, that raises the bar. It’s a different kind of engagement with the work, not less of one.

The Tools That Make This Possible Right Now

The tool I’m centring this PoC around is Claude Code from Anthropic. It runs in your terminal, reads your codebase, and can connect to external tools such as Jira and GitHub. You can give it a task in natural language and it handles the implementation end to end.

The other key component is a file called CLAUDE.md that sits in the project root. This is where you document project conventions, folder structures, coding standards, technical constraints, and rules the agent should follow. The agent reads this file at the start of every session.

From what I’ve read, the quality of this file has a significant influence on the quality and consistency of the generated code. I’ll find out whether that’s true.

I’m also using GitHub Copilot, but for a different purpose. Copilot stays in the IDE for interactive coding assistance. Claude Code handles the autonomous work. They’re not competing; they’re operating at different levels.

What I’m Building

I needed a project that was small enough to be realistic for a self-funded side project, but open-ended enough that there would always be another feature to ask the agent to build.

The project is jira-gateway. It’s a REST API wrapper over the Jira API built as an Azure Function App using TypeScript.

Familiar territory — my APIM PoC put Azure Functions in front of Jira too. The difference this time is who writes the code.

The concept is deliberately simple. Instead of calling Jira’s API directly, consumers call the gateway, which handles authentication and returns clean, normalised responses. It’s a boring enough problem that the implementation won’t distract from the experiment, which is the actual point.

The only endpoint I’m writing myself is:

GET /api/ticket/{ticketId}

Pass it a ticket key and receive a JSON response. Everything else gets built by the agent from Jira tickets that I create. Projects. Boards. Search. Create ticket. All of it.

(Full disclosure: I used Claude to help scaffold the repo, transcribe the seed code from my planning docs, and load my ticket specs into Jira — the patterns and the specs are mine, but you’ll find Claude in the contributors list before the first ticket was even written. That’s the reality of building anything in 2026. The experiment proper starts at ticket one: from there, no human-written implementation code.)

The Stack

The cost has two parts, and this catches people out. Claude Code in the terminal is covered by the $40/month Pro subscription. But the GitHub App — the @claude implement this workflow — runs on an API key and bills per token, completely separately from the subscription. I’ll load a small credit and report the actual per-run cost in Part 2. Beyond that: Azure’s Flex Consumption plan has a monthly free grant that more than covers a PoC, Jira Free is sufficient for a small project, and GitHub is free. Still a cheap experiment — just not quite the flat $40/month it first appears.

How I’m Planning to Run It

I’ve split the build into three layers, each becoming progressively more autonomous. Notice what doesn’t change as the layers get more autonomous: the PR review. That stays mine at every layer.

Layer 1: Assisted Development

I create the project, build the seed endpoint, write the CLAUDE.md file, and create Jira tickets for the initial feature set. Claude Code implements those features and opens pull requests. I review and merge them.

Layer 2: GitHub-Native Workflow

Once the Claude GitHub App is configured, I can create a GitHub issue and simply write:

@claude implement this

Claude handles the implementation and raises a pull request without the terminal being involved. Write issue. Tag Claude. Review PR.

Layer 3: Jira-Driven Workflow

The final step is connecting Claude directly to Jira through an MCP server. At that point the agent can retrieve work directly from Jira, prepare an implementation, and raise a pull request against the highest-priority ticket.

Human review and approval remain part of the process, but the amount of manual coordination drops significantly.

Phase 1 Features

The agent will build:

GET  /api/projects
POST /api/ticket
GET  /api/boards
GET  /api/tickets?jql=...

The Controls

Here’s the problem with most “I tried an AI agent” posts: every task is well-specified, so of course it works. Four well-templated tickets can’t tell me whether ticket quality matters — there’s no variation in the variable.

So the experiment includes three deliberate controls:

A vague ticket. One ticket’s entire description is: “We need to be able to add comments to tickets through the gateway.” No route, no response shape, no validation rules. If hypothesis 2 is right, the difference between this PR and the templated ones should be obvious — and measurable in review effort.

A no-CLAUDE.md run. I’ll run one comparable ticket on a throwaway branch with CLAUDE.md deleted, then run the same ticket normally, and diff the results. That turns “the instruction file matters” from received wisdom into evidence, one way or the other.

A hard ticket. The Phase 1 endpoints are all near-clones of my seed pattern. One extra ticket — transitioning a ticket’s status — requires two chained Jira API calls and error mapping with no precedent in the codebase. If the agent falls over anywhere, it should be here.

Here’s the backlog as it stands — six tickets, priorities descending in run order, nothing started:

What I Actually Want To Know

There are a few specific things I’m curious about going into this.

Does CLAUDE.md Make a Real Difference?

I’m assuming a well-written instruction file leads to noticeably better and more consistent code. The no-CLAUDE.md control run means I’ll have a with/without comparison to point at, rather than an assumption.

Where Does It Fall Over?

My suspicion is anywhere ambiguity exists. Vague ticket descriptions. Undocumented behaviours. Business rules that exist only in someone’s head. The vague ticket and the hard ticket are in the backlog specifically to find out, rather than waiting for ambiguity to happen by accident.

What’s the Real Cost?

Two channels: the flat $40/month subscription for terminal work, and per-token API billing for the GitHub App runs. I’m also curious how heavily you can lean on the Pro subscription before its usage limits become noticeable. I’ll track all of it per ticket.

Is It Actually Faster End-to-End?

I suspect yes for clearly defined work and no for fuzzy or exploratory work. But I’d rather have evidence than intuition.

Who Owns the Outcome?

This is the question I don’t see discussed often.

If an agent introduces a bug, creates an insecure implementation, or makes a poor architectural decision, the responsibility doesn’t transfer to the AI. It remains with the engineer who reviewed and approved the change.

I’m interested to see whether agentic development makes code review a more important skill than implementation itself. My suspicion is that it does.

How I’ll Measure Success

Rather than relying solely on gut feel, I’ll track a few practical metrics during the experiment.

I’ve set the tracking log up before ticket one — if the scaffolding isn’t there from the start, Part 2 becomes vibes reconstructed from memory, which is exactly what I’m trying to avoid.

The goal isn’t to produce a scientific study. It’s simply to gather enough data to separate genuine productivity gains from novelty and enthusiasm.

Why This Matters

The interesting question isn’t whether AI can write code. We’ve already answered that.

The more interesting question is whether software development starts to resemble engineering management at an individual level.

In that world, success depends less on how quickly you can type and more on how clearly you can communicate intent, establish standards, and evaluate results.

That’s the shift I’m trying to understand. Not whether AI can replace developers, but how the role of the developer changes when implementation is no longer the primary bottleneck.

What Comes Next

The seed endpoint is written and deployed — the gateway is live on Azure with its one human-written route, and the backlog is loaded. What’s left is the experiment itself: six tickets, one agent, and a scorecard to settle.

Every feature beyond the initial endpoint will be delivered through the agentic workflow. I’ll be tracking the quality of the generated code, the time involved, where the agent succeeds, where it fails, and how much effort remains with the human reviewer.

The repo is public — if you want to watch the PRs land (including the one from the deliberately terrible ticket), follow along: github.com/jamienicholls/jira-gateway.

Part 2 will be an honest retrospective. No polished demo. No benchmark gymnastics. Just a practical look at what happened when I attempted to build a real project using an agent-first development approach.

If the promise of agentic software development is that the work moves up a level, I want to find out whether that’s actually true in practice.