Before we can value a software decision, we must learn to name every dollar it touches — and notice the ones that never appear on a budget line.
Instructor
Dr. Zhijiang Chen
Session
No. 02 of 16
Date
4 June 2026
Room
YF302
Duration
110 minutes
Format
Lecture & Discussion
Builds directly on Lecture 1. The vocabulary introduced today appears in every remaining lecture.
Lecture IIAgenda02 / 26
Today's Plan
Vocabulary, then a worksheet.
By the end you will be able to put any software cost into a defensible box, and you will know the three classical and three behavioural concepts every NPV calculation depends on.
§
Topic
Minutes
I.
Classical cost taxonomies
20
II.
Behavioural cost concepts
25
—
Class discussion (sunk-cost stories)
10
III.
Software lifecycle cost & TCO
25
IV.
The 2026 cost stack (cloud, API, tokens)
15
V.
Hands-on: a 5-year TCO worksheet
10
Homework brief, questions
5
Part — One
I
Classical cost taxonomies — how the accountant sees a software project.
Three independent splits. Every line item lives in one cell of each.
§ ITaxonomies04 / 26
Three independent splits
Every software cost lives in a 2 × 2 × 2 box.
Split
One side
The other side
Behaviour with volume
Fixed — doesn't change with usage
Variable — scales with usage
Traceability
Direct — tied to a specific product
Indirect — overhead shared across many
Timing
Non-recurring — one-time
Recurring — every period
A line item like "annual Datadog subscription" is Fixed × Indirect × Recurring. Saying so out loud already reveals what raising it or moving it would mean.
§ IFixed vs Variable05 / 26
Behaviour with volume
The cost question that scales — and the one that doesn't.
Fixed
Constant within a relevant range, regardless of usage.
Examples: dev-team salaries, an on-prem licence, a leased GPU box.
In modern SaaS, the trend is unmistakable: variable costs grow as a share of total cost. A 2010 software product was 80% fixed cost. A 2026 AI product can be 70% variable — token spend dominates the P&L.
If your costs are mostly fixed, every additional user is nearly free. If they are mostly variable, every additional user has a price tag — and your business model must reflect it.
§ IDirect vs Indirect06 / 26
Traceability
The cost you can blame on a product — and the cost that everyone pays.
Direct costs
Engineers dedicated to the product.
A licence bought specifically for it.
Cloud spend in the product's account.
Indirect costs (overhead)
Shared platform team and tooling.
IT support, security, compliance.
The office itself — rent, electricity.
Allocation is half art, half politics. The same dollar can be made direct or indirect by where it sits on the org chart, and that changes which manager has the incentive to control it.
§ ITiming07 / 26
Non-recurring · Recurring · CapEx · OpEx
One-time vs ongoing — and the great migration of the 2010s.
Renting that GPU on AWS · paying for Anthropic API.
The cloud era was, in accounting terms, a giant CapEx → OpEx migration. The AI era now extends it down to the request: you no longer rent a server by the hour, you rent a forward-pass by the token.
Part — Two
II
Behavioural cost concepts — how the economist sees a software decision.
Three ideas that don't appear on any budget line yet decide every major engineering choice.
§ IIOpportunity cost09 / 26
The cost you don't see on an invoice
The most important cost engineers never name.
opportunity cost = (value of best alternative you didn't choose)
If your senior engineer spends a week refactoring a logging library, the opportunity cost is not zero — it is whatever the next-best use of that week would have produced.
Engineering example
Two PRs in the queue. Reviewing PR A means PR B sits a day. The opportunity cost of reviewing A is whatever shipping B sooner would have produced.
AI-era example
$1,000 of LLM budget can fix 500 tickets at $2 each or generate marketing copy. Spending it one way costs you the other.
§ IISunk cost10 / 26
A cost already paid — and therefore irrelevant
The trap that keeps dead projects alive.
A sunk cost is money or effort already spent that cannot be recovered. The economically correct response to a sunk cost is to ignore it.
future decisions should compare future benefits with future costs only
In practice we don't. We keep funding the failing migration because "we've already paid for the licences." We hold onto the rewrite because "we're 70% done." The sunk-cost fallacy is the most expensive behavioural pattern in software organisations.
If you wouldn't start this project today knowing what you know now — the right answer is to stop it, regardless of what you've already spent.
§ IIMarginal vs Average11 / 26
The cost of the next unit, vs the cost per unit
For pricing and capacity, marginal is the only thing that matters.
Average cost
Total cost ÷ total units. Useful for accounting, profitability reports, and storytelling.
Example: Annual infra spend ÷ requests served = $0.004 per request.
Marginal cost
The added cost of serving one more unit. Useful for pricing and capacity decisions.
Example: The next million LLM tokens cost $X — independent of what the previous billion cost.
Engineers pricing an API often quote the average cost. Customers buying access incur the marginal cost. Mismatch here is a familiar trap — quote a low average price, watch your margin disappear when the customer hits the variable-cost tail.
Discussion10 minutes12 / 26
Tell us a story
?
What's the most expensive sunk cost you've ever seen — or paid?
Share in pairs first (3 minutes), then we'll surface the best three for the whole room.
What was the sunk amount — money, time, headcount, brand?
What was the rational call at the decision point?
What was actually decided — and what made the team choose it?
No naming companies or colleagues. Identify by role and pattern, not person.
Part — Three
III
Lifecycle cost — or, the dollars that arrive after launch.
Most engineering decisions are made for a fraction of the cost they generate.
§ IIILifecycle phases14 / 26
From request to retirement
Five phases. Five buckets of money.
Phase
What happens
Cost drivers
① Acquisition
Decide to build or buy. Sign contracts.
Legal, vendor evaluation, opportunity cost.
② Development
Design, build, test, ship 1.0.
Engineers, infra, tooling, ramp-up.
③ Operation
Serve users in production.
Cloud, licences, on-call, observability.
④ Maintenance
Bug fix, enhance, refactor.
Engineering ÷ years; technical-debt interest.
⑤ Retirement
Sunset, migrate users, archive data.
Migration project, regulatory, communication.
Most software cost lives in ③ and ④. Most engineering attention lives in ②. That is the central misalignment of our profession.
§ IIIThe maintenance bias15 / 26
An empirical fact, not a rhetorical flourish
Sixty to eighty percent of a software product's lifetime cost is incurred after 1.0 ships.
60–80%post-launch
Established across many empirical studies of large software portfolios since Lientz & Swanson (1980). The exact ratio varies by domain, but the order of magnitude does not.
Implication for design choices: any 1.0 decision that reduces future maintenance — clearer code, fewer dependencies, better tests, simpler architecture — pays back at 3–4× against itself.
The dollar of effort you spend writing tests today is buying you 3–4 dollars of maintenance later. Refusing to write them is borrowing at a usurious rate.
§ IIITotal Cost of Ownership16 / 26
The unit of comparison we will use all term
Total Cost of Ownership (TCO).
TCO = Σ (all cash outflows over the asset's life), discounted to present value
A TCO calculation is honest only if it includes every category we have named so far — fixed and variable, direct and indirect, non-recurring and recurring — across all five lifecycle phases, including the opportunity cost of the engineers' time.
It is the standard unit of comparison whenever we choose between alternatives. Lecture 5 will turn this into a numerical decision. Today we just learn to construct the worksheet.
If two alternatives look similar but you can only compare one phase, you are not comparing TCO — you are comparing convenience.
Part — Four
IV
The 2026 cost stack — line items that didn't exist when Boehm wrote.
A modern software product has cost categories that no classical model anticipates.
§ IVThe 2026 stack18 / 26
Six new categories every modern budget needs
The new line items.
Category
Examples (2026)
Behaviour
Compute (cloud)
AWS / GCP / Azure VMs, autoscaling.
variable, OpEx
GPU / accelerators
H100, Blackwell, TPU v6, Trainium.
mixed — capex or rented
LLM API / tokens
Anthropic, OpenAI, Google, Chinese vendors.
variable, OpEx
Vector DB
Pinecone, Weaviate, pgvector, Turbopuffer.
recurring
Inference cache layer
Redis LangCache, semantic caches.
recurring, cost-saver
AI observability
Arize, Langfuse, Helicone.
recurring
A startup's pitch deck that lists "infrastructure" as a single number is not yet thinking about the AI cost stack.
§ IVWorked numbers19 / 26
A concrete AI feature cost build-up
Anatomy of a 100,000-task-per-month AI agent.
Line item
Per task
Monthly
Type
LLM input tokens (8K avg)
$0.020
$ 2,000
variable
LLM output tokens (2K avg)
$0.030
$ 3,000
variable
Tool / API calls (avg 4)
$0.008
$ 800
variable
Vector retrieval
$0.002
$ 200
variable
Observability / log storage
—
$ 500
fixed
Engineer time (0.5 FTE on-call)
—
$ 8,000
fixed, indirect
Total
$0.145
$ 14,500
—
Prompt caching alone could cut the input-token line by 80% — saving $1,600/month, or 11% of TCO. We will quantify caching in Lecture 13.
Part — Five
V
Hands-on — build a 5-year TCO worksheet before you leave.
In pairs, ten minutes. Bring it back next class.
§ VTCO worksheet21 / 26
Your starting template
Five years, ten categories, one decision.
Category
Y1
Y2
Y3
Y4
Y5
Development (non-recurring)
—
—
—
—
—
Cloud compute
—
—
—
—
—
LLM tokens / GPU
—
—
—
—
—
SaaS / licences
—
—
—
—
—
Engineering (FTE × salary)
—
—
—
—
—
Indirect (security, IT, etc.)
—
—
—
—
—
Opportunity cost (engineer)
—
—
—
—
—
Retirement / migration
—
—
—
—
—
Pair up. Pick a project from your job — real one preferred. Fill in any cells you can defend. We will not discount today (that's Lecture 3); just get the cash flows down.
BridgeTo Lecture 322 / 26
Where this is going
Tomorrow we discount.
Today you have learned to name every cash flow. But $1 spent in Year 5 is not the same as $1 spent today. Tomorrow we make that explicit.
In Lecture 3 we introduce Present Value and Future Value, the time-value of money, and the discount-rate machinery that turns the worksheet you just built into a single comparable number.
Bring your TCO worksheet to Lecture 3 — we will discount it together as the first warm-up.
HomeworkDue Lecture 423 / 26
Homework 02 — due Saturday 6 June, before class
Build a complete TCO.
Choose a software product or feature you have personally worked on (or are working on). It can be commercial, internal, or a side project.
Build a complete 5-year TCO worksheet. Include every category from today's lecture. Show your assumptions explicitly — they will matter more than the numbers.
In one paragraph, state which category surprised you the most when you wrote it down — and why.
Submit as a single PDF + the spreadsheet to /submissions/HW2/<your-name>/ in the course Git repository.