The QBR Transcript Pipeline: Sentiment as a Leading Indicator

The conversation that never gets followed through

Every CS leader I've worked with has at some point said "we should be doing more with our QBR recordings." And every one of them has then gone six months without doing more with their QBR recordings, because the path from "we have 47 recorded QBRs" to "we have a usable signal" is not obvious.

Vendors will tell you to buy their tool. The tools will give you a sentiment score per call. The sentiment scores will be useless — generic positive/negative/neutral classifications that don't survive five minutes of operator scrutiny. "The QBR was rated 'positive' but the customer just told us they're considering switching vendors next year — how is this useful?"

The pipeline below is different. It produces a delta — a quarter-over-quarter change in specific operator signals — and it produces it in a way that's tied to your business, your customers, and your renewal timing. The output is the sentiment delta input that feeds the Five-Signal Model from the previous piece.

The pipeline is roughly 200 lines of Python. The prompts are included verbatim in the article. It runs on a laptop in fifteen minutes per quarter for a 50-account portfolio. The hard part is not the code — it's avoiding the four pitfalls that make this useless if you build it naively. We'll spend more time on the pitfalls than on the code.

The four-step pipeline

Each QBR transcript goes through four steps. The output of step N is the input to step N+1. Each step is independently testable.

QBR audio file
    │
    ▼
[Step 1] Transcribe with speaker diarization
    │
    ▼
Structured transcript (JSON)
    │
    ▼
[Step 2] Extract signal phrases via LLM
    │
    ▼
Tagged phrases per signal category (JSON)
    │
    ▼
[Step 3] Score quarter-over-quarter delta per account
    │
    ▼
Sentiment delta per account (float, -1.0 to +1.0)
    │
    ▼
[Step 4] Aggregate into the Five-Signal Model's input
    │
    ▼
Trained model input (CSV)

The pipeline is deliberately modular. If your transcription tool changes (Whisper → AssemblyAI → Otter), only Step 1 changes. If you decide to extract a new signal category, only the prompt in Step 2 changes. The other steps are unaffected.

Step 1: Transcribe

The transcription tool you use doesn't matter much — Whisper, Otter, AssemblyAI, Rev are all serviceable. The pipeline only cares about three properties:

1. Speaker diarization. You need to know who said what. A QBR where the customer's CFO said "we're evaluating other options" is very different from one where your own AE said it as a hypothetical. Most modern transcription tools support this; verify before you commit.
2. Punctuation and capitalization. Raw transcripts without punctuation break the LLM extraction step downstream. Get this from your transcription tool, not by manual cleanup.
3. Timestamps. Each utterance should have a start time. You'll want this later when the operator needs to verify a specific quoted phrase.

The pipeline expects the transcript in a normalized JSON format. If your transcription tool outputs something different, write a one-time adapter:

# transcribe.py — normalize transcript to pipeline format
def normalize_transcript(raw_path: str) -> dict:
    """
    Output format:
    {
      "account_id": "acc_0001",
      "qbr_date": "2026-Q1",
      "duration_minutes": 47,
      "utterances": [
        {"speaker": "CUSTOMER_PRIMARY", "text": "...", "start_seconds": 0.0},
        {"speaker": "CSM", "text": "...", "start_seconds": 4.2},
        ...
      ]
    }
    """
    ...

The speaker field is normalized to a fixed vocabulary: CSM, CUSTOMER_PRIMARY, CUSTOMER_OTHER. Most transcription tools output speaker labels like "Speaker 1, Speaker 2" — you'll map those to your fixed vocabulary in the adapter. This mapping is the only step requiring human review of each transcript. Build a simple UI for it if your team is going to do this regularly.

Step 2: Extract signal phrases

This is where most pipelines go wrong. Generic sentiment analysis (positive / negative / neutral) is too coarse to be operationally useful. The pipeline extracts specific phrases that map to specific operator signals.

The five signals:

These five matter because they're predictive. A naive sentiment classifier reports current emotional state. These signals report trajectory — whether the customer is moving toward you or away from you.

The extraction prompt

You are extracting specific operator signals from a Customer Success QBR transcript.

The transcript follows. For each of the five signals listed below, identify all
verbatim phrases from the CUSTOMER speakers (not the CSM) that map to that signal.

A "verbatim phrase" is a direct quote from the transcript, not a paraphrase.
Include the speaker label and timestamp.

If a signal has no matching phrases, output an empty array for that signal.

THE FIVE SIGNALS:

1. roadmap_aligned: customer talking about expansion, multi-team rollouts, future use
   cases for your product, planned increased usage
2. roadmap_misaligned: customer reconsidering, evaluating alternatives, asking why they
   use you, considering reduced scope or vendor switch
3. operational_frustration: product issues, support problems, integration friction,
   recurring complaints
4. strategic_alignment: language treating your product as core/critical/foundational
   to customer's operation
5. strategic_distance: language treating your product as one of many tools, talk of
   consolidation, comparison to other vendors in their stack

OUTPUT FORMAT (JSON only, no preamble):
{
  "roadmap_aligned": [{"speaker": "...", "timestamp": "...", "phrase": "..."}],
  "roadmap_misaligned": [...],
  "operational_frustration": [...],
  "strategic_alignment": [...],
  "strategic_distance": [...]
}

CONSTRAINTS:
- Only extract verbatim quotes, not paraphrases.
- Only extract from CUSTOMER speakers (CUSTOMER_PRIMARY or CUSTOMER_OTHER).
- If unsure whether a phrase fits a signal, leave it out. False negatives are
  recoverable; false positives corrupt the downstream score.
- Do not include any phrase that's only ambiguously a signal.

TRANSCRIPT:
[paste transcript here]

The extraction prompt is unusually strict. False negatives (missing a real signal) are recoverable in the aggregate — you have many QBRs over time. False positives (mis-tagging a phrase) corrupt the downstream score in ways that take quarters to discover. The prompt explicitly trades off recall for precision.

Why this prompt works

Three patterns from the Prompt Library are visible here:

1. Specifies the audience twice. Once for the model's role ("you are extracting"), once for the output ("operator signals" — not "general business analysis").
2. Names the failure mode explicitly. Tells the model to err toward false negatives, not false positives. Most LLMs default to over-extracting; this instruction pulls them back.
3. Constrains the output format strictly. JSON only, no preamble, specific keys. Easy to parse downstream.

The full implementation:

# extract.py
import json
import anthropic  # or openai, etc.

EXTRACTION_PROMPT = """..."""  # the full prompt from above

def extract_signals(transcript: dict) -> dict:
    """
    Input: normalized transcript dict
    Output: dict with five signal categories, each containing list of phrases
    """
    transcript_text = format_transcript_for_prompt(transcript)
    prompt = EXTRACTION_PROMPT.replace(
        "[paste transcript here]", transcript_text
    )

    response = anthropic.Anthropic().messages.create(
        model="claude-sonnet-4-5",
        max_tokens=2000,
        messages=[{"role": "user", "content": prompt}],
    )

    raw_output = response.content[0].text.strip()
    # Strip markdown code fences if model added them
    if raw_output.startswith("```"):
        raw_output = raw_output.split("```")[1]
        if raw_output.startswith("json"):
            raw_output = raw_output[4:]

    return json.loads(raw_output)

The function returns a dict with five keys (the signal categories), each mapping to a list of extracted phrases. Save this output per QBR. You're going to want it for the next step and for the operator who later asks "why did the model say this account was at risk?" — the verbatim phrases are the audit trail.

Step 3: Score the deltas

The score is the change between this quarter's QBR and the previous quarter's QBR for the same account. Levels are misleading; deltas carry signal.

An account that's always at 5 operational-frustration phrases per QBR is stable. An account that jumped from 1 to 5 over one quarter is in trouble. The level alone (5) doesn't distinguish those two cases. The delta (+4) does.

The scoring formula

For each account, compute the sentiment delta as a weighted sum of signal-count deltas:

# score_deltas.py

SIGNAL_WEIGHTS = {
    "roadmap_aligned":         +1.0,
    "strategic_alignment":     +1.2,
    "roadmap_misaligned":      -1.5,
    "strategic_distance":      -1.0,
    "operational_frustration": -0.8,
}

def compute_sentiment_delta(current_signals: dict, previous_signals: dict) -> float:
    """
    Compute weighted delta between two consecutive QBRs.
    Returns a value in roughly [-3.0, +3.0], clamped and normalized.
    """
    delta = 0.0
    for signal, weight in SIGNAL_WEIGHTS.items():
        current_count = len(current_signals.get(signal, []))
        previous_count = len(previous_signals.get(signal, []))
        signal_delta = current_count - previous_count
        delta += weight * signal_delta

    # Normalize to roughly [-1.0, +1.0]
    return max(-1.0, min(1.0, delta / 5.0))

The weights deserve discussion. Three notes:

The negative weights are larger than the positive weights. This is intentional. Operationally, signs of trouble matter more than signs of strength because trouble is harder to recover from than strength is to build on. A customer who adds a roadmap- aligned signal this quarter is mildly encouraging; a customer who adds a roadmap-misaligned signal is a five-alarm fire.

Strategic-alignment carries higher weight than roadmap-aligned. Customers talk about expansion all the time (it's polite QBR language). Customers calling your product "core" or "foundational" is a much stronger signal of stickiness. Weight accordingly.

Operational frustration is weighted least. Counterintuitive but correct. Customers complain about products they use heavily. A high operational-frustration score from a deeply-engaged customer is a maintenance signal, not a churn signal. The other signals carry more predictive weight than the complaint count.

You will want to tune these weights for your business. Start with the values above. After 3-4 quarters of data, fit them via regression against actual outcomes.

What the score means

The output is a float in roughly [-1.0, +1.0]:

• +0.4 to +1.0: Account is meaningfully more aligned this quarter than last. Lean in — expansion conversation, executive engagement.
• +0.1 to +0.4: Mildly positive. Continue current strategy.
• -0.1 to +0.1: Stable. No action needed beyond standard cadence.
• -0.1 to -0.4: Mildly negative. Increase touch frequency, surface concerns in next CSM 1:1.
• -0.4 to -1.0: Meaningful negative shift. Escalate to manager, consider executive intervention.

These bands are calibrated to the weights above. Adjust if you adjust the weights.

Step 4: Aggregate into the Five-Signal Model input

The final step is plumbing — write the deltas to the format the Five-Signal Model expects.

# aggregate.py
import pandas as pd

def build_model_input(account_deltas: dict, account_metadata: dict) -> pd.DataFrame:
    """
    Combine sentiment deltas with the other four signals to produce
    the Five-Signal Model's input CSV.

    account_deltas: dict mapping account_id -> sentiment_delta float
    account_metadata: dict mapping account_id -> other features
    """
    rows = []
    for account_id, sentiment_delta in account_deltas.items():
        metadata = account_metadata.get(account_id, {})
        rows.append({
            "account_id": account_id,
            "usage_depth":        metadata["usage_depth"],
            "sponsor_engagement": metadata["sponsor_engagement"],
            "support_friction":   metadata["support_friction"],
            "sentiment_delta":    sentiment_delta,  # from this pipeline
            "renewal_proximity":  metadata["renewal_proximity"],
        })
    return pd.DataFrame(rows)

The other four signals come from different data sources (product telemetry, CRM, support ticket system, billing). This pipeline only produces one of the five inputs — the sentiment delta. But it produces it in a way the model can consume directly.

The four pitfalls

The code above is straightforward. The pitfalls are where teams ruin this.

Pitfall 1: Treating sentiment as a level, not a delta

The first time you run this pipeline, you'll be tempted to look at a QBR with five "operational frustration" phrases and conclude the account is in trouble. Don't. Some accounts have always had five operational frustration phrases per QBR. Their CFO is just direct. The signal is the change from their previous QBR, not the absolute count.

This is the single most common implementation failure. Vendors selling sentiment-analysis tools sell levels because levels are easier to demo. Operators using levels make worse decisions than operators using nothing. Only deltas survive contact with reality.

The rule: an account's first QBR in the pipeline produces zero signal. You can't compute a delta from one data point. Plan for this — the pipeline becomes useful in quarter 2, not quarter 1.

Pitfall 2: Generic sentiment instead of operator-specific signals

The temptation is to use an off-the-shelf sentiment classifier ("Hugging Face has one!") and skip the LLM extraction step. The classifier gives you a sentiment score. The sentiment score will be useless.

Why? Because "we're considering rolling this out to two more business units" and "we're considering whether to consolidate vendors" both register as similar mild-positive sentiment on a generic classifier. To your customer, they're opposite signals. The operator-specific signal vocabulary (roadmap-aligned vs roadmap- misaligned) catches the distinction that generic sentiment misses.

The five signals above aren't the only valid taxonomy. They're the one I've validated against real CS outcomes. If you want to add a sixth signal (e.g., "champion-departure language"), do it — but commit to the discipline of operator-specific categories rather than reverting to generic sentiment.

Pitfall 3: Ignoring who said what

A QBR transcript contains utterances from CSMs, AEs, customer primary stakeholders, and often customer secondary attendees. The pipeline must only extract signals from customer speakers. A CSM saying "so you're considering expanding to other teams?" is not a roadmap-aligned signal. It's a CSM asking a question.

The diarization step is critical. If your transcription tool produces unreliable speaker labels, the extraction step needs an extra prompt instruction: "if speaker identity is ambiguous, do not extract the phrase."

This pitfall is especially dangerous because it produces plausible-looking output. The signals look real. The model trained on them performs worse than it should. The CSM team blames "the AI" without realizing the input is garbage.

Pitfall 4: Trusting a single quarter's signal

Even with the delta framing, a single quarter's signal is noisy. A customer who had three roadmap-misaligned phrases this quarter and one last quarter might be in serious trouble — or might have had one unusually candid stakeholder in the room this time.

Treat the sentiment delta as one input among five. The Five-Signal Model is structured exactly this way for this reason. Never let an account be escalated on a single quarter's sentiment delta alone. Combine across quarters (the rolling-3 average is a reasonable smoother) and combine with other signals (usage depth, support friction).

The pipeline produces a signal. The model produces a forecast. The CSM produces a decision. The pipeline is the start of the chain, not the end.

The repo

The companion repo contains:

qbr_pipeline/
├── README.md
├── requirements.txt
├── pipeline.py              # End-to-end orchestration
├── transcribe.py            # Step 1: transcript normalization
├── extract.py               # Step 2: LLM signal extraction
├── score_deltas.py          # Step 3: delta computation
├── aggregate.py             # Step 4: model input building
├── prompts/
│   └── extraction.txt       # The verbatim extraction prompt
├── sample_data/
│   ├── transcript_q1.json   # Sanitized sample transcript, Q1
│   └── transcript_q2.json   # Same account, Q2 (for delta demo)
└── tests/
    └── test_score_deltas.py # Unit tests for the scoring math

The Google Sheets companion is for non-Python operators who want to test the methodology before committing to building the pipeline. It accepts a transcript paste-in, walks the user through manual extraction with the five-signal categories, and computes the delta from a previous-quarter score. Slower than the Python pipeline but produces the same output.

The point of shipping both is that building the discipline matters more than building the code. CS Ops leads with Python can stand up the pipeline. CSMs without Python can run the manual version. Both produce the same signal. The signal is the artifact.

What the pipeline gets you

In quarter 1: nothing. You're collecting the baseline.

In quarter 2: a delta per account. Two accounts will register significant negative deltas. One of them will be a real signal; the other will be noise. You won't yet know which is which. Don't act yet.

In quarter 3: deltas across three data points per account. Now you can compute trends. Accounts with two consecutive negative deltas are real signals. Surface them. Bring them into your CRO conversations.

In quarter 4 and beyond: the sentiment delta is feeding the Five-Signal Model. The model is predicting churn risk with sentiment delta as one of five inputs. The model's predictions are being overridden by CSMs at the account level. The override-tracking table is starting to fill in.

The pipeline takes a week to build. The signal takes a quarter to validate. The model takes a year to mature. The discipline pays for itself for the rest of your career.

The end of the slate

This piece is the last of the six pieces in the AI in CS pillar.

The Decision Matrix taught us how to classify every CS task by the role AI should play.

The Prompt Library gave us eight working prompts for the safe quadrants.

The AI Safety Rules established the discipline floor.

Four CS Copilots, Tested taught us how to evaluate vendor tools without getting sold.

The Five-Signal Model built the predictive layer that produces a CRO-defensible forecast.

The QBR Pipeline built the input layer for that model — the one signal that vendors can't sell you because it has to be built on your own data.

The pattern across all six is the same: the operating model is the artifact. The matrix isn't the answer; using it correctly is. The prompts aren't the answer; adapting them is. The model isn't the answer; the discipline of building, calibrating, and overriding it is. The pipeline isn't the answer; running it for four quarters is.

AI in Customer Success is not a vendor procurement problem. It is an operating model problem. The slate above is the operating model.

Build it. Ship it. Show your CRO. The next person reading this paragraph will be your replacement, in five years, looking back at what you built. Make it good.

Download the QBR Pipeline repo — working Python code, sample transcripts, and the extraction prompt. Eleven passing tests, runs end-to-end.