K² Agent Context Demo: Java R&D Case Study

One request, fully grounded

Same external coding agent. Better context before the edit.

Developer promptWhich guide rules, source files, and tests should I use before adding includeMissing to JobVertexWatermarksHandler?

What K² adds before the edit

The context package tells the agent what to use and why.

Follow this ruleREST handler checklist

Private guide. Use message parameters and preserve compatibility.

Use these docsFlink 2.2 REST API

Versioned docs. Pin the endpoint behavior to the right release.

Mirror this patternJobVertexWatermarksHandler and headers

Source precedent. Follow the existing handler and response-body shape.

Extend this testJobVertexWatermarksHandlerTest

Focused test. Verify the new flag before claiming success.

Protect this behaviorDefault response stays unchanged

Guide plus source precedent. Only change behavior when includeMissing=true.

The agent still searches and inspects code. K² adds the private guide, versioned docs, source anchors, test anchor, and review constraint as cited context before the edit.

Patch outcomeAdded the query parameter and extended the focused handler test.

Added includeMissing
Introduced JobVertexWatermarksMessageParameters
Extended handler test

4 tests0 failures • 0 errors

The core insight: source-role-separated context

Coding agents need to know whether retrieved text is a rule, an API contract, implementation precedent, or a test expectation. K² preserves that role through collections, metadata filters, named agents, and citations instead of flattening every source into one undifferentiated prompt.

Guides set constraintsConfluence guardrails, review rules, migration notes, and team conventions.

Docs set API contractsVersion-pinned public or private documentation for framework behavior.

Code shows precedentClasses, packages, handlers, message types, and neighboring implementation patterns.

Tests define acceptanceFocused checks and reviewable verification commands before a patch is claimed.

Source-role-separated context architecture

K² keeps guides, docs, code, and tests separated, asks bounded agents in a declared order, and lets a Knowledge Feed promote repeated source findings back into durable guidance.

How the context reaches the agent

Private knowledge becomes a cited context package before the edit.

Guides, docs, source, and tests stay separated through retrieval, then K² assembles the relevant pieces into a cited package the coding agent can use before it changes code.

Customer knowledgeGuides, docs, code, tests

K² platformCollections, Agents, Feed, Pipeline

MCP serverScoped evidence with citations

Coding agentPlans, edits, and explains

Java patchFocused tests and review trail

Two ways to act on the demo

Developers need a short reproducible setup. Enterprise buyers need a controlled pilot against their current coding-agent workflow. The same architecture supports both paths.

Developer quickstartTry in 10 minutes

Load the public demo bundle, connect the MCP server to a coding agent, and run one cited retrieval query.

Enterprise pilotRun a pilot on your codebase

Freeze tasks and scoring with the customer technical lead, ingest customer docs/code, and compare against their current workflow.

Pilot ask

The customer-facing claim should be earned on customer assets. Use this public benchmark to scope a bounded replication, not as a forecast for a financial Java application.

Enterprise path

Run the same workflow on customer assets.

Pick one representative Java module and one Confluence page tree.
Freeze 5-10 real feature-development tasks before the pilot run.
Run the same coding agent, for example Codex or Claude Code, with and without K² retrieval.
Score accepted patches, review rework, focused tests, token use, wall-clock time, and retrieval cost.

Design partners wanted

Validate on customer code before making any customer-specific claim.

This is a public benchmark and demo bundle, not a named customer replication.
Design partners should freeze tasks, expected files, guide checks, and scorer logic before indexing.
Publish customer-approved relevance findings, even if the customer name remains anonymized.

Add a Flink REST endpoint for checkpoint summary

The coding agent should identify guardrails, Flink 2.2 docs, Java implementation anchors, route registration, message classes, and tests before editing code.

Collections

Agents

Orchestration

Coding-agent output

Select a step or node to inspect how evidence moves from K² into the coding agent.Live smoke: 4 agent steps returned sources

Benchmark evidence, led by the ablated control

On the dimensions that exclude guide-compliance scoring, K² is narrowly ahead of the repo-only baseline and materially ahead of public-docs-only context. The full rubric then shows the additional lift from retrieving guide rules that the baseline does not have.

98 / 100K² MCP guardrail-ablated accepted patches

96 / 100repo-only baseline guardrail-ablated accepted patches

52 / 100Context7 public-docs MCP guardrail-ablated accepted patches

+65 ptsfull-rubric guide-compliance lift versus repo-only baseline

Full-rubric accepted patches were K² 96/100, repo-only baseline 31/100, and Context7 public-docs MCP 24/100. Read that gap as a guide-retrieval result: K² retrieved the same Confluence-style guardrails that the full scorer checks.

Token math is agent-side: retrieved snippets count once they enter the coding-agent prompt, but K² platform retrieval, ingestion, storage, and subscription costs are reported separately below. Do not quote this as a broad Context7 ranking or expected customer outcome.

Token economics

Same 100-task run, agent-side. Retrieved K² snippets count once they enter the coding-agent prompt; K² platform cost is reported separately in the methodology section below.

−27.1%Tokens per attempted task2.04M → 1.49M mean prompt + completion tokens.

−76.5%Tokens per accepted patch6.59M → 1.55M because K² produces more accepted work.

~$15.12Agent-token savings per accepted patchAt $3 / M input tokens, ≈5.8x the benchmark-scale K² platform allocation.

Token efficiency tracks accepted work, not only attempted work. The headline result is the 76.5% drop in tokens per accepted patch because K² produced 96/100 accepted patches versus 31/100 for the repo-only baseline.

Methodology and ablation

The circularity risk is explicit: K² retrieves guide rules and the full rubric rewards guide compliance. The table therefore reports the full score beside a guardrail-ablated pass rate, where guide-compliance failures are removed from pass/fail attribution.

Scoring rubric

Component	Weight
Focused tests + build verification	40%
Expected files/modules touched	25%
Required behavior or diff-pattern checks	15%
Confluence/internal guide compliance	10%
Review scope and safety	10%

Guardrail-ablated versus full pass rate

Arm	Guardrail-ablated score	Full score
K² MCPProject guides, source, tests, and versioned docs through K².	98 / 100	96 / 100
Repo-only baselineLocal checkout and model memory without an external context service.	96 / 100	31 / 100
Context7 public-docs MCPPublic documentation through Context7, without private guide/source/test corpora.	52 / 100	24 / 100

Authorship/freeze disclosure

The public artifact does not independently prove that task authors, guide authors, and scorer authors were blind to K² outputs before freezing. The defensible public claim is therefore narrower: K² improved this guide-retrieval-heavy benchmark, and customer-specific claims require a frozen customer replication before indexing or running either arm.

Cost model

Agent-token numbers count prompt and completion tokens captured by the benchmark runner. Retrieved K² snippets are included once they enter the agent prompt.
K² platform cost is not hidden in the token-savings number. It includes ingestion, retrieval queries, storage, and subscription.
Illustrative benchmark-scale platform allocation: Pro tier at $249/month for this demo corpus and run.
If full-rubric accepted patches are the customer-relevant outcome because guide violations create review rework, K² platform allocation is $249 / 96 = $2.59 per full-rubric accepted patch before model-token cost.
If the guardrail-ablated frame is used as the raw code-quality denominator, K² adds 2 incremental ablated accepted patches over the repo-only baseline, or about $124.50 per incremental ablated patch before model-token effects.
That is the point of reporting both frames: the public run shows a narrow ablated code-quality lead and a large guide-compliance/review-rework lead.
Break-even against the repo-only baseline on the full-rubric outcome occurs when blended agent-token price exceeds roughly $0.51 per million tokens saved.
At Claude Sonnet-style input pricing around $3 per million tokens, the 5.04M agent-token savings per full-rubric accepted patch is about $15.12, roughly 5.8x the benchmark-scale K² platform allocation.
These numbers do not include developer time. One avoided review or re-prompt hour at a $150 loaded engineering cost dwarfs the token and platform costs combined.

Full-rubric formula: K² cost per accepted patch = 1.55M agent tokens times the model-token rate plus $2.59 platform allocation; repo-only baseline = 6.59M agent tokens times the model-token rate.

The economic case should also count developer time. If K² saves even one review or re-prompt hour per accepted patch, the labor savings exceed both token cost and benchmark-scale platform allocation by a large multiple.

Reproducibility statement

Task definitions, scorer configurations, prompt templates, selected raw responses, patch artifacts, and demo asset manifests are published in the repository. An external reviewer can rerun the public benchmark with the same coding-agent model, a K² API key, and the published bundle; customer-specific claims still require customer-frozen tasks and customer-owned corpora.

RepositoryInspect the harness

Review the public scorecard inputs, scripts, selected answers, accepted patch diff, and K² asset bundle.

QuickstartReproduce one cited query

Load the demo corpora and verify that K² returns role-scoped evidence to an MCP-capable coding agent.

5 filesproduction/test files changed in accepted patch

2 new classesquery parameter and message parameters

4 testsfocused test run, 0 failures

K² MCParm: codex_with_k2_mcp

diff --git a/.../JobVertexWatermarksHandler.java b/.../JobVertexWatermarksHandler.java+import org.apache.flink.runtime.rest.handler.util.HandlerRequestUtils;+import org.apache.flink.runtime.rest.messages.job.metrics.IncludeMissingQueryParameter; +final boolean includeMissing =+        HandlerRequestUtils.getQueryParameter(+                request, IncludeMissingQueryParameter.class, false); -if (watermarkValue != null) {+if (watermarkValue != null || includeMissing) {     metrics.add(new Metric(id, watermarkValue)); } answer style:No retrieved K² source URI list; final response summarizes local files changed.

diff --git a/.../JobVertexWatermarksHandler.java b/.../JobVertexWatermarksHandler.java+import org.apache.flink.runtime.rest.messages.job.metrics.IncludeMissingQueryParameter;+import org.apache.flink.runtime.rest.messages.job.metrics.JobVertexWatermarksMessageParameters; -extends AbstractJobVertexHandler<MetricCollectionResponseBody, JobVertexMessageParameters>+extends AbstractJobVertexHandler<+        MetricCollectionResponseBody, JobVertexWatermarksMessageParameters> +final boolean includeMissingMetrics =+        request.getQueryParameter(IncludeMissingQueryParameter.class)+                .contains(Boolean.TRUE); -if (watermarkValue != null) {+if (watermarkValue != null || includeMissingMetrics) {     metrics.add(new Metric(id, watermarkValue)); }

new file: IncludeMissingQueryParameter.java
+public class IncludeMissingQueryParameter extends MessageQueryParameter<Boolean> {
+    private static final String QUERY_PARAMETER_NAME = "includeMissing";
+    public IncludeMissingQueryParameter() {
+        super(QUERY_PARAMETER_NAME, MessageParameter.MessageParameterRequisiteness.OPTIONAL);
+    }
+}

new file: JobVertexWatermarksMessageParameters.java
+public class JobVertexWatermarksMessageParameters
+        extends JobVertexMessageParameters {
+    public Collection<MessageQueryParameter<?>> getQueryParameters() {
+        return Collections.singleton(includeMissingParameter);
+    }
+}

test result:
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS

sources cited:
https://github.com/apache/flink/blob/release-2.2.0/.../JobVertexWatermarksHandler.java
https://github.com/apache/flink/blob/release-2.2.0/.../JobVertexWatermarksHandlerTest.java
https://nightlies.apache.org/flink/flink-docs-release-2.2/docs/ops/rest_api/
generated://guides/flink/rest-handler-checklist.md

Flink 2.2 REST docs used in corpusExternal source URL represented in the docs collection.Kafka 4.2 Connect docs used in corpusExternal source URL represented in the docs collection.Apache Flink source refExternal repository/ref represented by source and test documents.Coding-agent development flow documentPrompt, MCP setup, transcript command, evidence map, and readiness gate.Machine-readable flow exampleBaseline, K²-assisted flow, metadata filters, evidence, success criteria, and the run-customer-demo-flow transcript command.Context7 public-docs reference benchmarkThree-arm methodology and raw shard references. Interpret as a public-docs control, not a broad Context7 quality ranking.Real baseline Codex answerRepo-only answer for the same Java feature-development question. Archived arm: codex_repo_only.Real baseline Codex patch diffBaseline Java diff for the same includeMissing feature request. File: patch.diff.Real K²-assisted Codex answerExisting patch run answer with sources consulted and focused test result.Real K²-assisted accepted patch diffActual Java diff adding includeMissing, message parameters, headers wiring, and tests.

K² turns private knowledge into cited context for agents