DEV Community

Scarab Systems
Scarab Systems

Posted on

Scarab Diagnostic Field Test #034 - Prometheus Docker Swarm Label Source Boundary

Target: prometheus/prometheus

Issue: prometheus/prometheus#12244

Pull request: prometheus/prometheus#18979

Field Lab record: Prometheus #12244

This field test was about documentation, but not "just documentation."

The public issue reported confusion around Docker Swarm task discovery metadata in Prometheus.

The user expected __meta_dockerswarm_container_label_<labelname> to expose labels from the running container or image, such as OCI image labels.

The visible behavior was different.

The metadata existed, but the source was not the runtime container or image label surface the user expected.

That made this a boundary problem:

what does the documented metadata name mean, and which public source of truth does it actually represent?

Field Lab record

The public Field Lab record for this case is here:

https://github.com/scarab-systems/scarab-field-lab/tree/main/field-tests/prometheus-prometheus-12244

The record includes the public issue, the public pull request, a public comment, the repair scope, validation summary, and non-claims.

It contains public links, status, validation, and claim boundaries only. It does not contain SDS source code or non-public product material.

The case record is only the public evidence layer.

SDS result

SDS surfaced a documentation boundary around Docker Swarm task discovery.

The important finding was not that Prometheus should add a new Docker API call.

It was not that service discovery should begin inspecting image labels.

It was not that runtime behavior should change.

The finding was smaller:

Prometheus documentation described a metadata label in a way that could reasonably be read as "labels from the container," while the public code path populates that metadata from the Swarm task's container spec.

That difference matters because service-discovery metadata is used as an operational contract.

People write relabeling rules against these names.

If the source of a metadata field is ambiguous, users may look for data in the wrong place, configure the wrong thing, or assume a missing label is a Prometheus bug when it is really a source-boundary mismatch.

Failure shape

The failure shape was a mismatch between expectation and source.

The public issue described a reasonable operator expectation:

if the metadata name says container_label, it might refer to labels visible on the running container or inherited from the image.

But Docker Swarm task discovery has a more specific source.

The relevant metadata is populated from the task's container specification.

That is close enough in language to sound obvious after the fact, but not close enough for an operator reading the docs while debugging a production scrape configuration.

That is why the field test did not treat this as a runtime bug first.

The problem was not proven at the behavior layer.

The problem was at the wording boundary:

the docs did not make the source specific enough.

Boundary

The boundary in this field test was:

Docker Swarm task metadata should distinguish labels from the Swarm task container spec from labels on a running Docker container or image.

Those are different surfaces.

They may overlap in everyday language.

They may even contain similar-looking labels in some deployments.

But they are not the same source of truth.

For service discovery, that distinction is important because a metadata label name becomes something users build configuration around.

If the docs leave the source implicit, the user has to reverse-engineer the behavior from Docker API responses and Prometheus discovery code.

That is not a good boundary for public configuration documentation.

What changed

The repair is a one-line documentation clarification in the Prometheus configuration docs.

The pull request is here:

https://github.com/prometheus/prometheus/pull/18979

The changed public file is:

  • docs/configuration/configuration.md

The repair clarifies that the Docker Swarm task metadata label comes from the Swarm task's container spec.

That is the whole patch.

No runtime behavior change.

No new Docker discovery logic.

No new Docker API calls.

No attempt to make Prometheus inspect image labels.

Just a clearer public contract for the metadata that already exists.

Why this was not a code repair

One of the risks with code-agent work is that everything starts to look like a code patch.

This field test is a useful counterexample.

The public issue was real.

The confusion was real.

The operator experience was real.

But the repair boundary was documentation.

That matters.

If a code agent had treated this as "missing labels," it could have reached for runtime behavior: inspect more Docker surfaces, add a new metadata path, or change discovery behavior.

That would have been a much larger claim.

The narrow claim here is simpler:

the existing behavior should be documented with a more precise source.

That kind of restraint is part of the theory I am testing with Scarab:

the quality of the repair depends on finding the correct boundary before editing.

Sometimes the correct boundary is code.

Sometimes it is tests.

Sometimes it is docs.

In this case, it was docs.

Why the diagnostic result mattered

The larger point of this field test is not that a documentation line changed.

The larger point is that a code agent should not decide the repair shape from momentum.

It should not start from:

"what can I change?"

It should start from:

"where did the repo's public truth become unclear?"

For this case, the public truth was about the meaning of a service-discovery metadata label.

The failure was not that the metadata name existed.

The failure was that the name could point readers toward the wrong source.

Once the boundary was clear, the patch became small.

That is the pattern Scarab is trying to make repeatable:

find the truth,

find the boundary,

patch the boundary.

Validation

The repair was validated against the Prometheus repository with the relevant public checks recorded in the Field Lab.

Validation recorded in the Field Lab:

  • go test ./discovery/moby -count=1 passed
  • go build ./cmd/prometheus/ passed
  • GO_ONLY=1 make test passed
  • make lint passed
  • make test passed

At the time this draft was prepared on June 19, 2026, the public pull request was open and ready for review.

The public status is:

  • PR open
  • DCO passing
  • Netlify deploy-preview successful or informational
  • upstream review required

This field report does not claim upstream acceptance.

It claims a public diagnostic record, a narrow documentation repair, and a submitted PR.

Maintainers decide whether the clarification belongs upstream.

Field test result

Result:

diagnostic proof and documentation repair submitted.

The field test produced:

  • a public issue-to-boundary record
  • a narrow documentation patch
  • validation against the affected repository
  • a public upstream PR
  • a public status record in the Scarab Field Lab

The patch is deliberately small.

That is the point.

When the failure is a source-boundary ambiguity, the best repair may be a sentence that makes the source explicit.

Public claim

This field test supports a narrow public claim:

SDS identified a Docker Swarm task discovery documentation boundary in Prometheus where container-label metadata needed a clearer source, and a human-submitted documentation repair was prepared for that boundary.

It does not claim:

  • that Prometheus accepted the patch
  • that Prometheus should inspect Docker image labels
  • that Docker Swarm discovery behavior changed
  • that Scarab repairs projects by itself
  • that SDS source or product details are public
  • that maintainers endorsed Scarab or the Field Lab

The Field Lab exists to keep those claims separate.

Disclosure

This field report was prepared with AI-assisted editing from public field-test notes, public issue and PR records, and the public Field Lab record. The diagnostic claim, repair boundary, and final wording were human reviewed.

Scarab Diagnostic Suite is proprietary. The Field Lab publishes public case records, issue links, validation summaries, and claim boundaries only.

SDS finds evidence. People make claims. Maintainers decide.

Top comments (0)