Pipeline, ingestion and retrieval internals

1. 1 · Source selection

   Allowlisted first-party and sector-body domains only. The allowlist lives server-side in the capture function; arbitrary URLs are rejected before any fetch happens.

2. 2 · Firecrawl capture

   Read-only scraping of robots-permitted endpoints. Canonical URL, final URL, HTTP status, retrieval timestamp and SHA-256 content hash are all retained. No carts, forms, logins or paywalls, ever.

3. 3 · Deduplication

   Canonical URL and content hash are compared before anything is embedded. Two URLs serving identical bytes embed once.

4. 4 · Chunking

   Heading-aware chunks with section paths preserved, so a retrieved passage can be located in its document.

5. 5 · Voyage embeddings

   Generated server-side only. Every returned vector is dimension-checked before any content write; a short or malformed vector fails the whole job rather than half-writing it.

6. 6 · Atomic write

   Document and chunks are written in one transaction. A partial ingest is not a possible state.

7. 7 · Claim extraction

   Material statements become claims with a fixed status vocabulary and explicit source references.

8. 8 · Graph

   Entities are shared across markets and never duplicated. Edges are dataset-scoped and carry the claim that supports them.

9. 9 · Retrieval

   Query embedded with the same Voyage model, matched against dataset-scoped chunks, returned with provenance.

Verified · primary

Established directly from a primary source or from our own reproducible measurement of it.(22 in corpus)

Verified · independent

Corroborated by an independent third party that is not the subject of the claim.(11 in corpus)

Observed on retail surface

Directly observed on a retail or DTC surface at a stated capture date. Factual about that surface at that moment — not a market-wide fact.(6 in corpus)

Company claim

Asserted by the subject company about itself. Never upgraded without independent corroboration.(22 in corpus)

Analyst estimate

Our own derived figure. The method is stated alongside it.(2 in corpus)

Unresolved

Sources conflict, or evidence is insufficient. Not to be presented as settled.(6 in corpus)

Method · A company_claim is never promoted to verified on the strength of repetition. Only independent corroboration moves it, and the corroborating source is recorded when it does.

• No brand-level market share, either market. Production and export volumes exist; brand share does not. Every share figure in this category's public reporting is modelled, and none is ingested here.
• No Italian market metrics. The Italian dataset holds competitor and positioning evidence but no quantified category data. Italian views show insufficient-evidence states rather than borrowing Spanish numbers.
• No campaign or media-performance data. Communications analysis is limited to published first-party copy. Nothing in the corpus speaks to reach, spend or effect.
• DTC pricing is not shelf pricing. Price observations come from brand-owned storefronts. They are strong evidence of intended positioning and weak evidence of grocery reality, where the volume is.
• Longitudinal depth is one cycle for most brands. Catalogue deltas need two captures of the same brand. A single capture is a hypothesis; a structural pattern needs three agreeing cycles.

• The ANFACO functional-foods article (NEUROMICS cognitive-health research network) was captured but the extraction is dominated by cookie-consent boilerplate rather than article body. The page is ingested and searchable, but should not be treated as a reliable read of the research programme until re-captured.

  claim:es:anfaco-capture-quality:127 · Observed directly in the stored capture. A pipeline extraction issue, not a source problem — logged rather than silently relied on.

• The sustainable line is reachable at /marcas/consorcio-linea-sostenible/ in the 2026-08-02 sitemap and recorded, while the 2026-07-31 homepage recorded linked it as /marcas/consorcio-linea-eco/. Apparent rename from "Línea ECO" to "Línea SOStenible"; the eco path was not re-tested in this cycle.

  claim:es:linea-sostenible-naming-drift:122 · Two readings of the site on different dates disagree on the path. A single pair is a hypothesis, not a confirmed rebrand — it needs a third observation or an archived page to settle.

• Serrats' 0.0% promotion rate is a measurement blind spot, not a commercial fact: its product data does not expose a comparison price, so discounts are invisible to us rather than absent. Any competitive promotion comparison that includes Serrats at zero is wrong.

  claim:es:serrats-promo-blindspot:110 · High confidence that the blind spot exists; no confidence at all in the 0.0% figure.

• Spanish conservas turnover ladder is contested and must not be published as a single ranking. Ranking-Empresas: Frinsa del Noroeste S.A. EUR 625,863,433 (#1 sector), Jealsa Foods S.A. EUR 553,365,084 (#2). Mercasa 2026 reports a group-level ladder of EUR 781M / 741M / 727M / 320M. Only the 727M is identifiable (Nauterra). Entity-vs-group scope differs.

  claim:es:turnover-ladder-contested:105 · Sources disagree; reconciliation impossible from public data.

• Callipo anniversary figures conflict: evidence records ">EUR 100m for the first time in 112 years" (2024) while the company site states "sono passati 110 anni" against a 1913 founding (implying 111 in 2024). Logged rather than reconciled.

  claim:it:callipo-anniversary-conflict:211 · Minor discrepancy between two company statements; likely stale site copy, but unresolved.

• ~99% household penetration for conservas in Spain is UNVERIFIED; the circulating 97% figure refers to Peru. Pending a Kantar Spain category report.

  claim:market:penetration:unresolved · No verifiable Spanish source found this cycle