Every data contract pitch lands the same way: one YAML file, agreed between producer and consumer, versioned in Git, executable. The file genuinely is better than the wiki page it replaced. The trouble starts about two weeks after adoption, when someone asks what happened when the freshness SLA was missed on Sunday — and the answer turns out to be nothing, because nothing was watching. In this edition of datapro.news, we cover the question the announcements skipped. ODCS v3.1.0 shipped in December 2025, the competing Data Contract Specification was deprecated by its own maintainers, and Bitol graduated at LF AI & Data in July 2026. Authoring in ODCS is now the boring, correct default, which is the highest compliment a standard can receive. But ODCS is a declarative specification, not an execution engine — it defines the what and deliberately leaves the how and the when to whatever system processes the data. Quality rules describe what should be true; something else has to go and check. The word doing the most work in "executable data contracts" is not contract. Here are the 3 layers the decision actually comes down to: - 📐 ODCS v3.1.0: Learn what actually shipped — relationships that hold even where the store enforces nothing, SLAs that carry a schedule, a registered media type — and the asterisk on the backward-compatibility claim, because "no migration required" and "run the linter before you upgrade CI" are different instructions and only one of them is accurate. - 🏛️ Bitol governance: Discover why graduation is the fact that should drive a procurement decision rather than any feature list, since it is a checklist and not a sentiment — plus the two things to verify yourself, including a foundation page that still says incubation and adoption figures published by the standard's own chair. - ⚙️ The enforcement layer: See why choosing ODCS is now low-risk and choosing what runs it is not. Soda Core changed licence in January 2026, GX Core stewardship passed to Fivetran in May, and the dbt Labs merger closed on 1 June. The open spec consolidated at exactly the moment the engines beneath it did.

Every RAG project starts the same way: a few hundred PDFs, a chunking strategy copied from a blog post, an off-the-shelf vector store. The demo genuinely works and the budget clears. The trouble starts a quarter later, when the business stops asking about documents and wants the model to review the security footage. In this edition of datapro.news, we cover the rewrite nobody scoped. Unwinding a text-only stack is a real architectural project with real money behind it — but the reference diagram circulating to describe it (lakehouse, Kafka, Flink, Materialize, done) does not survive contact with the documentation of its own components. Three of the four boxes are doing jobs they do not do. Here are the 3 components the design actually turns on: * 📼 Apache Kafka: Learn why "Kafka ingests the video stream" tells a design review you have not sized a broker, and the pattern that already has a name you should be using instead. * ⚡ Apache Flink: Discover why the part of the pitch that sounds most like vapour became true most recently — plus the two footnotes the marketing omits, one of which means you will pay for some tokens twice. * 🧮 Materialize: See why listing it as a way to compute embeddings is the tell that a diagram was assembled from vendor landing pages, and where it genuinely belongs instead. We also get specific about the distinction vendors blur between Lance and LanceDB, why "batch is deprecated" is contradicted by the flagship deployment of this very architecture, and the compliance citation that will get your design second-guessed by legal. The issue closes with a scorecard: what each component does, and what to check before you commit. Parsing a PDF is a commodity, and it was one before this wave. The scarce thing is a system that embeds a live stream without a bespoke microservice holding it together and joins the result against governed relational data. That is buildable in 2026 — just not from the diagram everyone is drawing. Get the boxes right first.

Every Iceberg pitch lands the same way: your data, your object storage, an open format, queryable from anywhere. The demo genuinely works. The trouble starts about two weeks into production, when someone asks who is allowed to see column seven — and the answer turns out to live somewhere that isn't open at all. In this edition of datapro.news, we cover the fight nobody announced. A directory of Parquet files in a bucket is inert; it is storage, not a database. Something has to tell an engine which metadata pointer is the current valid state of a table, and whether the querying principal may read it. That something is the catalog. By commoditising the file format, the industry did not eliminate the vendor lock-in it spent a decade complaining about — it relocated it, from the storage layer where it was visible and much-discussed to the governance layer where it is neither. Your Parquet files stay portable. The 5,000 grants, masking rules and policy tags you authored do not. Here are the 3 catalogs the decision actually comes down to: - 🧭 Apache Polaris: Learn why graduating to Apache Top-Level Project in March 2026 is worth more to you than any feature comparison, and why the neutrality you get with it is also an on-call rota — self-hosting means owning HA for the one service every pipeline and dashboard in the company stops without. - 🌿 Project Nessie: Discover the most elegant idea in the field — Git branching, tagging and merging across your entire lakehouse state — plus the due-diligence detail that should stop you adopting it as a destination, because its own sponsor has stated it will fold those capabilities into Polaris and retire the project. - 🔗 Unity Catalog: See why the criticism you have probably repeated in an architecture review is now out of date, and where the real asymmetry sits instead: outbound it is a well-behaved Iceberg REST server, inbound a reluctant client, and that is a design choice rather than a missing feature.

Every AI pipeline works in the demo. The trouble starts the day real users arrive. Response times crawl as requests queue behind one another, the monthly API bill posts a number your manager forwards upward with a single question mark attached, and somewhere in the middle of it the system begins returning confident nonsense that nobody can date, because nothing was watching. In part 3 of the datapro.news series, we are covering the half no tutorial touches. None of those failures are model failures. They are operational ones, and they are where most pipelines quietly die on the road from a successful proof of concept to a system a business will actually rely on. Getting a demo running is the easy twenty per cent. Serving it under load, governing what it spends and being able to see inside it are the other eighty, and not by accident, they are exactly the parts a managed black box keeps out of your hands. Here are 3 open-source tools that give them back: - 🖥️ Ollama: Learn which three environment settings actually govern your throughput, and why its architecture makes time-to-first-token climb with concurrency, so you know the precise moment to graduate to a production serving engine instead of finding out during a spike. - 🚦 LiteLLM: Discover how one OpenAI-compatible gateway gives you per-key spend caps, load balancing and automatic failover, plus the async-logging detail that separates a gateway which scales from one that becomes your bottleneck. - 🔍 Langfuse: See why capturing input and output is not enough for a non-deterministic system, and how tracing the retrieval, reasoning and tool calls in between lets you debug a bad answer by replaying it and catch drift before a user does. Each one hands back a different piece of what a platform hides (latency, spend or truth), and the issue closes with a side-by-side table of what each tool does in production and what to watch out for. Before you hand the whole stack to someone else's dashboard, you need to decide whether "we think it is fine" is good enough, or whether you want to be able to see that it is. A demo proves your pipeline can work. Production proves it can be trusted. Only one of those keeps the lights on.

Most AI prototypes hit their ceiling long before the model runs out of capability. When the output quietly changes shape in production, everyone reaches for a bigger model, but that gap between "it worked yesterday" and "genuinely dependable" is almost never a capability problem. In part 2 of the datapro.news series, we are breaking down why the prompt string is the least trustworthy component in your entire stack. Hardcoded instructions are the only part of a modern pipeline with no tests, no types and no version history, meaning every fix is a guess and every provider update is a silent regression you find out about from a customer. Here are 3 open-source tools that we think change the game: - 🧩 DSPy: Learn how to treat prompts as compiled artifacts rather than text, so when a new model drops you recompile against a small validation set instead of hand-editing dozens of brittle strings. - 📐 Instructor: Discover how to define your output as a Pydantic schema and let an automatic retry loop catch the malformed JSON and missing fields before they ever reach your database. - 🔒 Outlines: See how constrained decoding makes invalid output structurally impossible rather than merely unlikely, so enum values, ISO dates and numeric totals come back correct every single time. Each one buys reliability with a different currency - compile time, latency or infrastructure - and the issue closes with a side-by-side table so you can choose on your constraints rather than on hype. Before you spend another week tuning wording, you need to look hard at whether your instructions are code or just text. Otherwise you don't have a product, you have a demo with good manners. Check out the video edition below 👇