Real Time Data Streaming for AI Startups

Every startup data stack starts as batch. Production data sits in Postgres, a sync runs every few hours or overnight, and the warehouse holds a copy of the world as it looked at 3 a.m. For most companies that is the right answer, and it stays the right answer for years.

Nobody notices the day it stops being right. The data does not get less accurate. The decisions being made on top of it just start happening faster than the sync.

Artie sells real time data streaming as the fix for that moment. Co-founders Robin Tang (CTO) and Jacqueline Cheong (CEO) went through Y Combinator in Summer 2023 and announced a $12 million Series A in January 2026, led by Standard Capital, the fund Dalton Caldwell and Paul Buchheit left Y Combinator in 2025 to start with co-founder Bryan Berg, with YC participating. The company says its customers now push more than 700 billion rows a year through the platform. The founders told the story on YC's Root Access.

The useful part is not the pitch. It is the numbers: what this actually costs to build in house, the specific failure modes that turn it into a two year project, and where the line between batch and streaming has moved now that software, not people, reads the data.

What real time data streaming actually means

There are two ways to get data out of a production database and into something else.

Batch runs on a schedule. Every few hours, or overnight, a job asks the database what changed since last time and copies it. It is simple, it is cheap, and by construction the destination is behind by up to one interval.

Streaming, in this context, means change data capture. Instead of asking the database what changed, a process reads the database's own replication log, the same log it already writes to feed its replicas, and ships each row-level change as it commits. The founders describe Artie as a way to help "companies move data across their systems in real time": a row changes in Postgres, and the change appears in Snowflake as it happens.

Two properties of that design matter more to founders than the speed.

The first is that reading a replication log does not add query load to your application database. That is the property that makes it safe to point at production, and it was the exact requirement of Artie's first customer.

The second is that the log contains every column of every changed row, including the ones your API never exposes. That is a security decision disguised as a plumbing decision.

The test: does anything act between your syncs?

Here is the decision rule, and it is one question. Does anything take an action inside the gap between syncs?

If a person opens a dashboard each morning and reads yesterday's numbers, a nightly batch is correct and streaming buys you nothing but a bill. If a workflow, a pricing rule, a support system, or an AI agent reads that data and acts on it many times a day, then every action taken inside the gap runs on data you already know is out of date.

Robin's version of this came from the buyer's side. At Zendesk, and later working on growth at Opendoor, he kept asking his data team for faster and fresher data, because he wanted to run experiments faster and to operationalize use cases in a tool like Retool. The answer he kept getting was some variant of: this is too hard for the data team, we do not have the capacity, so unless it is a company P0, do not bother.

That answer is usually correct at the time it is given, which is what makes it dangerous. It is a statement about the cost of the pipeline, not about the value of the freshness, and the two move in opposite directions as a company grows.

The second half of the test is what being wrong costs you. Same latency, completely different blast radius: the human catches it, the software does not.

What building it yourself actually costs

This is where the interview earns its keep, because Robin has now tried to build this three times.

At Zendesk, the team rebuilt the enterprise data warehouse more than once. They were already doing change data capture with Maxwell, the MySQL to JSON tool Zendesk built and open sourced, and using it for data integrations was still too hard. At Opendoor, the same Postgres to Snowflake problem came back, and the team never got a build to production readiness. He puts one of those attempts at about a year of engineering time that ended without a shippable system.

At Artie, with a founder who had already failed at it twice and nothing else to work on, it took about six months. He thinks he could redo it in two to three months today with current AI tooling. Read that number carefully before you use it in a planning meeting: six months is the best case, achieved by a specialist for whom the pipeline was the entire company.

Jacqueline's version of the build versus buy math, describing the alternative a customer like Substack faced, was more concrete: hire five to seven engineers with real distributed systems knowledge, ideally including one person who has built this before, and "give them like six to 12 months." The companies that did build it in house, she names DoorDash, Netflix and Instacart, put multi-year efforts and large teams into it to guarantee data integrity, prevent out of order writes, and recover from failures without a disruptive rebuild.

A vendor always has an incentive to say that building is expensive, so weigh the independent evidence instead. Netflix published DBLog, a watermark based change data capture framework, as an engineering paper and a blog post. Zendesk open sourced Maxwell. Companies do not publish papers and release infrastructure for problems that turned out to be easy.

Robin's own framing of the build decision is the one to keep: spending a year to two years building a Postgres to Snowflake connector "just seems nonsensical", a point his interviewer finished for him. It is nobody's core competency, and every company past a certain data scale ends up rebuilding the same thing.

Why AI agents moved the line

For twenty years, batch was correct because the consumer of the data was a person on a human schedule. A daily refresh matched a daily decision.

That assumption breaks when the consumer is software that reads and acts in the same second. The gap between the world and the record stops being a reporting delay and becomes an error rate.

You can see founders navigating exactly this trade-off in the wild. On another YC Root Access session, Ayush Garg, co-founder of AnswerThis, described how their internal operations agent learns the business logic only they know: by "giving the agent a read-only copy of both our database as well as our code base", with "a cron job that basically gives it an updated version every time we do a release."

That is a batch answer, and for their use case it is the right one. The thing the agent needs to know, how subscription logic works, changes when they deploy, so refreshing on deploy is exactly matched to the rate of change. Nobody needs a streaming pipeline to learn that a pricing rule changed twenty seconds ago.

That is the actual principle, and it is more useful than the slogan that agents need real time data. Match the refresh cadence to the rate of change of the thing the agent needs to know, not to how ambitious the roadmap sounds. Business logic changes on release, so refresh on release. Order status, inventory, account balance and fraud signals change continuously and get acted on continuously, so those are where the sync gap turns into wrong actions.

Artie's own read on their market is that companies are now treating real time data as a prerequisite for deploying AI workloads and agentic use cases, which is a vendor's claim about demand rather than a measurement. The checkable part is that the warehouses agree: Snowflake shipped Snowpipe Streaming and Google shipped the BigQuery Storage Write API, both aimed at getting rows queryable in seconds rather than hours. Snowflake documents ingest to query latency as low as five seconds, and BigQuery makes streamed rows available to query immediately. Artie has launched an events API on top of that shift and puts its own target at roughly 100 milliseconds from write to queryable. When the destinations start optimizing for freshness, the ceiling on what a pipeline can promise moves with them.

If your roadmap has agents acting on operational data, that decision is upstream of the agent and it belongs in the same conversation as your data strategy rather than in a later infrastructure sprint.

The failure modes that make this a two year project

Founders who estimate this project imagine the happy path, which genuinely is not hard. Robin's summary of why the estimate is always wrong: "You can set it up relatively easily on your local environment, but it's a completely different scale when you bring it into production scale." He calls data processing "a series of accumulated battle scars." Four of the scars are worth naming, because they are what the second year is spent on.

Backfilling while streaming. When you connect a new table you have to load the existing rows, and that can take days, during which changes keep arriving. Netflix's DBLog solves this by interleaving the backfill chunks with the live change log rather than pausing either one. Artie built an online backfill of its own, and then onboarded a customer with 10 billion rows in a single table, where the existing method would have taken about two and a half months. Working is not the same as working at the next customer's scale.

Somebody else's bug becomes your bug. Robin describes spending a year on one problem: when Kafka got overwhelmed it rebalanced consumers, and the SDK they used had an ordering bug in rebalancing, so messages published correctly were read out of order and the destination data was silently wrong. The fix required debugging the vendor's SDK, discovering it had not implemented the relevant Kafka improvement proposal correctly, and switching vendors. Your customer does not care where the bug lived. As the founders put it, even when a customer shoots themselves in the foot, "that's technically still your problem", because the obvious follow-up question is why the product did not prevent it.

Every customer's data is broken differently. "Everyone has messy data, but the messy data always looks different." One of the founders lists the shapes: single tenant designs that arrive as 20,000 to 30,000 separate databases to merge into unified tables, and companies sharding as they grow and then sharding again, which is exactly what breaks ordering guarantees and schema evolution. Robin's version of the same point is the small stuff that eats months, like discovering that MongoDB will accept a month value greater than 12.

The documented path is often the slow one. Building a SQL Server connector, he says, is hard because the documented path is not performant enough for enterprise workloads. SQL Server's built-in change data capture has an agent job read the transaction log into separate change tables, and that extra hop does not keep up at enterprise volume, so getting there means implementing "this backdoor method that's not documented."

None of that is exotic. It is the normal texture of infrastructure work. The build versus buy question is not whether your team could build it. It is whether you want a permanent on-call rotation for somebody else's SDK bugs.

The security question that shows up in procurement

Here is the part that gets skipped, and it is the part a security review will find. A change data capture pipeline is a process holding standing read access to your production database's replication log. Five things follow from that, whether you build it or buy it.

1. The log is not the API. It carries every column of every changed row, including columns your product deliberately never exposes. If you have not defined which columns leave the database, the default is all of them. Write a column level allow list, not a deny list, because a deny list silently leaks the next column someone adds.
2. Replication slots are an availability risk, not just a confidentiality one. On Postgres, a slot that goes inactive or falls behind makes the primary keep the write ahead log it would otherwise recycle, and the PostgreSQL documentation warns plainly that slots can retain enough WAL to fill the space allocated for it. A paused pipeline can therefore take down the database it was only supposed to read. Set a cap and alert on slot lag on day one, not after the first incident.
3. Streaming widens the blast radius of personal data. Batch gives you a natural filtering point in the job. A set and forget stream copies new columns as your schema evolves, which means a field added for a feature can land in the warehouse, and in every downstream AI system reading it, before anyone reviews it.
4. A managed pipeline is a subprocessor. Any vendor sitting in the path of your production data belongs on your published subprocessor list, with its data residency documented. Enterprise buyers read that list, and finding an unfamiliar name in it turns a line item into a conversation you have to win.
5. Replication credentials are the crown jewels. A credential that can read the log is functionally a read replica of your entire business. Scope it to the tables in the allow list, restrict it by IP, rotate it on a schedule, and log its use. If your test is whether you would notice that credential being used from a new location, most teams already know the answer.

None of this argues against streaming. It argues for deciding these five things while you are choosing the pipeline, when they cost an afternoon, instead of during procurement, when they cost a quarter. The same logic applies to the rest of your security posture: the cheap version done early beats the thorough version done late, which is the argument behind running a penetration test earlier than feels necessary.

If you are the one selling the infrastructure

Artie's go to market is worth studying separately, because a high trust cost changes what a fast first close means.

Start with the constraint Jacqueline names: "anything that needs to be real time is by definition like mission critical." That single fact sets the entire shape of the business. Deployments are slow, buyers are cautious, and no amount of product polish shortens the trust cycle.

Their first customer was Substack, from a cold email, when nobody was using the product. The email went out between 8 and 9 in the morning, the head of data replied within an hour asking to talk that afternoon, they spent the weekend building enough UI to make the system visible, and deployment started Monday. Substack then ran a proof of concept that pushed billions of rows through it with tight constraints on how Artie could touch the database. The reason a company with millions of users bet on an unused product is not charisma. The alternative was a five to seven engineer build with a six to 12 month timeline.

Then comes the part founders misread. That first win did not repeat. They finished the batch with seven or eight customers, most of them smaller early adopters, and it took roughly another nine months to land a second customer at Substack's scale. Infrastructure with a high trust cost does not produce a hockey stick early, and reading a fast first close as the new baseline is how teams over hire into a pipeline that has not arrived.

What they did instead is the number worth remembering: they crossed $1 million in ARR with four people, the two founders and two engineers. That ratio, not headcount, is the number worth tracking, which is the argument behind revenue per employee. Their explanation is straight from YC advice they took literally. "A bad employee is worse than no employee at all", and "don't hire unless that position is literally the biggest constraint to growth." Jacqueline also kept sales herself, describing herself as a very good SDR for the first year and a half, precisely so that objections and feature requests reached engineering with nothing in between. Customers, she says, are constantly surprised by how quickly feedback becomes product.

They are also honest that the rule has a limit. They say they took it a little too far, feel understaffed now, and are accelerating hiring with a plan to triple the team, starting with an in house recruiter. A constraint that was correct at four people is not automatically correct at forty.

Two pieces of context sharpen this if you sell infrastructure. First, Y Combinator's own sales guidance notes that "the cost of scaling an Enterprise sales team effectively creates a floor price for your product", usually around 10K for mid-market and 100K for enterprise. If your product cannot carry that price, founder led sales is not a phase you grow out of, it is the model, and which motion that implies is the top down versus bottom up sales question. Second, on YC's Lightcone, one of the hosts described the signal that adoption has really happened: "enterprises are bugging you to sign the security agreement" so they can keep using a product their teams already deployed. For an infrastructure company the security questionnaire is not friction at the end of the funnel. It is often the first evidence the funnel exists, which is another reason to have the answers written down before you need them. If your model for winning enterprise accounts is deep technical involvement rather than a traditional sales motion, the forward deployed engineer pattern is the version of this that scales.

What to do this week

1. Measure your actual sync gap. Pick the three most consequential reads of warehouse data in your company. For each, record when the underlying row last changed and when the data was read. If the gap never touches a decision, you are done and you just saved yourself a project.
2. List everything that acts on data automatically. Workflows, pricing rules, alerts, agents. For each one, write the rate of change of what it needs to know. Anything where the data changes faster than the refresh is where wrong actions are already being taken.
3. Price the build honestly before you scope it. Use the real numbers: five to seven engineers with distributed systems experience, six to 12 months, and a permanent maintenance burden. If that is more than the freshness is worth, the decision is made.
4. Write the column allow list. One page listing which tables and columns may leave the production database. Do it before you connect any pipeline, managed or homegrown, because the default is everything.
5. Cap and alert on your replication slots. Set the retention limit and add an alert on slot lag. This is a thirty minute task that prevents a read-only pipeline from filling the disk on your primary database.
6. Add the pipeline to your subprocessor list. If you use a managed vendor, put it on the published list with its region now, while it is a text edit rather than a procurement escalation.

The pattern underneath all of this is the one that runs through every operating decision an AI startup has to make: decide what has to be fresh, pay for exactly that, and instrument it so you find out when the answer changes. If you want the complete version of that operating system, from data and cost decisions to security posture to go to market, that is what we teach in AI Operating System for Startups.

Sources

• Artie: Real Time Data Streaming For The AI Age, Y Combinator's Root Access, the interview with Robin Tang, co-founder and CTO, and Jacqueline Cheong, co-founder and CEO, that this article distills.
• Artie on Y Combinator, for the company profile and Summer 2023 batch, and Artie's $12 million Series A for the round details and the figure of more than 700 billion rows processed for customers per year.
• Standard Capital, the fund that led the round, founded by Dalton Caldwell, Paul Buchheit and Bryan Berg. Background on the Caldwell and Buchheit departure from Y Combinator: Forbes.
• DBLog: A Generic Change-Data-Capture Framework, the Netflix Technology Blog, for the watermark based approach to backfilling while streaming.
• Maxwell, the MySQL change data capture daemon open sourced by Zendesk.
• PostgreSQL documentation on replication slots, for the warning that slots can retain enough write ahead log to exhaust the space allocated for it.
• Snowpipe Streaming and the BigQuery Storage Write API, the warehouse side of the shift toward streaming ingestion.
• From the knowledge base: How to Build an Internal AI Agent That Evolves Itself, Y Combinator's Root Access, for Ayush Garg of AnswerThis on refreshing an agent's copy of the database on every release.
• From the knowledge base: Which Sales Strategy Is Best For Your Startup?, Y Combinator, for the enterprise sales price floor, and He Built the World's #1 Open-Source Coding Agent, Y Combinator's Lightcone Podcast, for the security questionnaire as an adoption signal. That transcript does not label which host is speaking, so the line is credited to the panel rather than to any one host.