FY26 Faire

🧠 Free Recall ❓ Quiz 🔍 Cues 🔮 Simulation 🥷 ShadowBox 📐 Encode Cards

Account Overview and Executive Summary
Faire Wholesale, Inc. is a rapidly scaling, data-driven marketplace that has become a major consumer of Confluent's data streaming solutions. Having started as a paying Confluent customer in early 2024, Faire's aggressive adoption—particularly of TableFlow (Confluent’s governed streaming lakehouse solution)—reflects both trust and transformation in their data infrastructure. Over a relatively short time, Faire transitioned from exploratory usage and pilot rollouts to production-scale workloads, driving a sustained increase in consumption and spend, and prompting renewed commercial and technical dialogue at the executive level.
The partnership’s success is matched by its complexity: Faire has become a lighthouse customer for several Confluent offerings, providing critical feedback that now materially influences Confluent’s roadmap. Their experience exposes crucial product gaps, especially around data governance, observability, and cost management. With major contract renewals on the horizon and new procurement leadership in place, preparing for renewal and further expansion demands a nuanced understanding of Faire’s evolving needs, internal priorities, and remaining friction points.

Key Contacts and Organizational Stakeholders
Key stakeholders at Faire include Greg Bernard (Lead Platform Engineer, primary technical owner for TableFlow and Kafka usage), Harsh Shah (Data Platform Lead, historically the key architect and main product champion), Khaled Said (current operational point-person, focused on overall efficiency, cost, and expansion across the business), and Ash Mohan (Procurement Lead, responsible for commercial negotiations and contract renewal strategy since June 2026). Bob Berry, Chris Holly, Mike Agnich, and Richard Nwankwo represent the core customer-facing team at Confluent, providing continuity across sales, product leadership, and post-sales engagement.
Greg Bernard: Day-to-day product owner, technical lead on integration and adoption, primary source of product feedback.
Harsh Shah: Data platform visionary and internal influencer with a focus on strategic platform bets on Confluent technology, champion for “all-in” adoption.
Khaled Said: Responsible for vendor management, billing oversight, efficiency improvements; tasked with ensuring expansions serve broader business outcomes.
Ash Mohan: New lead for procurement; driving commercial discussions and ensuring appropriate discounts and contract alignment with actual usage patterns.
Chris Holly and Mike Agnich (Confluent Product Leadership): Closely involved in roadmap discussions, escalation handling, and executive sponsor roles.
Robert Berry and Richard Nwankwo (Confluent Account/Sales Team): Orchestrate customer success, drive solution adoption, and maintain the commercial relationship.
Customer Priorities and Engagement Themes
The Faire team is laser-focused on achieving high developer velocity, data governance, and operational reliability at massive scale. Their priorities can be grouped into a few critical categories:
Expansion of TableFlow as the foundation of governed, cloud-native data lakes for analytics (including large-scale, high-volume, and cross-region workloads).
Deep integration with Confluent’s infrastructure-as-code approach—specifically, native Terraform support for pipeline and data retention automation.
Cost governance, billing transparency, and spend predictability, particularly as usage surpassed initial contract estimates and entered overage territory.
Enhanced observability, error reporting, and actionable alerting to prevent silent failures in mission-critical data pipelines.
Removal of platform blockers and product gaps—especially features required for data management (TTL, large-scale deletes, Iceberg V3 support).
Securing best-in-class commercial terms and executive partnership as usage grows to multi-million dollar levels.
Risks and Churn Considerations
While Faire expresses strong intent to standardize on Confluent, the willingness to surface blockers and procurement-led scrutiny means there are clear risks that could threaten renewal and expansion.
TableFlow Product Gaps: Critical blocking issues—particularly around inability to increase partitions, manage TTL via Terraform, and perform large-scale deletes efficiently—have caused escalations and could force technical stakeholders to evaluate alternatives if not addressed.
Cost Sensitivity: Faire’s rapid growth drove them quickly into overage, and procurement leadership has flagged concerns about missing earlier opportunities for discounts. Price-performance is an active factor in renewal calculus.
Observability, Reliability, and Trust: Silent product failures, lack of actionable metrics, and confusion in the UI have led to incidents jeopardizing the perceived reliability of Confluent in production. Failure to address these gaps could erode trust.
Procurement Efficiency: As the procurement team seeks to streamline vendor count and maximize ROI, any perceived misalignment between solution value and spend, or lack of timely commercial responsiveness, could introduce risk.
Technical Overhead: Delays or lack of clarity in feature delivery (e.g., Terraform support for TTL, Iceberg V3, error surfaced in UI) drive shadow process and manual workarounds, undermining the platform’s centrality within Faire’s stack.
Product Interest and Adoption—Current Focus Areas
Faire’s core interest is split across several key Confluent offerings, with TableFlow as the marquee adopted product and platform for sustained feedback:
TableFlow: Central to both operational analytics and long-term data lake projects. Faire is standardizing on TableFlow for governed, real-time data landing in Snowflake and other analytical targets, with emphasis on automation, cost control, and compliance.
Schema Registry: Enables strong data contracts, type safety, and pipeline integrity, removing layers of complexity from previous Snowpipe-based approaches.
Confluent Terraform Provider: Considered critical for all configuration management, platform rollout, and audit-compliance needs—especially for automating data governance, TTL, and table provisioning.
Freight Clusters: Seen as a cost-optimized platform for high-throughput workloads, especially as Dedicated clusters are viewed as too expensive for their network usage patterns.
Flink (in pilot/consideration): Used for advanced stream processing, but encountering integration and schema compatibility barriers that remain to be fully resolved.
In summary, TableFlow, its supporting ecosystem (Schema Registry, Terraform, Flink), and cost-optimal Kafka infrastructure form the technological backbone Faire seeks to build upon.

Product Feedback: Deep Dive
The feedback from Faire is both broad and deep, reflecting the maturity of their adoption. Product-level feedback is split as follows:

Product Feedback: Negative
TableFlow’s inability to efficiently support large-scale deletes (without tombstone records via Kafka) is a major blocker for customers handling terabytes of append-only data. Faire sees this as unworkable at their scale and a direct hindrance to adoption.
Critical lack of partition-level observability in TableFlow metrics prevents effective alerting for data loss or incomplete replication, leading to silent failures in production. This gap has resulted in “missing data” incidents and an inability to proactively detect problems.
The Terraform provider for TableFlow lags behind operational requirements, with the inability (until very recently) to configure or display TTL settings. This forced manual API workarounds, undermining auditability, automation requirements, and confidence in configuration.
TableFlow’s user interface often displays “healthy” states even when underlying issues exist (e.g., suspended tables, delayed data, or missing records). Customers cite occasions where there is no error surface in the UI or actionable logs.
Monitoring and metrics within the UI lack clarity—metrics like “rows per minute” do not show the time window, and ‘skipped messages’ are un-attributable to specific sources or times, making them functionally useless for troubleshooting or reporting.
Iceberg V3 support is absent in TableFlow, blocking Faire’s migration of semi-structured, high-volume data pipelines and driving resource-intensive workaround projects.
Network pathing and S3 path generation are not human-readable, leading to confusion and reporting issues with internal business stakeholders (e.g., finance and audit teams).
Certain schema compatibility issues (such as with Proto2 optional fields in Flink) have blocked end-to-end implementation for specific data flows.
Lack of support for direct partition increases after running upsert tables: this blocks vital scaling actions and has directly impacted performance.
Previous attempts at using Dead Letter Queue (DLQ) features resulted in catastrophic failures on private networking clusters, discouraging the adoption of built-in error handling tools.
Insufficient cost management and billing breakdown reporting—finance teams need more granular cost attribution and automated reporting, now largely addressed but previously causing friction.
Product Feedback: Positive
TableFlow dramatically simplified Faire’s analytics pipelines and eliminated an entire layer of data contract and transformation code, unlocking faster delivery and higher assurance for governed data.
Schema Registry integration, alongside Confluent’s commitment to data contracts and strong typing, is seen as a ‘big win’, enabling exacting control and reducing error rates in production.
TableFlow achieves high stability and good latency on standard workloads, and is viewed as a near “switch-on” experience for simple use cases, leading to successful early outcomes.
The recent addition of TTL support via the Terraform provider has been enthusiastically received and is enabling internal adoption at scale.
Partition scaling, when possible, has been shown to improve latency, confirming architectural assumptions and reinforcing confidence in Confluent’s technical direction.
Faire acknowledges Confluent’s “listening ear,” responsiveness to product feedback, and willingness to escalate and engage cross-functional expertise to address blockers.
Product Feedback: Urgent
Native support for partition-level metrics in TableFlow (latency, throughput, and completeness by partition) is a top urgent need, as it directly impacts Faire’s ability to trust the platform for governance-sensitive use.
Ability to perform bulk deletes directly on TableFlow tables, ideally using a straightforward SQL-like or targeted interface, is the single most frequent and urgent request. The inability to do this is directly blocking rollout to the highest volume topics.
Iceberg V3 implementation is an urgent requirement to unblock ingestion of semi-structured data and accelerate migration away from legacy data pipelines.
Improved error surfacing in the UI (status visibility, clear error messages, actionable remediation guidance) is critical, as trust in the health reporting directly influences operational safety in production environments.
Terraform provider maturity for all TableFlow configuration parameters—including human-readable TTL units, visibility of applied settings, and support for expanded use cases—remains highly time-sensitive to support internal rollout.
S3 pathing and Glue catalog customization to allow readable, meaningful resource names—demanded by both technical and business reporting needs—remains an outstanding action item.
Ongoing need for automated, granular, and transparent cost/billing breakdowns to support procurement and budgeting cycles.
Current Commitments and Active Opportunities
As of July 2026, Faire is at a strategic inflection point. The multi-year renewal (valued at $1.2M) is in the ‘Define Solution’ stage; Ash Mohan is leading procurement and has requested better alignment of discounts to actual usage, citing historical overages and advocating for more aggressive commercial terms given record-high consumption. Spend governance and contract optics are active topics in the renewal negotiation. At the same time, internal product rollouts hinge on timely delivery of requested roadmap capabilities, especially around TableFlow automation, data retention, and reliability improvements.
There is also a recent win for a Professional Services carve-out (PS Carve Out for AWS Commit) and a track record of successful expansions, driven by ongoing trust in TableFlow and its supporting stack. However, the loss of a single, very large opportunity (‘Faire-AWS-Commit’ at nearly $400K) in late 2024 remains a visible datapoint for procurement leadership, underscoring the need for close commercial engagement.

Key Takeaways and Actionable Insights
Ensure TableFlow continues to receive highest-level engineering attention for open feature gaps—especially partition scaling, bulk delete capabilities, and advanced observability.—as these are intrinsically linked to Faire’s ongoing and future expansion.
The Terraform provider—now a success story—still requires investment in usability (clearer TTL settings, visibility, elimination of manual processes) to secure full organizational rollout at Faire.
Proactive, bi-weekly reporting on consumption trends, billing anomalies, and optimization opportunities must continue as a standing agenda, both as a service to procurement and as a tool to identify/resolve future overages before they threaten the commercial relationship.
The renewal process must be tightly managed with Ash Mohan and Khaled Said, anticipating procurement’s focus on discounting, packaging, and operational guarantees tied to actual usage patterns.
Ongoing feedback loops from Greg Bernard and Harsh Shah are essential to catching issues early; these stakeholders must remain closely involved and informed as features move through the roadmap and into release.
Any signs of reliability, silent data loss, or billing confusion should trigger immediate escalation; maintaining trust at scale is paramount to renewal likelihood.
Deep partnership—visible commitment to executive engagement and roadmap influence—will be decisive in securing Faire’s loyalty amid a procurement-oriented vendor evaluation environment.

Faire is using Confluent for:
Kafka / event streaming for logging, service-to-service data, and large-volume topic pipelines.
Schema Registry to enforce data contracts and type safety.
TableFlow to land governed streaming data into Iceberg tables and move data out of Snowflake.
Terraform-based configuration and automation for TableFlow, including TTL management.
Flink evaluation/usage for stream processing and possible vendor consolidation away from their current managed Flink provider.
Freight clusters for cost-optimized, high-throughput workloads and private networking needs.

Snowflake
Faire currently uses Snowflake as the main landing zone for a lot of their data, especially raw ingestion and logging data. Their current setup limits additional compute options like Spark because the data remains tied to Snowflake compute. They are trying to move large workloads out of Snowflake into Iceberg tables.

Databricks
Faire appears to use Databricks as a downstream compute and analytics engine, especially with Iceberg data. They discussed Spark/Databricks-style queries over data flattened from Tableflow, and the goal is to support workloads outside Snowflake with more flexible compute.

Expand on this and how critical this actually is "Kafka / event streaming for logging, service-to-service data, and large-volume topic pipelines."

This is very critical for Faire.
It is not just generic event streaming; it is tied to core production workflows and a company-wide push to move data out of Snowflake and into governed Iceberg tables. They are using Kafka to ingest massive logging streams, service-generated data, and high-volume model outputs, with the goal of enabling Spark, better performance, lower cost, and stronger schema enforcement.

Why it matters
It supports petabyte-scale migration work.
It helps replace Snowflake as the default landing zone for data.
It reduces risk from poorly structured JSON and missing schema enforcement.
It is a prerequisite for broader Spark adoption and downstream analytics flexibility.
Faire has described this as a major company-wide initiative, so the stakes are high.
Business impact
If Kafka is not reliable or scalable enough, Faire cannot complete the migration path they want. That would keep them tied to Snowflake, preserve cost and performance constraints, and slow down their broader data platform strategy.

Kafka
Before moving to Kafka, Faire’s business pain was that their Firehose and Snowflake-based flow was slow, expensive, and limited their ability to support additional compute engines like Spark. They also had a logging incident tied to weak schema enforcement, which increased operational risk.
Later, Kafka helped Faire standardize on stronger schemas and data contracts, reduce dependence on Snowflake, and unlock a broader plan to move large logging workloads into Iceberg tables for better performance and cost control.

Tableflow
Before moving to Tableflow, Faire faced business issues around scaling governed data, unreliable observability, manual workarounds, and spend governance concerns. The lack of features like bulk deletes, partition metrics, and mature Terraform support slowed rollout and increased operational overhead.
Later, Tableflow simplified analytics pipelines, removed custom transformation logic, improved stability on standard workloads, and enabled Faire to adopt governed, cloud-native data lake patterns at scale. It also supported their wider push to migrate petabyte-scale data out of Snowflake.

Kafka move: negative consequences
Faire ran into networking and private-network setup friction, including confusion around serverless clusters and trouble accessing Flink/Kafka through the console. The console behavior and topic access also became harder to use in their private network environment.

Kafka move: later realized business outcomes
The move helped Faire push data off Snowflake, improve developer experience, and support a broader company-wide effort to build high-spec iceberg tables and scale future data workloads. It also positioned Confluent as a foundational part of their analytics and platform strategy.

Tableflow move: negative consequences
Tableflow brought several blockers: missing partition-level observability, weak error surfacing, confusing or misleading UI states, limited TTL/Terraform maturity at first, no Iceberg V3 support, and difficulty with bulk deletes and partition scaling. These issues created trust, reliability, and rollout friction.

Tableflow move: later realized business outcomes
Despite the pain points, Tableflow simplified Faire’s analytics pipelines, reduced custom data-contract and transformation code, improved stability on standard workloads, and enabled rollout at scale. TTL support in Terraform was especially valuable, and Faire became one of Confluent’s earliest and biggest Tableflow adopters.
──────────────────────
Q: What are the material differences in using Snowpipe for data ingestion into the analytics estate v. Kafka?

Snowpipe vs. Kafka for Data Ingestion: A Deep Comparison
This is a genuinely important architectural question, and the Faire context makes it very concrete. Let me break this down across several dimensions.

What Snowpipe Actually Is
Snowpipe is Snowflake's continuous, serverless data ingestion service. It watches for files landing in cloud storage (S3, GCS, Azure Blob), triggers on arrival events, and loads them into Snowflake tables. It's file-based, pull-oriented, and deeply coupled to Snowflake's compute and storage model.
The typical flow looks like: Application → writes files to S3 → Snowpipe event trigger → Snowflake table

The Material Differences
1. Compute and Storage Lock-In
This is the biggest strategic issue for Faire specifically.
With Snowpipe, your data lands inside Snowflake's proprietary storage format. It is only accessible via Snowflake compute. You cannot run Spark on it. You cannot query it with Databricks natively. You cannot point other engines at it without copying or exporting the data first.
With Kafka + TableFlow landing into Iceberg tables, the data is written in an open format on your own cloud storage (S3). Any engine that speaks Iceberg — Spark, Databricks, Trino, Athena, DuckDB — can query it directly. This is exactly why Faire is executing this migration: they want compute flexibility, and Snowpipe structurally prevents that.

2. Latency Profile
Snowpipe is micro-batch, not truly streaming. Files accumulate, then get loaded. In practice, Snowpipe latency is typically in the range of 1–5 minutes, sometimes longer depending on file size thresholds and trigger behavior. Snowflake has improved this over time but it remains fundamentally batch-oriented under the hood.
Kafka is event-by-event, with end-to-end latency measurable in milliseconds to low seconds. For logging pipelines, service-to-service data, and model outputs — all things Faire is running — this difference matters both operationally and for real-time use cases downstream.

3. Schema Enforcement and Data Contracts
This was a direct pain point for Faire. With Snowpipe, schema enforcement is weak. Files arrive as JSON or CSV, and Snowflake will ingest malformed or schema-drifted data without strong guardrails unless you build significant custom validation logic around it. Faire had a logging incident tied specifically to this weakness.
Kafka with Schema Registry enforces schemas at the producer level — data that doesn't conform to the registered schema is rejected before it ever enters the pipeline. This is a fundamentally different and much stronger guarantee. It moves schema enforcement left, closer to the source, rather than discovering problems after data has already landed in your warehouse.

4. Cost Structure
Snowpipe charges for compute used during ingestion, billed per credit. As data volumes grow, this becomes expensive — and the cost is opaque because it's bundled into Snowflake's credit model. You're also paying Snowflake storage costs for all that data.
Kafka separates ingestion cost from storage and compute. With Confluent's Freight clusters (which Faire is specifically using for high-throughput workloads), you can optimize the cost profile for volume. And because the data lands in your own S3 as Iceberg, you're paying commodity cloud storage rates rather than Snowflake's premium storage pricing. At petabyte scale, this difference is very significant.

5. Replay and Reprocessing
Kafka retains messages for a configurable period (TTL — which is actually one of Faire's active feature discussions). This means if a downstream consumer fails, you can replay from any offset in the log. You can reprocess historical data, backfill new consumers, or recover from pipeline failures without re-ingesting from source.
Snowpipe has no native replay capability. Once data is loaded into Snowflake, the ingestion event is gone. If your downstream transformation logic was wrong, you need to either re-export from source or work entirely within Snowflake's table history features (Time Travel), which has its own limitations and costs.

6. Multi-Consumer Fan-Out
A Kafka topic can be consumed by multiple independent consumers simultaneously — each with their own offset, their own processing logic, their own destination. You can have TableFlow writing to Iceberg, a Flink job doing real-time aggregation, and a separate consumer writing alerts, all from the same topic, independently.
Snowpipe is a one-way pipe to one destination: a Snowflake table. There is no native fan-out. If you want data in multiple places, you either duplicate ingestion pipelines or export from Snowflake after the fact, adding latency and cost.

7. Operational Observability
Snowpipe provides limited visibility into ingestion health. You can query SYSTEM$PIPE_STATUS and look at load history, but partition-level or message-level observability is essentially nonexistent.
Kafka exposes consumer lag, partition-level throughput, offset positions, and producer metrics. This is richer — though notably, Faire's complaints about TableFlow's observability gaps (missing partition metrics, silent failures) show that Confluent hasn't fully realized this potential in the TableFlow layer yet. The underlying Kafka infrastructure is more observable; the TableFlow product layer still needs work.

8. Decoupling of Producers and Consumers
Snowpipe tightly couples the ingestion mechanism to Snowflake as the destination. If you want to change your analytics backend, you're rebuilding your ingestion pipeline.
Kafka is a durable, decoupled message bus. Producers don't know or care what's consuming their events. You can swap out consumers, add new ones, or change destinations (from Snowflake to Iceberg, for example) without touching producers at all. This is architecturally much more resilient and future-proof — which is precisely the strategic value Faire is extracting right now as they migrate away from Snowflake.

Why This Matters for Faire Specifically
The document notes that Schema Registry eliminated "layers of complexity from previous Snowpipe-based approaches." That's a telling phrase. What they're describing is the accumulated workaround complexity that builds up when you use a file-based, weakly-typed ingestion system at scale:
Custom validation scripts to catch bad JSON
Transformation code to normalize schema drift
Manual monitoring to catch silent load failures
Data duplication to serve multiple consumers
Inability to run Spark or Databricks on the data
Kafka + Schema Registry + TableFlow + Iceberg replaces all of that with a governed, open, replayable, multi-consumer streaming backbone. The architectural shift isn't just technical preference — it's the prerequisite for Faire's entire data platform strategy: compute flexibility, cost reduction, schema enforcement, and the ability to run petabyte-scale workloads outside Snowflake's walled garden.
──────────────────────
Q: What are the material differences between Firehose and Kafka as a tool for data ingestion?

Firehose vs. Kafka for Data Ingestion: Material Differences
These are fundamentally different tools built around different philosophies, and the distinction matters a lot in Faire's context.

What They Actually Are
Amazon Kinesis Data Firehose is a fully managed, fire-and-forget delivery service. You point data at it, and it buffers and delivers that data to a destination (S3, Redshift, Snowflake, etc.). It is designed for simplicity and operational ease — you don't manage infrastructure, partitions, or consumers.
Apache Kafka (and Confluent's managed version) is a distributed, durable, partitioned log. It is a platform — not just a pipe. Data is written to topics, retained for a configurable period, and consumed by any number of independent consumers at their own pace.

The Material Differences
1. Retention and Replayability
This is arguably the most important difference.
Firehose: No retention. Data flows through and lands at the destination. Once delivered, it's gone from Firehose. If a downstream system fails or you need to reprocess, you're going back to the destination (S3, Snowflake), not the source.
Kafka: Data is retained on the broker for a configurable window (hours, days, weeks, or indefinitely with compaction). Any consumer can replay from any offset. This is transformative — it means you can reprocess historical data, add new consumers retroactively, and recover from downstream failures without data loss.
Why this mattered for Faire: Their old Firehose → Snowflake flow had no replay capability. If something went wrong, or if they wanted to land the same data in a new system (like Iceberg tables), they had to go back to Snowflake — which is exactly the dependency they were trying to break.

2. Consumer Model
Firehose: Push-based, single destination (or fan-out only via multiple Firehose streams). One stream, one target. Adding a new consumer means duplicating the stream or reading from the destination.
Kafka: Pull-based, multi-consumer. Many independent consumer groups can read the same topic simultaneously, each maintaining their own offset. You can add Spark, Flink, a microservice, and a TableFlow sink all reading the same topic with zero coordination overhead.
For Faire, this is critical. Their goal is to land data in Iceberg tables and serve it to Databricks and potentially run Flink transformations — all from the same source of truth. Firehose simply cannot do this natively.

3. Schema Enforcement
Firehose: Schema-agnostic. It moves bytes. There is no built-in mechanism to enforce data contracts or validate structure. You can send malformed JSON and Firehose will happily deliver it.
Kafka + Schema Registry: Producers can be required to register and validate schemas (Avro, Protobuf, JSON Schema) before data is accepted. This enforces data contracts at the point of ingestion.
Faire's logging incident — where weak schema enforcement caused operational risk — is a direct consequence of Firehose's schema-agnostic nature. Kafka + Confluent Schema Registry solved this by rejecting malformed data at the source rather than discovering the problem downstream.

4. Latency Profile
Firehose: Buffers data and delivers in micro-batches. Minimum buffer interval is 60 seconds (historically; AWS has improved this, but it's still fundamentally batch-oriented). This introduces latency by design.
Kafka: Sub-second, near-real-time delivery. Consumers can read messages milliseconds after they're produced. This enables genuine real-time stream processing with Flink or Spark Structured Streaming.
5. Throughput and Partitioning
Firehose: Scales automatically but opaquely. You don't control sharding or partitioning behavior — AWS manages it.
Kafka: Explicitly partitioned. You control the number of partitions, the partition key, and therefore the parallelism and ordering guarantees. This matters enormously for high-throughput workloads — more partitions = more parallelism for both producers and consumers.
This is directly relevant to Faire's pain point around partition scaling in TableFlow. The ability to increase partitions is a Kafka-native concept that gives them a lever to tune throughput and latency — something Firehose never exposed.

6. Operational Model and Ecosystem
Firehose: Fully managed, low operational overhead, AWS-native. Simple to set up, limited to configure. Tightly coupled to AWS ecosystem.
Kafka: More operational complexity (even managed Confluent requires understanding of topics, consumer groups, offsets, schema registry, etc.), but vastly more extensible. Integrates with Flink, Spark, dbt, Terraform, and essentially the entire modern data stack.
7. Cost Model
Firehose: Pay per GB ingested and delivered. Cheap for simple, low-volume delivery pipelines. Costs grow linearly with volume but there are no compute costs.
Kafka: Pay for cluster compute and storage (in Confluent's model, also network egress). Can be more expensive at the infrastructure level, but the value per byte is much higher because of replayability, multi-consumer access, and stream processing capability.
Faire's move to Freight clusters (Confluent's cost-optimized cluster type) is a direct response to managing Kafka's cost model at high throughput — something that has no equivalent in the Firehose world.

Summary Table
| Dimension | Firehose | Kafka | |---|---|---| | Retention / Replay | None | Configurable, full replay | | Consumer model | Single destination | Unlimited independent consumers | | Schema enforcement | None | Full (via Schema Registry) | | Latency | Seconds to minutes | Sub-second | | Partitioning control | Opaque | Explicit and tunable | | Ecosystem | AWS-native | Broad (Flink, Spark, Terraform, etc.) | | Operational complexity | Low | Medium-High | | Cost model | Per GB delivered | Compute + storage + egress |

The Bottom Line for Faire
Firehose was adequate when Faire's data architecture was simple — land data in Snowflake, query it there, done. The moment they wanted to serve multiple consumers, enforce schemas, enable real-time processing, and break free from Snowflake as the sole compute layer, Firehose became a structural ceiling. Kafka isn't just a better Firehose — it's a fundamentally different primitive that enables an entire platform strategy. That's why the switch is tied to such a large company-wide initiative for them.

Unstarred

Dunk (exclude from study)

FY26 Faire

Review Questions

Todos

Sensemaker