Snowflake brought Iceberg Spark APIs to Snowpark Connect for Apache Spark with write-audit-publish on branches.
The detail that matters is one config, spark.wap.branch, which routes a DataFrameWriterV2 append to audit_run instead of main.
Publishing is a single call to cat.system.fast_forward that moves main to the branch snapshot without rewriting data files.
Readers of main see nothing until that commit, so bad rows stay isolated where Spark and other engines can query branch_audit_run like a table.
If checks fail, you drop the branch and main was never touched, which replaces shadow tables and swap-and-rename scripts with one table and one schema.
Snowflake says the path works across Snowflake-managed and externally managed tables in Polaris, AWS Glue and Unity Catalog through catalog-linked databases.
Spark jobs run on Adaptive Compute with no clusters to size, under Snowflake Horizon Catalog governance and lineage.
DataFrameWriterV2 support is in public preview and flexible partitioning with expression and bucket transforms is in private preview, per Snowflake.
Incremental reads process only changes between snapshots, while Time Travel and tags in public preview give stable checkpoints for rollback.
Type mapping still decides if that write lands cleanly, since Spark types convert to Iceberg types on both creation and writes per Iomete reference.
Klarent, a Telenav company, moved off Amazon EMR and runs 3.6x faster at 60% lower total cost with its longest pipeline down from over 10 hours to under three, Snowflake said.
Kumar Maddali, VP of Engineering and AI at Klarent, said his team assumed Snowflake would cost more and found the opposite.
The limit is preview status, with branch publishing in private preview and partitioning still rolling out.
For AI-generated pipeline changes, you get a real-data trial with an auditable snapshot before production ever sees it.





