> ## Documentation Index
> Fetch the complete documentation index at: https://resources.athenaintel.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Operate lakehouse transforms and pipelines

> Build transform outputs, check data freshness, and resume pipelines after failed runs

You use lakehouse pipelines to keep copied or derived tables available for analysis. A sync copies source data into a namespace. A transform project uses SQL models to produce derived tables in a target namespace. Use their asset pages to review runs, check freshness, and recover when repeated failures pause scheduled work.

## Build a transform project

You need edit access to the target namespace to create a project from it. Transform projects are Beta.

1. Open the target namespace and select **Create transform project**.
2. In **Models**, add or edit your SQL model files. Set their sources and output definitions for the tables you need.
3. Use **Preview** to inspect a sample result without building outputs.
4. Select **Check** to capture and validate the draft. This checks model blocks, sources, and the target namespace; it does not run the models.
5. Select **Plan** to review what a build changes. Inspect the changes before selecting **Apply plan** on the plan's card in the **Build** panel. It appears for the newest successful plan when that plan has changes to apply.
6. Alternatively, select **Build** to check, plan, and apply the draft in one workflow. Review the run details to confirm publication.

<Frame caption="How to plan and publish a transform, then resume a project paused after repeated failures">
  <iframe src="https://player.vimeo.com/video/1234516237?h=2cd2947302&title=0&byline=0&portrait=0&dnt=1" className="w-full aspect-video rounded-xl" frameBorder="0" allow="autoplay; fullscreen; picture-in-picture" allowFullScreen title="How-to: Plan and publish a transform, then resume a project paused after repeated failures" />
</Frame>

<Accordion title="Steps in this video">
  1. Check the draft
  2. Plan the change
  3. Apply the plan
  4. Review what published
  5. Read why it paused
  6. Resume the project
  7. Run it now
</Accordion>

## Add publication checks

1. Add SQL audit files under `audits/`. An audit query returns rows that violate a rule. Attach the audit by name in a model's `audits` field.
2. Add YAML fixture tests under `tests/` to define input rows and expected query output for a model.
3. Declare `grain` in the model when a set of columns defines a unique row. The project adds checks for nulls and duplicate combinations on those columns.
4. Run **Plan** or **Build** and review test and audit results. Fix failed fixture tests or blocking audits before publishing.

Fixture tests run during planning and applying. Blocking audit failures prevent output publication. A successful preview does not prove these checks pass.

## Check freshness

1. On a sync, select **Edit definition** and set **Freshness target (hours)**. Select **Save definition**. If you leave the target empty, a scheduled sync uses twice the schedule's longest gap.
2. On a transform project, set **Freshness target** and **Default schedule** in its configuration. Build the changed draft to apply it to the deployment.
3. Review **Freshness** and the **Stale** badge. Sync freshness measures time since a successful run copies rows. Transform freshness measures time since output publication.

For transforms, a default schedule overrides the schedule implied by the freshness target. A model's own cron takes precedence over the default schedule.

## Resume after repeated failures

1. Open the paused sync or transform project and read its failure reason and run details. By default, a project pauses after five failed runs in a row.
2. Fix the source connection, permissions, SQL, or publication check that fails.
3. Select **Resume**. This resets the failure count and allows the schedule to run again. A transform's next scheduled run is due right away. A sync waits for its next scheduled time unless you choose **Run now**.
4. Use **Run now** on a sync or a built transform project to check the recovery.

## Good to know

* Readers can inspect shared pipeline assets. Editors operate them. Publishing transforms also requires source namespace access and edit access to the target namespace.
* Failure pauses hold manual and scheduled runs until you resume.
* Transform runs execute the deployed revision. Editing a draft does not change scheduled work until you build or apply it.
* A completed computation does not by itself confirm publication. Review the publication outcome before using new outputs.

## Related links

* [SQL queries](./query)
* [Databricks connection](../../integrations/databricks)
* [Spaces and assets](../concepts/spaces-and-assets)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.