---
title: LAT Overview
slug: VTnYfCrEtUlDr-KgvJNTS
description: Learn about the Loading and Transformation system (LAT) in Ocient, essential for creating pipelines from data sources.
docTags: 
createdAt: Thu Jul 21 2022 17:05:13 GMT+0000 (Coordinated Universal Time)
updatedAt: Fri Apr 10 2026 20:35:55 GMT+0000 (Coordinated Universal Time)
---

:::hint{type="warning"}
Data Pipelines are now the preferred method for loading data into the Ocient® System. For details, see [Load Data](<././Load Data.md>).
:::

The Ocient Loading and Transformation (LAT) is a service used for transforming and streaming data into Ocient. The LAT transforms records from a configured Source using the [JMESPath](https://jmespath.org/) transformation language and streams records into Ocient tables.

## LAT Benefits

- Exactly-once processing from a Source
- Selecting data and arrays from nested JSON data
- Transforming records and routing into one or more tables
- Scalar data transformation (e.g., concatenate strings, uppercasing, rounding)
- Dynamic schema change handling

## LAT Data Flow

The LAT consumes JSON records, transforms them, and loads them into one or more Ocient tables. This process is referred to as a "pipeline", with a single pipeline running per LAT instance. The pipeline is described by a JSON-based configuration that instructs the LAT how to consume records from a Source, how to transform the records, and which Ocient table to route the records to. As records are consumed, the transformations described in the configuration are applied to each record before forwarding the resulting records to Ocient.

Each pipeline configuration can map a record to 1 or more Ocient Tables, each table with its own transformation configuration. LAT can optionally be configured to write records which were unable to be processed to an Apache® Kafka® error topic.

Multiple LAT instances can be used in tandem to further parallelize a pipeline. The LAT makes use of the Kafka Consumer library to ensure that each Kafka Partition is only assigned to a single instance. To use multiple instances, configure each LAT instance with the same pipeline configuration. The LAT instances will automatically coordinate and distribute the load across the instances.

The LAT is controlled by an HTTP endpoint and a command line interface (CLI) is supplied to perform common actions such as starting, stopping, or updating the LAT pipelines.

## Related Links

[Connect to Ocient](<././Connect to Ocient.md>)

[Ingest Data with Legacy LAT Reference](<./Ingest Data with Legacy LAT Reference.md>)

## Related Links

[At the Whiteboard with Ocient: Loading and Transformation](https://www.youtube.com/watch?v=D3exgC20REc)

