Formats
Formats define how data is parsed on input and serialized on output. A Format is not the same as a file type — it's a schema that tells layline.io how to interpret incoming bytes and structure outgoing data.
Overview
Formats are the bridge between raw bytes and structured messages. When data enters a workflow through an Input Processor, the Format parses it into the internal Data Dictionary structure. When data exits through an Output Processor, the Format serializes it back to the target representation.
Unlike simple file type detection, Formats provide:
- Schema-aware parsing — Define the exact structure of your data
- Bidirectional support — Read and write using the same format definition
- Data Dictionary integration — Parsed data becomes accessible throughout your workflow
- Built-in testing — Test grammars and schemas against real data in edit mode
Available Formats
| Format | Description |
|---|---|
| ASN.1 Format | Abstract Syntax Notation One parsing for telecom, cryptography, and banking protocols. Supports BER/DER encoded data, X.509 certificates, and custom ASN.1 schemas. |
| Data Dictionary Format | Centralized field definitions and custom data structures. Define reusable types that can be referenced across multiple formats and workflows. |
| Formats Introduction | Overview of format concepts and general guidance on when to use each format type. Start here if you're new to working with formats. |
| Generic Format | Flexible grammar-based parsing for CSV, fixed-width, delimited text, binary formats, and custom structured data. Highly configurable with regex-based grammar language. |
| HTTP Format | HTTP request/response message parsing for REST APIs, webhooks, and HTTP-based integrations. Handles headers, query parameters, and body content. |
| XML Format | XML document parsing with XSD schema validation. Supports SOAP messages, configuration files, and XML-based APIs with namespace and XPath support. |
How to Choose a Format
| If your data is... | Use this Format |
|---|---|
| Structured text (CSV, delimited, fixed-width) | Generic Format |
| Hierarchical markup (XML, SOAP) | XML Format |
| HTTP payloads (REST APIs, webhooks) | HTTP Format |
| Telecom/banking protocols (TAP, CAMEL, X.509) | ASN.1 Format |
| Shared across multiple formats | Data Dictionary Format |
| You're unsure where to start | Formats Introduction |
Common Patterns
Format Selection by Data Type
| Data Type | Recommended Format | Notes |
|---|---|---|
| CSV files | Generic Format | Define delimiters, headers, and field types in grammar |
| JSON payloads | Generic Format | Use JSON grammar or HTTP Format for REST endpoints |
| XML files | XML Format | XSD validation ensures schema compliance |
| SOAP services | XML Format | Native SOAP and WSDL support |
| REST APIs | HTTP Format | Path and method definitions with request/response handling |
| Fixed-width records | Generic Format | Define field positions and lengths precisely |
| Binary protocols | ASN.1 or Generic Format | ASN.1 for standard protocols, Generic for custom binary |
Validation and Testing
Some formats support testing within the Configuration Center:
- Define your format grammar or schema
- Upload sample data files
- Verify parsing results in the preview panel
- Iterate without deploying to a cluster
This testing workflow applies to Generic Format, ASN.1 Format, and XML Format.
See Also
- Input Processors — Entry points that use Formats to parse incoming data
- Output Processors — Exit points that use Formats to serialize outgoing data
- Data Dictionary Concept — How parsed data is structured and accessed in workflows
- Data Dictionary Format — Creating reusable type definitions