Sink S3

Purpose

Defines the outbound connection parameters for a Kafka sink. This Asset is required by:

• Kafka Output Processor

Prerequisite

You need:

• Kafka Connection

Configuration

Name & Description

Name : Name of the Asset. Whitespaces are not allowed in the name.

Description : Enter a description.

The Asset Usage box shows how many times this Asset is used and which parts are referencing it. Click to expand and then click to follow, if any.

Required roles

In case you are deploying to a Cluster which is running (a) Reactive Engine Nodes which have (b) specific Roles configured, then you can restrict use of this Asset to those Nodes with matching roles. If you want this restriction, then enter the names of the Required Roles here. Otherwise, leave empty to match all Nodes (no restriction).

Kafka Connection

Select the Kafka Connection to use with this Asset. If it does not exist, you need to create it first.

Kafka Producer Settings

General

• Parallelism: Tuning parameter of how many messages can be sent to Kafka in parallel. If nothing is specified, the default is 10,000.

Additional Kafka Properties

Use this section to add configuration parameters available for Kafka Platform such as compression.type or even ssl.keystore.key.

For a list of available properties please check Confluent Producer Configurations (opens new window).

Attention: Kafka properties take precedence

Please note that properties defined here, take precedence over all other settings you may have provided in this UI. You can use this to add, or override Kafka properties using these settings.

Exclusive partition access

In case you want to exclusively write to one or more partitions, then Enable exclusive partition access.

Click ADD TOPIC to add new topics. Enter the Topic in the new table row.

Related Topics

Internal

• Kafka Input Processor
• Kafka Output Processor
• Kafka Source
• Create and manage secrets

External

• Confluent Kafka: Producer Configurations (opens new window)

Potential problems

Can't find what you are looking for?

Please note, that the creation of the online documentation is Work-In-Progress. It is constantly being updated. Should you have questions or suggestions, please don't hesitate to contact us at support@layline.io .