Sink Kafka
Purpose
Defines the outbound connection parameters for a Kafka sink.
This Asset can be used by:
Asset type | Link |
---|---|
Output Processors | Frame Output Processor |
Prerequisite
You need:
Configuration
Name & Description
Name
: Name of the Asset. Spaces are not allowed in the name.
Description
: Enter a description.
The Asset Usage
box shows how many times this Asset is used and which parts are referencing it. Click to expand and then click to follow, if any.
Required roles
In case you are deploying to a Cluster which is running (a) Reactive Engine Nodes which have (b) specific Roles configured, then you can restrict use of this Asset to those Nodes with matching
roles.
If you want this restriction, then enter the names of the Required Roles
here. Otherwise, leave empty to match all Nodes (no restriction).
Kafka Connection
Select the Kafka Connection to use with this Asset. If it does not exist, you need to create it first.
Kafka Producer Settings
General
Parallelism
: In order to accelerate writing to a Kafka topic, you can parallelize this process. Entering100
here, for example allows layline.io to have 100 messages "in-flight" to Kafka in parallel to be written to a Kafka topic.
Additional Kafka Properties
Use this section to add configuration parameters available for Kafka Platform such as compression.type
or even ssl.keystore.key
.
For a list of available properties please check Confluent Producer Configurations.
Please note that properties defined here, take precedence over all other settings you may have provided in this UI. You can use this to add, or override Kafka properties using these settings.
Exclusive partition access
In case you want to exclusively write to one or more partitions, then Enable exclusive partition access
.
Click ADD TOPIC
to add new topics. Enter the Topic
in the new table row.
Related Topics
Internal
External
Potential problems
Please note, that the creation of the online documentation is Work-In-Progress. It is constantly being updated. should you have questions or suggestions, please don't hesitate to contact us at support@layline.io .