Source SQS
Purpose
Amazon Simple Queue Service (Amazon SQS) is a fully managed message queuing service that makes it easy to decouple and scale microservices, distributed systems, and serverless applications. Amazon SQS moves data between distributed application components and helps you decouple these components. Next to Amazon's SQS there are various other message queuing service providers which grant SQS compatible access to their storage solutions as well (e.g. Google Cloud Storage, IONOS, et al).
This UI helps to define the specific bucket and folder source of an SQS connected endpoint.
This Asset can be used by:
Asset type | Link |
---|---|
Input Processors | Frame Input Processor |
Prerequisite
You need:
Configuration
Name & Description
-
Name
: Name of the Asset. Spaces are not allowed in the name. -
Description
: Enter a description.
The Asset Usage
box shows how many times this Asset is used and which parts are referencing it.
Click to expand and then click to follow, if any.
Required roles
In case you are deploying to a Cluster which is running (a) Reactive Engine Nodes which have (b) specific Roles
configured, then you can restrict use of this Asset to those Nodes with matching
roles.
If you want this restriction, then enter the names of the Required Roles
here. Otherwise, leave empty to match all
Nodes (no restriction).
SQS Settings
Configure the parameters for your SQS endpoint:
SQS Connection
Use the drop-down list to select an AWS Connection that should support this Asset. If it does not exist, you need to create it first.
SQS Queue
Once you picked an SQS Connection above the system will try to test the connection and
show available queues within the drop-down list under SQS Queue
. Pick the topic you want to process messages from. Another approach
could be: You can use ${...} macros to expand variables defined in environment variables.
Further settings
Polling duration [sec]
: Configuring the value 0 in here, SQS "short polling" is activated and the request will only query a subset of the servers to find messages. Amazon SQS sends the response right away, even if the query found no messages. Configuring any value greater than 0 in here, SQS "long polling" is activated and the request will query all of the servers for messages. Amazon SQS sends a response after it collects at least one available message, up to theMaximum message count
specified in the request. Amazon SQS sends an empty response only if the here configured polling wait time expires. Find more details about SQS Short and Long Polling through the link.Maximum message count
: parameter relates to the message polling behavior described above: either of the configured limits reached first, the querying of messages will end afterMaximum message count
resp. elapsedPolling duration [sec]
.Visibility timeout [sec]
: The visibility timeout begins when Amazon SQS returns a message. During this time, the consumer has to process and delete the message. You can define this timeout period in here. After the visibility timeout period the message becomes visible to other consumers again if it is not deleted. If a message must be received only once, your consumer should delete it within the duration of the visibility timeout. ( see also SQS Visibility Timeout). Within lalyine.io you need to ensure this behavior by adding the Input Frame Committer processor into your workflow since this is the instance that would finally delete the message from the queue. Please note: DELETE is only executed when message has reached the Input Frame Committer!
Please note, that the creation of the online documentation is Work-In-Progress. It is constantly being updated. should you have questions or suggestions, please don't hesitate to contact us at support@layline.io .