Source Email
Purpose
Defines the specific source parameters for an Email connected endpoint.
This Asset can be used by:
Asset type | Link |
---|---|
Input Processors | Stream Input Message |
Prerequisite
You need:
Configuration
Name & Description
-
Name
: Name of the Asset. Spaces are not allowed in the name. -
Description
: Enter a description.
The Asset Usage
box shows how many times this Asset is used and which parts are referencing it.
Click to expand and then click to follow, if any.
Required roles
In case you are deploying to a Cluster which is running (a) Reactive Engine Nodes which have (b) specific Roles
configured, then you can restrict use of this Asset to those Nodes with matching
roles.
If you want this restriction, then enter the names of the Required Roles
here. Otherwise, leave empty to match all
Nodes (no restriction).
Polling & Processing
This source does not reflect a stream, but an object based storage source which does not signal the existence of new objects to observers. We therefore need to define how often we want to look-up (poll) the source for new objects to process.
You can choose between Fixed rate polling
and Cron tab style
polling:
Fixed rate
Use Fixed rate
if you want to poll in constant and frequent intervals.
Polling interval [sec]
: Enter the interval in seconds in which the configured source should be queried for new objects.
Cron tab
Use Cron tab
if you want to poll at determined times. The Cron tab expression
follows the cron tab style convention which may be familiar to you.
In all other cases you can read more about crontab and the syntax here.
You can simulate cron settings using this smart website.
Or you can use the Cron expression editor provided underneath the calendar symbol on the right hand side:
Configure your expression with the help of this editor. The Next trigger times at the top helps to visualize the configured expression. Press OK towards the end of this editor window to store the given values.
Polling timeout
The Polling timeout [sec]
defines the time in seconds to wait until a polling request fails.
Depending on the endpoint and its responsiveness you may want to change this number to something higher or lower.
You should set it high enough, so that you are confident that the endpoint responds under normal operation.
Stable time
The Stable time [sec]
defines the number of seconds that file statistics must stay unchanged for considering it as stable.
Configuring a value in here will check the appropriate file stability before processing the file.
Ordering
When listing objects from the source for processing, you can define in what order they should be processed. Pick one of the following self-explanatory settings:
Alphabetically, ascending
Alphabetically, descending
Last modified, ascending
Last modified, descending
Reprocessing mode
Configuring the Reprocessing mode
relates to layline.io's Access Coordinator feature.
You can pick between three modes:
Manual access coordinator reset
: any source element processed and stored in layline.io's history needs manual reset within the Sources Coordinator before reprocessing of a re-ingested source is performed (default mode).Automatic access coordinator reset
: this mode allows the automatc reprocessig of already processed and re-ingested sources as soon as the respective input source has been moved into the configured done or error directory.When input changed
: this mode behaves as described inManual access coordinator reset
while it performs an additional check whether the source has potentially changed; i.e. the name of the source is identical but the content differs. In case the input in fact changed, the reprocessing will start without manual intervention.
Wait for processing clearance
Activating this checkbox identifies new input sources as Wait for processing clearance
. This means, the input source
leaves unprocessed in the input directory until a) a manual intervention through Operations provides active clearance for processing the respective file or
b) some Javascript executes the AccessCoordinator.giveClearance(source: string, stream: string, timeout?: number)
method.
Email Settings
Configure the parameters for your Email endpoint:
Connection
Use the drop-down list to select an Email Connection that should support this Email configuration. If it does not exist, you need to create it first.
Your Email Connection needs to have the following configured scope:
- mail.readwrite
Stream name
By its nature, an Email Source does not appear with a classical stream name. At this point you define the stream name for later identification during processing within a workflow:
Stream name
: name to apply for the email workflow stream processing
You need to ensure that the name is unique. As you can see from the example above, you can use Macros here.
In the above case we chose to include several constants like subject:
/ from:
/ received:
etc. In addition, we included
several classical email identification elements deducted from Macros to form the email stream name. Finally, the Audit trail for
data processed through the configured Email source will look similar to a classical email inbox folder:
Folders
-
Input folder
: The directory to read emails from. The configured input folder must be accessible to the Reactive Engine trying to access the Email source. You can use ${...} macros to expand variables defined in environment variables. -
Done folder
: The directory to which emails are moved when fully processed. Leaving it empty, the emails will not be moved. The configured done folder must be accessible to the Reactive Engine trying to access the Email source. You can use ${...} macros to expand variables defined in environment variables. -
Error folder
: The directory to which emails are moved in case of a problem with the email during processing. Leaving it empty, the emails will not be moved. The configured error folder must be accessible to the Reactive Engine trying to access the Email source. You can use ${...} macros to expand variables defined in environment variables. -
Ignore folder
: The directory to which emails are moved in case they are configured to be ignored in theIgnore emails
section (see next). Leaving it empty, the emails will not be moved. The configured ignore folder must be accessible to the Reactive Engine trying to access the Email source. You can use ${...} macros to expand variables defined in environment variables.
A final configuration could look like this:
Ignore Emails
By default, all emails arriving in the configured Input folder will be processed. Though it is possible to define which emails are not relevant (Blacklist) resp. explicitly relevant (Whitelist) for processing by activating the Ignore Emails checkbox.
From filter
: Blacklist resp. Whitelist definitions to apply towards email sender valuesSubject filter
: Blacklist resp. Whitelist definitions to apply towards the email subject values
Regular expression values can be configured by pushing the respective + ADD TO BLACKLIST
resp. + ADD TO WHITELIST
button.
Logically layline.io will check any configured blacklist configuration first and evaluate any whitelist configuration afterwards.
In case any Blacklist expression is matching an incoming email, the email will be ignored. Having configured an Ignore folder, that specific email will be moved towards that folder. Otherwise, the email remains unprocessed in the Input folder.
Any incoming email matching the Whitelist expression will lead to processing that email and moving it into the Done folder
or the Error folder
afterwards.
In case no specific folder is configured the email will remain in the Input folder.
-
If you configure only Whitelist expressions, this will implicitly mark all others to be ignored
-
If you configure only Blacklist expressions, this will implicitly define all others to be processed
-
Configuring a combination of both (i.e. whitelist AND blacklist have at least one entry each!), it is important to understand the following:
The order of expression evaluation will always be:
- First Step: Ignore emails matching blacklist
- Second Step: Process emails matching whitelist
- Third Step: Ignore emails not falling in any category
Attachments settings
By default, attachments to emails are not loaded. Though it is possible to allow attachment loading by activating the Load attachments checkbox and configure the details:
-
Maximum size of all loaded attachments [kb]
: limit the overall size of attachments per processed email to not overload your system capacity (value configured in kb) -
Mime type filter
: filter conditions to further restrict the loading based on type of attachment and / or size:Blacklist
:Mime type regular expression
: similar as explained above, a regular expression for type of attachments that should be ignored (e.g. 'application/.*' to exclude all attachments of 'application'-related mime types)Size constraints [kb]
: optional parameter that could be used to exclude only those type of attachments with a size greater than the configure size value; no configured value will ignore this parameter
Whitelist
:Mime type regular expression
: similar as explained above, a regular expression for type of attachments that should be included for processing (e.g. 'application/.*' to include only attachments of 'application'-related mime types)Size constraints [kb]
: optional parameter that could be used to include only those type of attachments with a size less or equal than the configure size value; no configured value will ignore this parameter. Please note that the configured maximum size takes precedence in the overall setup!
-
If you configure only Whitelist expressions, this will implicitly mark all others to be ignored
-
If you configure only Blacklist expressions, this will implicitly define all others to be processed
-
Configuring a combination of both (i.e. whitelist AND blacklist have at least one entry each!), it is important to understand the following:
The order of expression evaluation will always be:
- First Step: Ignore emails matching blacklist
- Second Step: Process emails matching whitelist
- Third Step: Ignore emails not falling in any category
Please note, that the creation of the online documentation is Work-In-Progress. It is constantly being updated. should you have questions or suggestions, please don't hesitate to contact us at support@layline.io .