For Each Scope

Mule Runtime Engine versions 3.5, 3.6, and 3.7 reached End of Life on or before January 25, 2020. For more information, contact your Customer Success Manager to determine how you can migrate to the latest Mule version.

The For Each scope splits a collection into elements and processes them iteratively through the processors embedded in the scope, then returns the original message to the flow.

Complete Example Code

Using For Each versus Splitter and Aggregator Pairs

Typically, to create a flow that processes messages containing a collection of elements you must use one flow control to split the collection into individual elements, which the flow processes iteratively, then you must use another flow control to re-aggregate the elements into a new collection so they can be passed out of the flow. Several of Mule ESB’s flow controls support such splitting-and-re-aggregation, but they cannot match the convenience of the For Each scope.

This use of splitter and aggregator pairs also presents several potential limitations:

Under some circumstances, splitting a message collection into pieces can cause certain vital bits of XML — metadata in the header or footer, for example — to be dropped from the re-aggregated XML.
When you split certain collection types — Java, for example — into many pieces for processing, the collection may be re-aggregated into a different collection type — MuleMessageCollection, for example. (As a result, you may need to add extra flow steps to transform the processed message collection back into its original collection type.)
When you split, process, or aggregate a message collection, you must choose among several splitter and aggregator types. Sometimes, it proves difficult to determine which splitter/aggregator combination best suits your message processing needs.

For Each provides several advantages over splitter-aggregator pairs:

For Each splits collections into elements, then processes them iteratively without losing any of the message payload.
After For Each splits a message collection and processes the individual elements, it doesn’t re-aggregate those individual elements into a MuleMessageCollection; rather, it returns the original message. (This results in "Java in, Java out" rather than "Java in, MuleMessageCollection out.")
The For Each scope is versatile; it can iteratively process elements from any type of collection, including maps, lists, arrays, and MuleMessageCollections.
The For Each scope can split and process collections of elements that are not part of the message payload. For example, For Each can process message property collections (metadata) from the message header.

If the elements are mutable objects (for example, a bean with different fields in it) and any of the message processors embedded in your For Each scope modifies a value of the original object (for example, one of the fields), then the changes to the fields will persist once the For Each returns the original message to the flow.

In other words, For Each does not make a deep copy of the data when processing a message nor when returning the original message to the flow.

Configuring For Each

STUDIO Visual Editor

Add a For Each scope in your flow at the point where you want to initiate the For Each scope processing.

Insert one or more message processors inside the For Each scope area to define how Mule should process each element within the message collection. The For Each scope can contain any number of message processors as well as references to child flows.

The only type of message processor you cannot drag into a For Each scope is an inbound connector.

If you drag a two-way connector into a For Each scope, Mule automatically converts it to an outbound-only connector.

Field Default Value Description XML

Field	Default Value	Description	XML
Display Name	For Each	Customize to display a unique name for the scope in your application.	`doc:name="For Each"`
Collection		(Optional) Enter an expression that tells For Each where to find the data it must split and process. For example, enter an expression that instructs For Each to split and process a collection from the header section – rather than the payload. Unless this field specifies otherwise, For Each assumes that the message payload is the collection.	`collection="#[payload.topic]"`
Counter Variable Name	counter	(Optional) Enter a name in this field to label the variable that For Each uses to record the number of the elements it has processed. If your collection already uses the label `counter` for another variable, this field will be blank and you will need to enter a different label for the Counter Variable Name, such as `index`.	`counterVariableName="counter"`
Batch Size	1	(Optional) Enter an integer to indicate the number of elements in each batch that For Each processes. Potentially, these batches promote quicker processing. If greater than one, each batch is treated as a separate Mule message. For example, if a collection has 200 elements and you set the batch size to 50, For Each will iteratively process 4 batches of 50 elements, each as a separate Mule message.	`batchSize="50"`
Root Message Variable Name	rootMessage	(Optional) Enter a name in this field to label the variable that For Each uses to reference the complete, unsplit message collection. If your collection already uses the label `rootMessage` for another variable, this field will be blank and you will need to enter a different label for the Root Message Variable Name.	`rootMessageVariableName="rootMessage"`

Display Name

For Each

Customize to display a unique name for the scope in your application.

doc:name="For Each"

Collection

(Optional) Enter an expression that tells For Each where to find the data it must split and process. For example, enter an expression that instructs For Each to split and process a collection from the header section – rather than the payload. Unless this field specifies otherwise, For Each assumes that the message payload is the collection.

collection="#[payload.topic]"

Counter Variable Name

counter

(Optional) Enter a name in this field to label the variable that For Each uses to record the number of the elements it has processed. If your collection already uses the label counter for another variable, this field will be blank and you will need to enter a different label for the Counter Variable Name, such as index.

counterVariableName="counter"

Batch Size

(Optional) Enter an integer to indicate the number of elements in each batch that For Each processes. Potentially, these batches promote quicker processing. If greater than one, each batch is treated as a separate Mule message. For example, if a collection has 200 elements and you set the batch size to 50, For Each will iteratively process 4 batches of 50 elements, each as a separate Mule message.

batchSize="50"

Root Message Variable Name

rootMessage

(Optional) Enter a name in this field to label the variable that For Each uses to reference the complete, unsplit message collection. If your collection already uses the label rootMessage for another variable, this field will be blank and you will need to enter a different label for the Root Message Variable Name.

rootMessageVariableName="rootMessage"

XML Editor or Standalone

Add a For Each element to your flow at the point where you want to initiate a For Each processing block. Refer to the code sample below.

Element Description

For Each

Use to create a block of message processors that iteratively process elements within a collection.

Element	Description
For Each	Use to create a block of message processors that iteratively process elements within a collection.

Configure the scope according to the table below.

Element Attribute Default Value Description

Element Attribute	Default Value	Description
doc:name	For Each	Customize to display a unique name for the async scope in your application. Note: Attribute not required in Mule Standalone configuration.
collection	Payload	(Optional) Enter an expression that tells For Each where to find the data it must split and process. For example, enter an expression that instructs For Each to split and process a collection from the header section – rather than the payload. Unless this attribute specifies otherwise, For Each assumes that the message payload is the collection.
counterVariableName	counter	(Optional) Specify to label the variable that For Each uses to record the number of the elements it has processed. If your collection already uses the label `counter` for another variable, you will need to select a unique name.
batchSize	1	(Optional) Specify an integer to indicate the number of elements in each batch that For Each processes. Potentially, these batches promote quicker processing. For example, if a collection has 200 elements and you set the batch size to 50, For Each will iteratively process 4 batches of 50 elements.
rootMessageVariableName	rootMessage	(Optional) Specify to label the variable that For Each uses to reference the complete, unsplit message collection. If your collection already uses the label `rootMessage` for another variable, you will need to select a unique name.

doc:name

For Each

Customize to display a unique name for the async scope in your application.

Note: Attribute not required in Mule Standalone configuration.

collection

Payload

(Optional) Enter an expression that tells For Each where to find the data it must split and process. For example, enter an expression that instructs For Each to split and process a collection from the header section – rather than the payload. Unless this attribute specifies otherwise, For Each assumes that the message payload is the collection.

counterVariableName

counter

(Optional) Specify to label the variable that For Each uses to record the number of the elements it has processed. If your collection already uses the label counter for another variable, you will need to select a unique name.

batchSize

(Optional) Specify an integer to indicate the number of elements in each batch that For Each processes. Potentially, these batches promote quicker processing. For example, if a collection has 200 elements and you set the batch size to 50, For Each will iteratively process 4 batches of 50 elements.

rootMessageVariableName

rootMessage

(Optional) Specify to label the variable that For Each uses to reference the complete, unsplit message collection. If your collection already uses the label rootMessage for another variable, you will need to select a unique name.

Add nested elements beneath your foreach element to define how Mule should process each element within the message collection. The For Each scope can contain any number of message processors as well as references to child flows.
```
<foreach collection="#[payload.name]" doc:name="For Each" counterVariableName="counter" rootMessageVariableName="rootMessage" batchSize="5">
    <some-nested-element/>
    <some-other-nested-element/>
</foreach>
```

For Each Error Handling

The exception strategy defined for your flow handles all the exceptions thrown within the For Each scope. (If you have not explicitly defined an exception strategy for your flow, Mule implicitly applies the default exception strategy to handle exceptions.) If a message in a collection throws an exception, For Each stops processing that collection and invokes the exception strategy.

For example, Foreach throws an IllegalArgumentException whenever two conditions hold true:

it receives a message payload that is not a collection
you have not identified a message collection outside the message payload (defined by entering an expression in the Collection field in the Studio Visual Editor or including the collection attribute in XML configuration.)

Considerations when Persisting Data

In case the message inside the For Each scope is persisted, not only the item in the collection will be serialized but also all the variables associated with the current message. The rootMessage variable, associated with the message, contains a reference of the complete, unsplit message collection that could potentially be holding thousands of items. Therefore, serialization/deserialization of the rootMessage variable could impact memory consumption considerably when this collection is large enough.

To avoid this issue you must first remove the rootMessage variable from the message before persisting it. For this you can use the <remove-variable> element like so:

<remove-variable variableName="rootMessage" doc:name="Variable"/>

In Studio, you can drag a Variable message processor inside your scope and set it to "Remove Variable".

Example

The following example illustrates a flow that uses For Each to add information to each message in a collection.

The HTTP connector receives a request from a client, then queries a JDBC database, where a table indicates the model names and the model years of various cars. For Each breaks the collection (the table) apart into a list of elements (rows), each of which contains information such as about individual elements (maps) model:'ford sierra', model_year=1982}}. For Each sends each element through the message processors in its scope.

The flow adds a new entry to each element’s map; if the model year is less than 2001, Mule adds type='20th century car', then sends the element to the JMS connector; otherwise, Mule adds type='21st century car' and sends the element to the File connector. For Each returns a collection at the end of the flow and sends it to the transformer.

This particular example replaces the main flow’s default exception strategy with a custom Catch Exception Strategy that leverages the Set Payload and HTTP Response Builder building blocks.

Complete Example Code

<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:db="http://www.mulesoft.org/schema/mule/db" xmlns:spring="http://www.springframework.org/schema/beans" xmlns:jdbc-ee="http://www.mulesoft.org/schema/mule/ee/jdbc" xmlns="http://www.mulesoft.org/schema/mule/core"
      xmlns:http="http://www.mulesoft.org/schema/mule/http"
      xmlns:file="http://www.mulesoft.org/schema/mule/file"
      xmlns:jdbc="http://www.mulesoft.org/schema/mule/jdbc"
      xmlns:jms="http://www.mulesoft.org/schema/mule/jms"
      xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting"
      xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
      xmlns:core="http://www.mulesoft.org/schema/mule/core"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/ee/jdbc http://www.mulesoft.org/schema/mule/ee/jdbc/current/mule-jdbc-ee.xsd
http://www.mulesoft.org/schema/mule/jms http://www.mulesoft.org/schema/mule/jms/current/mule-jms.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd">

    <jms:activemq-connector name="JMSConnector" doc:name="Active MQ"></jms:activemq-connector>
    <http:listener-config name="HTTP_Listener_Configuration" host="localhost" port="9091" doc:name="HTTP Listener Configuration"/>
    <db:derby-config name="Derby_Configuration" url="jdbc:derby:${app.home}/muleEmbeddedDB;create=true"   doc:name="Derby Configuration"/>

    <flow name="process" >
        <http:listener config-ref="HTTP_Listener_Configuration" path="process" doc:name="HTTP">
            <http:error-response-builder statusCode="500" reasonPhrase="You need to populate the Database first"/>
        </http:listener>
        <db:select config-ref="Derby_Configuration" doc:name="Database">
            <db:parameterized-query><![CDATA[SELECT * FROM cars]]></db:parameterized-query>
        </db:select>
        <foreach doc:name="Foreach">
            <choice doc:name="Choice">
                <when expression="payload.'MODEL_YEAR' &#38;lt; 2001">
                    <processor-chain doc:name="Processor Chain">
                        <expression-component doc:name="Set payload type"><![CDATA[payload.'TYPE' = '20th century car']]></expression-component>
                        <jms:outbound-endpoint connector-ref="JMSConnector" queue="in" doc:name="JMS"></jms:outbound-endpoint>
                    </processor-chain>
                </when>
                <otherwise>
                    <processor-chain doc:name="Processor Chain">
                        <expression-component doc:name="Set payload type">payload.'TYPE'='21st century car'</expression-component>
                        <file:outbound-endpoint path="/tmp" responseTimeout="10000" doc:name="File"></file:outbound-endpoint>
                    </processor-chain>
                </otherwise>
            </choice>
        </foreach>
        <set-payload value="#[payload.size()] cars where processed: #[payload]" doc:name="Set response"></set-payload>
        <parse-template location="foreach_info.html" doc:name="Parse Template"/>
        <catch-exception-strategy doc:name="Catch Exception Strategy">
            <parse-template location="foreach_error.html" doc:name="Parse Template"/>
        </catch-exception-strategy>
    </flow>
    <flow name="populate" >
         <http:listener config-ref="HTTP_Listener_Configuration" path="populate" doc:name="HTTP">
            <http:error-response-builder statusCode="500" reasonPhrase="DB already populated"/>
        </http:listener>

        <scripting:component doc:name="Script to populate DB">
            <scripting:script engine="Groovy">
                <scripting:text><![CDATA[jdbcConnector = muleContext.getRegistry().lookupConnector("JDBCConnector");
qr = jdbcConnector.getQueryRunner();
conn = jdbcConnector.getConnection();
qr.update(conn, "CREATE TABLE cars (model varchar(256), model_year integer)");
qr.update(conn, "INSERT INTO cars values('Ford Sierra', 1982)");
qr.update(conn, "INSERT INTO cars values('Opel Astra', 2001)");]]></scripting:text>

            </scripting:script>
        </scripting:component>
        <set-payload value="Successfully populated the database" doc:name="Set Payload"></set-payload>
        <parse-template location="foreach_info.html" doc:name="Parse Template"/>
        <catch-exception-strategy doc:name="Catch Exception Strategy">
            <parse-template location="foreach_error.html" doc:name="Parse Template"/>
        </catch-exception-strategy>
    </flow>
</mule>

Gartner names MuleSoft a Leader

Unleash the power of Salesforce Customer 360 through integration

Anypoint Platform Fundamentals

The world of AI is a world of APIs