DataWeave Output Formats and Writer Properties

When you import a Copybook definition, the Transform component converts the definition to a flat file schema that you can reference with schemaPath property.

To import a copybook definition:

For example, assume that you have a copybook definition file (mailing-record.cpy) that looks like this:

Note: Copybook definitions must always begin with a 01 entry. A separate record type is generated for each 01 definition in your copybook (there must be at least one 01 definition for the copybook to be usable, so add one using an arbitrary name at the start of the copybook if none is present). If there are multiple 01 definitions in the copybook file, you can select which definition to use in the transform from the dropdown list.

Note: COBOL format requires definitions to only use columns 7-72 of each line. Data in columns 1-5 and past column 72 is ignored by the import process. Column 6 is a line continuation marker.

When you import the schema, the Transform component converts the copybook file to a flat file schema that it stores in the src/main/resources/schema folder of your Mule project. In flat file format, the copybook definition above looks like this:

After importing the copybook, you can use the schemaPath property to reference the associated flat file through the output directive. For example: output application/flatfile schemaPath="src/main/resources/schemas/mailing-record.ffd"

Not all copybook features are supported by the Cobol Copybook format in DataWeave. In general, the format supports most common usages and simple patterns, including:

The most common issue with copybook imports is a failure to follow the Cobol standard for input line regions. The copybook import parsing ignores the contents of columns 1-6 of each line, and ignores all lines with an '*' (asterisk) in column 7. It also ignores everything beyond column 72 in each line. This means that all your actual data definitions need to be within columns 8 through 72 of input lines.

Tabs in the input are not expanded because there is no defined standard for tab positions. Each tab character is treated as a single space character when counting copybook input columns.

Indentation is ignored when processing the copybook, with only level-numbers treated as significant. This is not normally a problem, but it means that copybooks might be accepted for import even though they are not accepted by Cobol compilers.

Both warnings and errors might be reported as a result of a copybook import. Warnings generally tell of unsupported or unrecognized feature, which might or might not be significant. Errors are notifications of a problem that means the generated schema (if any) will not be a completely accurate representation of the copybook. You should review any warnings or errors reported and decide on the appropriate handling, which might be simply accepting the schema as generated, modifying the input copybook, or modifying the generated schema.

When defining an input of type Copybook, there are a few optional parameters you can add in the XML definition of your Mule project to customize how the data is parsed.

Note that schemas with type Binary or Packed don’t allow for the detection of line breaks, so setting recordParsing to lenient only allow for long records to be handled, not short ones. These schemas only work with certain single-byte character encodings (so not with UTF-8 or any multibyte format).

When defining an output of type Copybook, there are a few optional parameters you can add to the DataWeave output directive to customize how the data is written:

In CSV, you can assign any special character as the indicator for separating fields, toggling quotes, or escaping quotes. Make sure you know what special characters are being used in your input so that DataWeave can interpret it correctly.

When defining an input of type CSV, there are a few optional parameters you can add in the XML definition of your Mule project to customize how the data is parsed.

When defining an output of type CSV, there are a few optional parameters you can add to the output directive to customize how the data is parsed:

All of these parameters are optional. A CSV output directive might for example look like this:

In the Transform component, you can define a CSV type through the following methods:

When defining an input of type Excel, there are a few optional parameters you can add in the XML definition of your Mule project to customize how the data is parsed.

When defining an output of type Excel, there are a few optional parameters you can add to the output directive to customize how the data is parsed:

All of these parameters are optional. A DataWeave output directive might for Excel might look like this:

In the Transform component, you can define a Excel type through the following method:

When defining an input of type Fixed Width, there are a few optional parameters you can add in the XML definition of your Mule project to customize how the data is parsed.

When defining an output of type fixed width there are a few optional parameters you can add to the output directive to customize how the data is written:

All of these parameters are optional. A DataWeave output directive might for Excel might look like this:

In the Transform component, you can define a Fixed Width type through the following methods:

When defining an input of type Flat File, there are a few optional parameters you can add in the XML definition of your Mule project to customize how the data is parsed.

Note that schemas with type Binary or Packed don’t allow for line break detection, so setting recordParsing to lenient only allows long records to be handled, not short ones. These schemas also currently only work with certain single-byte character encodings (so not with UTF-8 or any multibyte format).

When defining an output of type flat file there are a few optional parameters you can add to the output directive to customize how the data is written:

In the Transform component, you can define a Flat File type by pointing to a schema file.

You can set the boundary for the reader to use when it analyzes the data.

Note that in the DataWeave read function, you can also pass the property as an optional parameter. The scope of the property is limited to the DataWeave script where you call the function.

The writer output form data using the DataWeave header directive:

In the output directive, you can also set a property for the writer to use when it outputs the data in the specified format.

For example, if a boundary is 34b21, then you can pass this:

Note that in the DataWeave write function, you can also pass the property as an optional parameter. The scope of the property is limited to the DataWeave script where you call the function.

There are a couple of custom Java types:

Java developers use the class metadata key as hint for what class needs to be created and sent as an input. If this is not explicitly defined, DataWeave tries to infer from the context or it assigns it the default values:

The code above defines the type of the required input as an instance of com.anypoint.df.pojo.User.

In order to put an enum value in a java.util.Map, the DataWeave Java module defines a custom type called Enum. It allows you to specify that a given string should be handled as the name of a specified enum type. It should always be used with the class property with the java class name of the enum.

In the Transform component, you can define a Java type through the following method:

When defining an output of type JSON, there are a few optional parameters you can add to the output directive to customize how the data is parsed:

In the Transform component, you can define a JSON type through the following methods:

When defining an input of type XML, there are a few optional parameters you can add in the XML definition of your Mule project to customize how the data is parsed.

When defining an output of type XML, there are a few optional parameters you can add to the output directive to customize how the data is parsed:

The inlineCloseOn parameter defines whether the output is structured like this (the default):

It can also be structured like this (set with a value of empty):

See also, Example: Outputting Self-closing XML Tags.

In the Transform component, you can define a XML type through the following methods:

Mime Type: application/xml

CData is a custom data type for XML that is used to identify a CDATA XML block. It can tell the writer to wrap the content inside CDATA or to check if the input string arrives inside a CDATA block. CData inherits from the type String.

Here is the DataWeave output directive for writing form data:

In the output directive, you can also set a property for the writer to use when it outputs the data in the specified format.

Note that in the DataWeave write function, you can also pass the property as an optional parameter. The scope of the property is limited to the DataWeave script where you call the function.