AI Nodes enhances the capabilities of Large Language Models (LLMs) beyond Decision Flow authoring, which is currently supported by Copilot, to include execution. This extension provides a versatile tool for a wide range of tasks. It integrates seamlessly with Decision Flows, passing relevant context to improve model responses, ensuring more consistent and usable outputs. 

The AI Node can be used for tasks such as extracting key financial data from documents, translating text, classifying industries, and gathering geographic information, enabling the automation of complex workflows efficiently.

After adding the AI node a a Flow, follow the steps below to configure the node.

Model selection

The AI Node allows you to select from various AI models through the Select AI model dropdown. Available models may differ depending on your workspace region, but all regions offer at least two options with distinct performance characteristics.

For example, in the image below, you can see the models available:

Your choice of model significantly affects the node's performance - selecting a model essentially involves balancing output quality against node execution time.

• Claude 3.7 Sonnet: Provides accurate, complete, and consistent responses but is slightly slower. Best suited for complex or less well-defined tasks.
• Claude 3.5 Sonnet: The predecessor to Claude 3.7 Sonnet — best suited for the same class of tasks, but with lower complexity.
• Claude 3.5 Haiku: Fastest reasoning model available, but may struggle with complex instructions or data. It is ideal for simple, high-volume tasks.
• Claude 3 Haiku: The predecessor to Claude 3.5 Haiku — best suited for the same class of tasks, but with lower complexity.
• Amazon Nova Pro: Cost-efficient multimodal model — best suited for medium-complexity tasks without strict latency requirements.

The exact set of models available to you depends on the AWS region where your workspace is located. 

Prompt authoring and output configuration

The AI Node supports two distinct output formats:

1. Plain text output - Stored in a single data object property
2. Structured output - Comprises multiple user-defined properties mapped to corresponding data object properties

The output format you choose will affect how you construct your prompt to achieve optimal reliability and effectiveness. To demonstrate how to configure both output types we will use a credit report analysis scenario:

Scenario: You have a Decision Flow that receives a company's credit report from an external data vendor as a JSON object. This report contains both the company's transactions and those of its subsidiaries for the previous 90 days.

You need to extract these specific insights from the file:

The company's legal name

The legal names of all company subsidiaries

Whether any payment defaults (i.e. missed a scheduled payment) occurred in the last 30 days

The total value of assets (including subsidiaries) converted to EUR using exchange rates from the report's generation date

If the credit report's "item code" (assigned by the provider) is 250, no insights should be extracted.

Plain text output configuration

1. Enter task instruction, input parameters, and insights in the Prompt > Message text box:
   1. You can access properties of Decision Engine objects (data, params, and meta) within the prompt using access expressions prefixed with $. These expressions will be substituted for their values at decisioning time.
   2. To improve parsing accuracy in Anthropic's Claude models, structure the prompt using XML tags.
   3. Include specific presentation instructions for consistent output formatting (e.g. “Place each name on a separate line…”).
2. Set Configure AI response to Text output.
3. Assign the output to a designated data object property.

Structured output configuration

1. Enter task instructions and input parameters in the Prompt > Message text box:
   1. Unlike the text output, the prompt for structured output does not include insights, as these will be manually configured in the next steps.
   2. Set Configure AI response to Structured output.
2. For each of the insights you want to extract, add a separate field using the Add field button:
   1. Map each output field directly to a data object property.
   2. Specify the data type for each field. The node will error if the AI model is unable to provide a value of that type (e.g. due to conflicting instructions).
   3. Add field-specific instructions int he description. 

Advanced settings

The AI node offers additional customization options through the Advanced settings section in the node editor. These settings allow you to fine-tune the AI model's behavior and output. 

System prompt

The system prompt allows you to assign a role to the AI model or provide additional context about the domain it should operate in. This helps improve the accuracy of the AI's responses and ensures it remains focused on the desired domain.

Similar to the prompt message, you can reference Decision Engine object properties here using $-notation.

Environment configuration

With these settings, you can configure whether the AI Node uses mock data or responds with a live model in various Decision Engine environments. Using mock data can speed up Decision Flow testing and prevent using up your usage limits. However, live model responses are ideal for refining prompts or performing final/acceptance testing. In production, the AI Node must use live model responses.

Other settings

Stop Sequences

You can define up to four sequences of characters that will prompt the AI model to stop and return its output once detected.

Max Tokens

You can limit the response length of the AI model by setting a maximum number of tokens (roughly equivalent to words). If the model approaches this limit, it will stop and return its output.

Temperature

The Temperature setting controls the level of randomness in the AI's response, with values ranging from 0 (more deterministic) to 1 (more random). A higher value makes the model’s output more creative, but can also decrease the clarity or coherence of the response.

Custom Timeout

By default, the AI Node runs until completion for synchronous decisions (up to a 30-second decision timeout) or for up to 2 minutes for asynchronous decisions. You can set a custom timeout value, and if this limit is reached, the AI Node will throw an error.

Skip AI Node on Error

When enabled, this setting allows the AI Node to be skipped if it exceeds a workspace's usage limit (e.g., quota or rate limit per minute). When this occurs, all output fields will be set to None, regardless of whether text or structured output is being used.

For more details on the AI Node usage limits, refer to the Limitations section below.

Limitations

The AI Node within each workspace is subject to the following usage restrictions free of charge:

• 100,000 input and output tokens per month
• 5 requests per minute
• 10,000 tokens per minute
• 5 rows for test runs with a live AI Node
• Documents must be smaller than 4.5 MB

You can also check your workspace's AI Node usage by hovering over Usage limits.

Assignment nodes are used to add fields to the data running through your Decision Flow or change the value of an existing field.

You can make as many assignments in a single node as you would like. Use Add Field to add an assignment.

Each assignment consists of two parts: A name on the left and a value on the right. We recommend using names that start with a letter and consist of upper and lower case letters, numbers, and the underscore symbol (_). This makes it easier to reference the new variables later in the Decision Flow.

Allowed values

The value can be any basic Python object or expression:

1. strings/text must be quoted using either double or single quotes.
2. numbers can be Python integers or floating point numbers. _ can be used as a separator for thousands and scientific notation (e.g. 3.45e-4) is also possible.
3. list items should be in square brackets and comma-separated. Items can be of any type including mixed types. As above, strings must be quoted.
4. python dictionaries are also valid values. They can be defined in curly brackets or using the dict keyword
5. as elsewhere within the Decision Engine, you can use None to indicate that a value is not applicable

It is also possible to use more complex expressions and short formulas in assignment nodes, including references to and combinations of other fields in data and built-in Python functions such as min or max or functions from supported modules.

Code Nodes are the most flexible nodes in a Decision Flow. Any Python code, including imports and function definitions, can be used.

Code nodes support basic autocomplete and syntax highlighting. You can open a search/replace menu by pressing cmd + f (Mac) or ctrl + f (Windows)

Python version and package availability

The Decision Engine's runtime is based on Python 3.10, and all packages in the standard library are available for use. In addition, the following packages are supported:

boto3 1.26.90
cryptography 43.0.3
numpy 1.26.4
numpy-financial 1.0.0
onnxruntime 1.16.3
pandas 1.5.3
requests 2.32.3
textdistance 4.6.0
tokenizers 0.15.0
rapidfuzz 3.12.2

Raising HTTP errors from Code Nodes

Sometimes, it may not be possible to complete a Decision Flow calculation, for example, because a key piece of data is missing. In this case, it is possible to abort the Decision Flow calculation and return an error code via the API.

Code Nodes have access to a special error type called HTTPError with the signature HTTPError(status_code: int, message: t.Optional[str]). If this error is triggered, the flow will be aborted and an error message shown in the UI, or the chosen status code returned in case of an API call.

Common use cases

Calculating age from a date of birth variable

You can calculate the age from a 'date_of_birth' variable (assuming the date type for the date_of_birth field) using the following code in a code node. Please note that the code assumes the application_date variable is part of the input schema, which is generally recommended as a best practice for easy backtesting of your decision logic.

from datetime import date

def calculate_age(date_of_birth: date, now: date) -> int:
    """
    Calculate the age based on date of birth and the current date.

    :param date_of_birth: Date of birth as a datetime.date object.
    :param now: Current date as a datetime.date object.
    :return: Age as an integer.
    """
    # Calculate initial age
    age = now.year - date_of_birth.year

    # Adjust age if the current date hasn't reached the birthday of the current year
    if (now.month, now.day) < (date_of_birth.month, date_of_birth.day):
        age -= 1

    return age

data.age = calculate_age(data.date_of_birth, data.application_date)

Using current date

To get the current date in Python, you can use the following line of code:

datetime.date.today()

This will return the current date as a datetime.date object in the format YYYY-MM-DD.

Custom Connection Nodes are flexible tools for integrating with external APIs. Unlike Integration Nodes, they support various authentication methods and allow you to configure distinct credentials for both sandbox and live environments.

There are two main components to using a Custom Connection Node within Decision Flows:

1. Custom Connection: This can only be set up by a workspace administrator. It involves selecting the authentication type and providing all the necessary information to connect to the external API.
2. Custom Connection Node: Decision authors use this component to interact with the external connection within a Decision Flow.

The Database Connector allows you to connect and work with external databases. This means you can effortlessly link your database systems with the Decision Engine, helping you make smarter decisions and improve business results.

There are two main components to integrating databases in Decision Flows:

1. Database Connection: Only a workspace administrator can set this up. It involves choosing the authentication type and filling in all the necessary details to connect to the database.
2. Database Connection Node: Decision authors can use this to interact with the database within a Decision Flow.

The Decision Flow Node allows you to call another Decision Flow in your workspace. This can help you abstract and reuse core parts of your logic.

Choosing between Grouping and Decision Flow nodes

When deciding whether to use Grouping or Decision Flow Nodes in a Decision Flow, consider the following:

Use Decision Flow Nodes when:

• You need to reuse logic across multiple Decision Flows or multiple times within the same flow.
• You want to develop and test a part of the logic separately from the rest of the Decision Flow.

Use Grouping when:

• You want to organize and visually segment logic within a single Decision Flow without creating a separate flow.
• You need an easier way to mock Integration Nodes during testing.
• You want to avoid the slight performance overhead associated with calling a separate Decision Flow.

In summary, use Decision Flow Nodes for reusability and modularity, while Grouping is better for organization and performance within a single flow.

Setting up a Decision Flow Node

Let's start with an existing Flow. In the example above, you can see a basic consumer loan underwriting Flow. It starts with a group of simple early reject rules in the form of Rule Nodes. Let's assume that we would like to reuse this early rejection logic in a different flow. One way to achieve this in a maintainable way is to move it to its own Flow.

Let's create a new Decision Flow called Knock Out Rules and copy over our Rule Nodes. In the Underwriting Flow, remove the early rejection related nodes and add a Decision Flow Node, instead. You will be prompted to select first a Flow and then a version. Note how your Decision Flow Node is automatically named according to the Child Flow you selected (1, below) - you can always change this name later. Click on the arrow symbol (2) to open a Child Flow editor in a new window.

If your Child Flow does not have a schema yet you will receive a warning because a schema is required on the called Flow. Click on "Define Schema" (3) to jump to its schema panel. After you have set an input and output schema, return to the Parent Flow and click on the Decision Flow Node again. The schema of your Child Flow is now reflected on the node and you can start mapping the inputs and outputs of the Child Flow.

Mapping inputs and outputs

The mapping screen displays the input and output fields of the Child Flow, along with their types. It also indicates whether each field is required, nullable, or marked as sensitive.

Mapping can be performed either manually or automatically by clicking on Auto-map fields. Auto-mapping leverages the following information:

1. The Decision Flow’s Schema
2. Available types and variables in the data object, meta and params
3. Previously defined mappings (e.g., if data.user.age has been mapped to age and data.user.name to name, the LLM can infer that data.user.email should be mapped to email)
4. Typing information (e.g., recognizing that data.name is a string and data.age is an integer)

When assigning inputs, all required fields, indicated by an asterisk (*), must be set, while nullable fields, indicated with “null,” can be left as None.

Fields marked with the eye symbol are sensitive and will not be stored in Decision History. Inputs can reference data, parameters, or short Python expressions. Optional fields can be left unassigned, in which case the Decision Engine defaults them to None in the Child Flow.

Outputs from the Child Flow can be assigned to fields in the current Decision Flow, either creating new fields or overriding existing ones. If you wish to ignore an output, leave the corresponding field blank.

Preventing sensitive data from being stored in Decision History

To ensure that sensitive data is never stored in the Decision History of either the Parent Flow or the Child Flow, the relevant fields in both Decision Flows must have the Do not store in Decision History toggle enabled.

Example:

Consider a scenario where:

• The Parent Flow contains a field ssn, which is passed to the Child Flow as social_security_number.
• The Child Flow generates a field name, which is returned and mapped to full_name in the Parent Flow.

To maintain data privacy, enable the Do not store in Decision History toggle for the following fields:

• ssn and full_name in the Parent Flow
• social_security_number and name in the Child Flow

Testing and debugging

Once you have mapped the inputs and outputs of the Child Flow in your Decision Flow Node, you can test run your Parent Flow. If the test run succeeds, the Decision Flow Node's intermediate results tab will show the results of the Child Flow execution that were mapped back to the Child Flow output.

With no mock data set (see below) the Child Flow will execute just like it would be if you test ran it directly. That is, Connect Nodes will not reach out to external APIs unless explicitly configured to do so during test runs. If the Child Flow contains a Connect Node that is not configured to make a live call during testing, the corresponding test sample will be skipped.

Any errors that occur during Child Flow execution are shown on the Decision Flow Node card. The error message also indicates the node where the issue occurred. To further debug an issue, you can create a sample dataset for the Child Flow that is based on your current test run. Click on "Create dataset" on the Decision Flow Node. You will be asked to enter a data set name. You can then jump to the Child Flow and test run it with the new data set.

Exploring dependencies and publishing

Now that your Decision Flow Node has been set up, you can see the dependencies between the two Flows on the Flow Versions page.

Certain safety mechanisms are in place to ensure that published logic does not accidentally get altered or broken:

1. You can only publish a Flow if all of its Child Flows have been published.
2. You can only archive a Flow if all Flows depending on it have already been archived.

Decision History

When calling the Parent Flow from the API, an entry will be created in both the Decision History of the Parent Flow and that of the Child Flow. This allows you to, for example, create a backtesting dataset for a Child Flow called by multiple other Flows. Child Flow Decisions which were triggered by another Flow show a "Called by" entry in their Decision History. Click on it to jump to the corresponding entry in the calling Flow's history (2).

To find a specific entry for the Child Flow based on the Parent Flow's decision id, head to the Child Flow's Decision History and search for the Parent Flow's decision id (1).

Changing the called Flow version

Having split up your logic across two Flows now allows you to develop each piece independently. For example, you might create a new version of the knock-out Flow, test it, and publish it once you are satisfied with its performance. The Decision Flow Node makes it easy to update to a new version. Simply click "Change Version" and select the new version. Where possible, input and output mappings will be preserved when switching versions.

Decision Flow Node limitations

Child Flow Call Limit: You have a limit of 20 Child Flow calls in a single Decision Flow. This includes both Loop Nodes and Decision Flow nodes. So, you can have a combination of these nodes, as log as the total doesn't exceed 20.

Decision Table Nodes allow you to express complex conditions in a tabular format. Each row in the table represents a case, white columns represent conditions, and green columns represent results.

Similarly to the Rule Node, conditions are evaluated from top to bottom. Results are set for the first case that has all of its conditions met (AND conditions). Decision Table Nodes also have a special is any operator that will always match for the given case.

In the example below, the data fields delinquencies and income are required inputs. The node will set the fields to accept and limit based on the first matching condition. For example,

• an application with a delinquency would trigger the case "delinquent" regardless of income. data.accept would be set to False and data.limit would be set to None.
• an application with no delinquencies and an income of 80,000 would trigger the case “income medium”, data.accept would be set to True and data.limit would be set to "medium".

Creating a Decision Table

When adding a Decision Table node, it initially contains a simple table with one case, one condition, and one result.

• You can use Add Case, Add Condition, and Add Result to add cases, conditions and results.
• Set your condition and results headers. Condition headers typically represent fields in the data object used as conditions, results header represents the variable name that is being created/modified.
• To better track your cases, you can also name them.
• To add new cases, conditions, or results in specific places or to duplicate or delete them, you can use the context menus. Hover over a cell to find these.
• You might find that your use case requires a bigger table than the one that can fit into the side pane. Click the double arrow to enter full-screen mode.

Decision Table Logic

Most conditions in the Decision table consist of three parts: a left-hand side (1), an operator (2), and a right-hand side (3). Some operators, such as is any or is None do not require a right-hand side.

Results always have a target field (4) and a value (5). The target field always has to be of the format data.<field name>.

As many other places across the Decision Engine, you can use the power of Python expressions to create more complex logic.

Debugging your Decision Table

In case of an error, cells with issues will be highlighted. Hover over the exclamation mark to see details.

FAQ

Webhooks are URLs that allow external systems to send real-time updates to your application, making them essential for real-time data synchronization and instant notifications. In simpler terms, when an event happens in an external system, the Decision Engine can be instantly notified through an Inbound Webhook. 

Inbound Webhook Handling in the Decision Engine involves 2 key components:

1. Inbound Webhook Connection
   This defines how the Decision Engine webhook handler should process requests from providers, including authentication, and filtering of events. Each connection can define multiple events to be processed as separate resources, each of which can then be added to the Decision Flow as an individual Inbound Webhook Node.
2. Inbound Webhook Node
   This enables Decision Authors to define where in the Decision Flow the execution should pause and wait for the provider's data to become available. When the Decision Flow execution reaches the Inbound Webhook Node, it pauses until the Decision Engine webhook handler receives a request from the provider. Once that happens, the execution resumes from the point where it paused.

Every Decision Flow graph starts with an Input Node and ends in an Output Node. These are the only two nodes that cannot be added or deleted.

Input Node

The Input Node is where every API request and test data sample enters the Decision Flow. Its key function is to apply the Input Schema. If an Input Schema is provided, this node checks whether the data sample entering the Decision Flow has the correct format. 

During the authoring, it also controls which fields from the test data flow through the Decision Flow graph: Any fields not listed in the Input Schema are not available to the Decision Flow's nodes if an Input Schema was specified.

Output Node

The Output Node is where every request and test data sample exits the Decision Flow. It also checks whether the data adheres to the Output Schema and raises an error if this is not the case. It can be used to control which fields are returned by the flow.

Integration Nodes are custom-made nodes that make fetching data from third-party data providers easy, safe, and effective. Before an Integration Node can be added to a Decision Flow, an administrator from your organization must configure a connection to the data provider.

Configuring the connection

1. Click on the Settings icon located at the top-right corner of the Decision Flows page.
2. Navigate to the Connections tab.
3. Click the + button in the top-right corner of the page.

1. Select from the list of Integration Connections
2. Select your desired environment(s) and input your credentials.
   Many commonly used data providers in the Decision Engine support multiple environments (e.g., Production and Sandbox). Here's how environments work:
   • Sandbox and test runs: Any non-production environment you configure can be accessed during test runs and Sandbox decisions.
   • Production decisions: Only production environments are available in live, published decisions.

You can setup a connection without configuring any environments. In this case, sample reports will be available for testing without credentials. However, to go live, you must add valid credentials for a production environment.

1. Click Add resource to choose the reports you want to include.
   For each resource, you can configure the following options:
   • Include raw provider response - Adds the full response received from the third-party provider to the API response in published Decision Flows.
   • Include raw requests made to the provider API - When available from the third-party provider, this option logs and includes the original API requests sent to the provider in the published Decision Flow response. This feature is helpful for auditing and debugging purposes. However, be aware that it may expose sensitive data in the API response.

Configuring the Integration Node

After adding the Integration Node to your Decision Flow, you need to configure two key components:

The API Request

The API request contains the data you send to the external provider. When configuring the input, keep in mind that specific inputs will vary depending on the provider and resource (API endpoint). You can reference the data object using data. and enter literal strings in double quotes, similar to other input boxes you may have encountered in different nodes.

The API Response

The API Response includes the data returned by the provider, which the Decision Engine structures and ensures is free of personally identifiable information (PII). The output from all Integration Nodes will feature Insights, which represent the most commonly sought-after aspects of the API response, along with any Decision Engine calculated metrics that may be available.

Additionally, you have the option to toggle the Bundle API Response. This feature allows you to output the entire API response into the data object for use by downstream nodes. You can also easily retrieve the Provider entity ID and Provider request ID, which are identifiers provided by the external provider and included in the API response.

Advanced Settings

Target environment selection

For connections that support multiple environments, different Decision Engine requests can be mapped to use a given environment.

For test runs, use

This option allows you to choose between mock data, or any non-production environment configured for the connection during test runs.

For sandbox API calls, use

This option allows you to choose between using mock data, an empty response, or any non-production environment configured for the connection during Sandbox decisions.

For live API calls, use

All live (Production) decisions will use the configured production environment.

Make use of cached responses when published

This toggle enables pulling existing responses from cache, specifying a duration for caching API responses. This can minimize repeated queries to the external provider for fresh data, helping to cut costs and improve the node's runtime (latency).

A cached response will only be used when a node receives an identical (case sensitive) set of input fields in the request body. Changes in request headers or other parameters will not affect caching. Note that cached reports are also environment specific, meaning two identical requests to two different non-production endpoints will not use reports from cache.

Cache Validation

Caching an API response is useful if the response is complete, however there are a handful of cases where we don't want to cache the response. For example, an API requests resulting in a “no match” response, or “locked” or “frozen” credit reports (a feature of US fair credit legislation). To handle these cases, our Premium Connections have built in cache validation mechanisms, whereby invalid API responses are automatically not cached.

Use custom request timeout

Here you can specify a custom timeout behavior for a particular Integration Node. If the timeout is reached before the data integration responds, the request is aborted. Note that your chosen timeout has to be below the overall timeout for the decision. For synchronously invoked decisions, all nodes taken together have to execute in less than 30 seconds. For asynchronously invoked decisions, each Custom Connection node by itself can take up to 15 minutes (900 seconds) to complete.

Do not error with 4xx/5xx responses

These toggles allow you to override the node's native behavior whereby 4xx and 5xx responses the external provider triggers an error for the node and halts the flow.

This allows you to handle the 4xx and/or 5xx responses on your own in downstream nodes, which may allow you to successfully complete your decision despite an error from an external provider returning an error.

It is not advisable to enable this option without implementing a strategy for handling a response with a status in the 400 and/or 500-range, for instance by using a Split Node that splits based on the response range, creating a branch that tries an alternate strategy in case of an issue.

FAQ

The Loop Node enables you to apply the same logic to each item in a collection. This node is particularly useful if the logic is complex and involves multiple nodes, such as ML models, Data Integrations, or Manual Review steps.

The Loop Node itself should be added to the top-level / Parent Flow while the logic applied to each item is expressed in a Child Flow. This separation makes it easier to manage and maintain your workflows. As you can focus on the high-level logic in the Parent Flow and the detailed processing in the Child Flow.

Setting up a Loop Node

The Loop Node allows you to iterate over a list of items, which can be a list of dictionaries (e.g., directors, transactions) or a simpler list of names or ids. The field for choosing the collection to loop over is a Python code field, allowing for more advanced operations, such as looping over the keys in a dictionary, splitting a string and looping over the words, and so on.

The Decision Flow above loops over a list of directors for different companies and then applies the logic in the Loop Node Child Flow to perform a simple age check on each director in the company. 

You can select any Decision Flow version within your workspace as your loop logic.

Mapping inputs and outputs

Before mapping, ensure that the Input and Output Schemas of the Child Flow are properly defined. Once the schemas are set, you can map data from the current Decision Flow to the corresponding inputs and outputs in the Child Flow.

Mapping can be performed either manually or automatically by clicking on the Auto-map fields. Auto-mapping leverages the following information:

1. The Decision Flow's Schema
2. Available types and variables in the data object, meta and params
3. Previously defined mappings (e.g., if data.user.age has been mapped to age and data.user.name to name, the LLM can infer that data.user.email should be mapped to email)
4. Typing information (e.g., recognizing that data.name is a string and data.age is an integer)

In addition to the usual data and params objects, you have access to item and index, where item represents a single element in the list you loop over, and index indicates the numerical position of this element, starting with 0.

The Child Flow outputs are aggregated into a dictionary and appended to a list on the data object.

Advanced settings

Optionally, you can set a break condition to stop the loop once a condition is met, similar to the break statement in Python. The break condition is validated at the end of each loop iteration, after the Child Flow's results have been mapped back to the data object.

Testing and debugging

When executing a Loop Node, the Child Flow is called n times, where n is the number of items in the original collection being looped over - or until a failure occurs of the break condition is met.

The results returned by the Child Flow are aggregated into a single list of objects. For example, in the image above, the Child Flow runs three times on the first row of our sample data and returns four fields for each iteration: decision, last_name, first_name, and rejection_reason. The final output is aggregated in a list called kyc_results that contains three objects, one for each iteration of the Child Flow and each object in the list has the four fields mentioned.

Sometimes, you may want to understand in more detail how each iteration is processed in the Child Flow. To do so, test-run the Parent Flow first. Then, click Create dataset on the Inspect tab to create a dataset on the Child Flow and continue developing there.

Intermediate results

Loop Nodes have a special iteration panel that allows you to inspect the inputs and outputs and outputs for each iteration. This can be accessed in two ways:

After a test run is completed, the Inspect data panel shows the final state of the data object, reflecting the cumulative changes made across all iterations. Additionally, it displays the full list of items processed by the loop, as well as the outputs from each iteration's Child Flow.

If an error occurs during any of the loop iterations, the entire flow execution will fail. In such cases, the error message on the Loop Node will indicate which specific iteration encountered the issue.

Loop Node limitations

To ensure your workflows function as expected, it's important to be aware of its limitation. Here are the main restrictions to keep in mind:

1. No Nested Loops: The Decision Flow called within the Loop Node (the Child Flow) cannot contain another Loop Node. This means you can only have one level of looping in your workflow.
2. No Embedded Decision Flows: Similarly, the Child Flow cannot contain any Decision Flow nodes.
3. Child Flow Call Limit: You have a limit of 20 Child Flow calls in a single Decision Flow. This includes both Loop Nodes and Decision Flow nodes. So, you can have a combination of these nodes, as long as the total doesn't exceed 20.

data.transaction_amounts = {
    "transaction_00001": 10000,
    "transaction_00002": 15000,
    "transaction_00003": 2000,
}

# ids, only
list(data.transaction_amounts.keys())

# amounts, only
list(data.transaction_amounts.values())

data.transaction_amounts = {
    "transaction_00001": 10000,
    "transaction_00002": 15000,
    "transaction_00003": 2000,
}

# list of (key, value) tuples, e.g.
# ("transaction_00001", 10000)
# ("transaction_00002", 15000)
list(data.transaction_amounts.items())

transactions = [
    {'id': 1, 'amount': 10.0},
    {'id': 2, 'amount': 15.0},
    {'id': 3, 'amount': 20.0}
]
[t['amount'] for t in data.transactions]

transactions = [
    {'id': 1, 'amount': 10.0},
    {'id': 2, 'amount': 15.0},
    {'id': 3, 'amount': 20.0}
]
sum([t['amount'] for t in data.transactions])

companies = ["companyA", "companyB", "companyC", "companyD", "companyE"]

if "companyC" in companies:
    ...

Many decisions cannot be fully automated, at times requiring the input from a human to reach a conclusion. With the Manual Review node a Decision Author can choose a point in their Decision Flow where the Flow will pause, create a detailed “Manual Review Case”, and await a decision from a human operator.

The Decision Engine's Machine Learning Node allows you to add machine learning models directly to your Decision Flow. It supports any model framework that can be converted to the ONNX format, such as SciKit Learn, XGBoost, LightGBM, Catboost, pytorch, and many more.

Under the hood, the ML Node builds on ONNX, an open format for the representation of machine learning models. ONNX is backed by major players in the ML space and designed for long-term storage of models with strong backwards compatibility guarantees. It is optimized for inference speed and excels when making predictions on individual samples.

Model requirements & converting to ONNX

The Machine Learning Node is compatible with all ONNX models that are supported by ONNX Runtime version 1.16, ONNX version 1.14.1, and ONNX Opset version 19.

Rather than train ONNX models directly, the typical workflow is to create a model using the framework of your choice and converter library to save it in the ONNX format. Below is a table of resources for learning how to convert models created in different frameworks.

Model upload

Once you have added an ML Node to your graph, you can upload your onnx file. Note that the model file size has to be smaller than 5mb and that execution is limited to a single CPU core.

Once uploaded, the node's logic pane will show the following information:

1. Model metadata
2. Model inputs
3. Model outputs
4. Auto-generated Code

Model metadata

This is where metadata attached to the onnx model is shown. Below is a table of the model properties that are recognized, the equivalent property on an InferenceSession called inference_session, and how to set the property on a converted onnx model named onx.

Inputs, outputs & types

You can examine all named node inputs and outputs in an ONNX model, including their expected types and shapes. For detailed information about ONNX types, refer to the official ONNX specification. All inputs in ONNX models are tensors with a flexible first dimension. This first dimension corresponds to the number of individual predictions you want to make, allowing for batch processing.

For instance, a model that takes 12 features as input in one tensor, will have an input dimension of [None, 12]. However, a converted sklearn model with 12 separate named inputs (i.e with a Column transformer), you will have 12 individual inputs each with a dimension of [None, 1]. 

The python ONNX runtime which the Decision Engine uses for inference expects numpy input and there is a direct mapping of ONNX types to numpy types which you can see in the following table:

Code generation and inference

Under the hood, the ML Node functions a bit like a Code Node. Just like with regular Code Nodes, you can write arbitrary python code and import any libraries available in the execution time. In addition, your code has access to an InferenceSession object named inference_session which can be used for model inference. The model behind the inference session is loaded at startup time and available to all future request for rapid inference. 

Upon model upload, the Decision Engine will also auto-generate the code required to call the model. This code is generated to do the following tasks:

• Map each input of the model to a top-level field in the data with the same name. For example, a feature named income on the model will be set to data["income"].
• Assign each model output to a field on data with the same name.
• Parse the required data type and generate the code required to convert python types to numpy / onnx and vice versa.

When swapping out the model for a new version, you can choose to either re-generate the code or keep your current code in place.

Code generation is often a good starting point for how to work with onnx models. It works out of the box only if all inputs required by the model are already present in the data and have the same name. Otherwise, adjustments will be required.

The advantage of this approach is that you have full access to the inference_session object and can use it to programmatically extract metadata, make multiple predictions, run XAI, and much more. You should also be able to easily simulate inference outside of the Decision Engine, for example in a jupyter notebook. Run the following code to create your own inference_session:

import onnxruntime as ort

# from a file
inference_session = ort.InferenceSession("path_to_model_file")

# or directly from the model object
inference_session = ort.InferenceSession(onx.SerializeToSTring())

Further reading

Below is a collection of other interesting ONNX-based resources.

Rule Nodes can be used to assign one or more values based on an IF . . . ELSE IF . . . ELSE . . . condition. They are conceptually very similar to split nodes and can often replace a split node plus multiple Code Nodes / Assignment Nodes more concisely.

Define a Rule Node

Steps

1. Choose one or multiple output fields that should be set. These can be new fields to be added to data or existing ones which should be updated.
2. Define one or more conditions, and assign output field values when the condition is met.

Rule node syntax

All fields in Rule Nodes - including output fields - are fully evaluated Python fields. This allows for some powerful expressions:

Setting a nested field

Setting an output value to a combination of other fields

Appending to a list

FAQ

IF data.age is not None OR data.variable < 18 THEN "Reject"

The Scorecard Node allows you to compute a score by calculating the weighted sum of various sub-scores or factors. Each sub-score is derived from a series of conditions and is assigned a weight between 0% and 100%.

Setting up a Scorecard Node

1. Begin by specifying the field where the total score will be assigned.
2. Click + Add factor to include the fields that will contribute to the score computation. For each factor, you can set up multiple cases by clicking on + Add case.
3. Assign weights to each factor, ensuring that the total weight sums up to 100%. Alternatively, you can click on Assign weight equally across factors to weigh all factors equally.
4. Optionally, you can extract individual factor sub-scores by clicking on Also extract individual factor scores. These sub-scores are unweighted and independent of other factors.

Understanding factors

Most factors in the Scorecard Node consist of four components:

• Factor: The field being evaluated.
• Weight: The importance assigned to the factor. This must add up to 100%.
• Operator: The condition applied (e.g., <, ≤, >, ≥).
• Value: The value of the factor that should be evaluated.

Some operators, like is any or is None, do not require a value. The outcome of these factors is always a numeric sub-score.

Scorecard conditions are evaluated using a “first hit” policy, similar to the Rule Node or Decision Table Node. This means conditions are assessed from top to bottom, and the first true condition is selected. To ensure conditions are mutually exclusive, organize your scorecard in either ascending or descending order and use the appropriate operators, such as < or ≤ for ascending or > or ≥ for descending.

When creating scorecards, a common practice is to categorize a factor into various ranges and assign a specific score to each range. However, there are situations where you might prefer to use linear interpolation over a range of scores. To do this, you can substitute the fixed entry in the Score field with a formula. For instance, in the example above, loan amounts that exceed the maximum or fall below the minimum are given a score of 0, while amounts within that range are assigned a score that is linearly interpolated between 0 and 10.

Debugging your Scorecard

In case of an error, cells with issues will be highlighted. Hover over the error icon to see details.

Split Nodes are used to direct a request to one of multiple branches. This way, some requests can be processed differently from others.

For example, you have one branch for new customers and one for repeat customers as these are usually treated very differently as less data is available for new customers.

Each branch of the Split Node has a corresponding condition with the first condition corresponding to the left-most branch, the second condition to the next branch to the right, and so forth. The last branch is the default branch, i.e. if none of the conditions are met, the data sample is routed via the default branch.

To decide which branch a given request should be routed to, the conditions are evaluated from top to bottom and the branch corresponding to the first matching condition is chosen.

Split Node Condition Syntax

All cells in a split node condition are fully evaluated Python Expressions. This allows for complex conditions and formulas, e.g.

1. using Python formulas
2. comparing two fields in data
3. using built-in Python string manipulation

Furthermore, it is possible to chain multiple expressions from the same branch using the logical operators and, or. Note that it is not possible to combine both and and or for the same branch as this can lead to an ambiguous condition.

FAQ