API – Transaction Warnings Overview

Introduction

The purpose of transaction warnings is to help users avoid creating duplicate data within the system. If the data you submit through the API looks like it might be redundant, LoanPro will return a transaction warning.

Since loans and customers are completely separate in LoanPro, different checks are made within the system for each. When a new customer is submitted, LoanPro checks existing customers to see if matches exist in fields such as the following:

  • Social Security/Social Insurance Number
  • Email Address
  • Phone Number
  • Primary Address
  • Mailing Address

When a new loan is submitted to the API, the system will check the display ID (the loan ID designated by your company) to see if it already exists in your account. There is also a warning for collateral if a duplicate VIN is submitted.

1. Catch the Warning

To catch a warning, you need to wait for a response. This is important—you may assume your data is missing, when in reality, a data submission has likely failed due to a failed response. Unfortunately, this happens on occasion, and we understand that waiting around for a response can be frustrating. Forgetting to wait for a response is sort of like ordering food at a window and then immediately heading home hungry: it wouldn't make much sense to leave before receiving your order just as it won't be very helpful to submit data through the API without waiting for a response.

Once you receive a response, the next step is knowing how to tell if the response is a transaction warning. Before we discuss that, it's important that our response was a success; a transaction warning means very little if the request was unsuccessful.

There are two requirements for any response to be considered a success. The two requirements are as follows:

  1. The HTTP response code is 200
  2. A “d” object exists in the root

Here's an example of the "d" object:

{
"d": {
// All data goes here
}
}

If the HTTP response code is not 200 or there is not a “d” object in the root, then it is not a successful response. Only if both requirements are met do we consider it a success.

2. Detecting a Transaction Warning

Once we know what a success looks like, it’s easy to know what a failure looks like—anything that falls outside the requirements of a success. So how do we distinguish a transaction warning? Transaction warnings will have a data object with a nested error object, and in that error object will be a type field with the value "TransactionWarningException". An example of this is shown below:

Transaction Warnings Response:

{
"data": {
"error": {
"message": "Transaction warnings found.",
"type": "TransactionWarningException", // type field
"code": 500
},
...
},
...
}

When coding to recognize a transaction warning, first test the server response to see if it is successful. If the response is not successful, check it to see if it contains a “data” object with a nested “error” object that includes a “type” of “TransactionWarningException”.

Here is a pseudo-code example for recognizing a transaction warning:

response = sendRequestToServer()
if not_successful(response) then:
// problem with request
if exists_in(response, "data") and exists_in(response.data, "error") and
response.data.error.type equals "TransactionWarningException" then:
// Handle transaction warning
else:
// Some other error
end if
else:
// it was successful
end if

3. Handling the Warning

There are several options on how to handle transaction warnings. We will only cover a few here.

  1. Ignore Warnings

Transaction warnings are, after all, just warnings. They can be overridden simply by adding "__ignoreWarnings": true to every object in the payload (including the nested ones).

Here is an example of a payload that will ignore transaction warnings:

{
"displayId": "Hi",
"LoanSetup": {
"loanClass": "loan.class.carLoan",
"loanType": "loan.type.installment",
"__ignoreWarnings": true
},
"__ignoreWarnings": true
}

That’s it! Your data will be saved, so you won’t lose anything. The only downside is that you may enter duplicate data. You won't receive warnings when you use this method, but you should still wait for a response since there may still be errors.

  1. Analyze Duplicates (Recommended)

This is the recommended solution, but it’s also the hardest to implement. This method requires you to look at each transaction warning individually and determine an approach depending on the information it provides.

Here's an example of a request response for a duplicate customer:

{
"data": {
"error": {
"message": "Transaction warnings found.",
"type": "TransactionWarningException",
"code": 500
},
"warnings": [
"Email jd@none.com already exists in the system (Customer ID 1: John Doe).",
"Email address jd@none.com may not be a valid email account.",
"Ssn 012345678 already exists in the system (Customer ID 1: John Doe).",
"(Phones) Phone '2524732111' already found in the system (Customer ID 1: John Doe).",
"(PrimaryAddress) Address '123 OAK LANE, SCHENECTADY, NY, 123456789, USA' already found in the system (Customer ID 1: John Doe).",
"(MailAddress) Address '123 OAK LANE, SCHENECTADY, NY, 123456789, USA' already found in the system (Customer ID 1: John Doe)."
]
},
...
}

And here's an example of what a warning response for a duplicate loan looks like:

 ...
"warnings": [
"A duplicate collateral with a VIN number of 'JTHGL46F975057216' was found on Loan identified by 'JK-1002541111' and created on '2017-12-07 21:08:26' UTC.",
"A duplicate loan with an ID of 'LN-5450-343' was found."
]
...

How to use regex :

regex = get_regex_to_test()
for each warnings as warning do:
if warning.matches(regex) then:
// Regex matched!
else:
// Regex didn’t match.
end if
loop
3.1. Customer Transaction Warnings

For customer transaction warnings, we’ll use the following regexes:

Regex

Description

/ID (\d+)/

This will grab the customer IDs from the warnings as capture groups. Capture groups allow you to get only the part of your string that matches the portion of the regex that is between the parentheses.

/may not be a valid email/

This checks if the problem is a possibly-invalid email address (i.e. the email address fails LoanPro’s verification).

In the case of warnings about duplicate customers, we will use capture groups to grab the IDs of the associated customers. Most often, when getting a regex match, you’ll get an array or iterable where the first element is the text that matched the full regex and the rest are matches for the capture groups. In this case, the ID of the original or duplicate customer would be the second element of the regex match. If the user decides to use the original customer instead of proceeding to create a new one, the ID can be used to reference the customer or link the customer to a loan.

We’ll talk more about handling duplication warnings because we need to understand how to link a loan and a customer to properly handle these warnings. For now, if you get a possible-duplicate warning, don’t create the customer. We’ll handle this particular case by giving the user the option to proceed with the creation of a new customer or link the existing customer to the loan.

If you get a match on the second regex (invalid email), then we know that the email may not be valid (LoanPro performs a lot of checks to make sure an email is valid). If this is the case, you can simply display the warning to the user and prompt the user to enter a valid email address.

3.2. Loan Transaction Warnings

Loan transaction warnings include a "duplicate display ID warning" and a "duplicate VIN warning" for the collateral entity. The collateral is a nested entity within a loan. Here are the regexes to use to pull the loan IDs from each type of warning:

Regex

Description

/duplicate collateral .* found on Loan identified by '([^']+)'/

This will capture the loan ID of the loan associated with the collateral entity that contains the duplicate VIN.

/duplicate loan .* ID of '([^']+)'/

This will detect a duplicate loan display ID warning and capture the loan ID.

If the first regex matches, that means you are submitting collateral with the same VIN as another piece of collateral that already exists in your account. Currently, there is not a way to link an existing collateral entity to a loan, so we will simply ignore the warning. If a warning matches the second regex, the display ID you are submitting is already being used by another loan. The best practice is to create and submit a different display ID.

As an example, suppose we get the messages:

"A duplicate collateral with a VIN number of 'JTHGL46F975057216' was found on Loan identified by 'JK-1002541111' and created on '2017-12-07 21:08:26' UTC."

"A duplicate loan with an ID of 'LN-5450-343' was found."

We can then handle these warnings as follows:

// Sample data
// In this case it's just a VIN and display id
// but a real scenario would have the entire loan info
var loan_to_create = {
Collateral: {
vin: 'JTHGL46F975057216'
},
displayId: 'LN-5450-343'
}

// get the sample warnings
// In a real scenario, this would be done in an error handler
var warnings = get_warnings();

// A list of objects describing how to handle loan warnings
var loan_warning_handlers = [
{
regex: /duplicate collateral .* found on Loan identified by '([^']+)'/,
resolver: fix_collateral
},
{
regex: / duplicate loan.* ID of '([^']+)'/,
resolver: new_display_id
}
]

// Handle warnings
handle_warnings(warnings, loan_to_create, loan_warning_handlers)
.then(fixed_loan => {
// fixed_loan holds the correct Collateral info
// here is where you'd resubmit your loan
console.log(fixed_loan);
})

function handle_warnings(warnings, entity_to_fix, warning_handlers) {
// for each warning
var promises = warnings.map(warning => {
// test the warning against each regex
return Promise.all(warning_handlers.map(function (handler) {
var matches;
if ((matches = warning.match(handler.regex))) {
return handler.resolver(matches, entity_to_fix)
}
return Promise.resolve(null)
})).then(values => {
return values.filter(v => v);
}).then(values => {
// one match means return the result (will be merged into the entity object)
if(values.length === 1)
return values[0]
// no match means return an empty object (won't change the result)
else if (values.length === 0)
return {}

// If we get multiple matches that's a problem
// The regexes should be designed to have at most one match for any given warning
throw new Error("Too many regex matches")
})
})

return Promise.all(promises).then(values => {
// Merge all values into one object (allows for each handler to update a subset of the entity)
return values.reduce(((a, c) => Object.assign(a, c)), entity_to_fix)
})
}

function fix_collateral(regex_matches, loan_to_fix) {
// simply ignore warnings
return Promise.resolve({
"__ignoreWarnings": true,
Collateral: Object.assign(loan_to_fix.Collateral, {"__ignoreWarnings": true})
})
}

function new_display_id(){
// You'll want to replace this with a way to generate a unique id
return Promise.resolve({displayId:'Some-new-id'})
}

// This would get the warnings from the returned message in your code
// It only returns the sample warning for purposes of this demonstration

function get_warnings() {
return [
"A duplicate collateral with a VIN number of 'JTHGL46F975057216' was found on Loan identified by 'JK-1002541111' and created on '2017-12-07 21:08:26' UTC.",
"A duplicate loan with an ID of 'LN-5450-343' was found."
];
}

The example above handles the transaction warnings for creating a loan. It does not generate a new display ID. We do this to ensure users retain control over how display IDs are handled. It is best practice is to determine a standardized way to create display IDs that conform with your company’s established convention.

4. Avoiding Duplicates

Arguably, the best method for dealing with Transaction Warnings is to implement measures to avoid ever submitting duplicate data. While there are more techniques available, we will discuss three techniques for avoiding duplicates here.

Before we get started, it's important to remember that using any cloud-based API or database is rooted in network communications. In other words, it takes time for messages and requests to travel through cyberspace. When you get a response from a server, the server state could be different when you receive the response than when it was sent.

This happens most often when the server is serving multiple clients at a time. As soon as the server finishes one request it starts on another request. Consequently, by the time you’re ready to send another request, the server’s state is already different. Any response you get back reflects the server’s state at the time the response was sent, not the time you got the response. It's sort of like sending a letter through the mail—your letter will have the date that you sent the letter, not the date it arrived at its destination.

4.1. Query the Server

This is a common technique; however, for the reasons we briefly mentioned above, requesting more from the server may not always be the best option available.

The idea here is to query the LoanPro API to see if an entity already exists (for example, if there’s a loan with a matching display ID or a customer with a matching address). If no such entity exists, then you can create the entity. Otherwise, you handle the situation like you would if there were a transaction warning.

We won't go into too much detail on how to do this as it's not our recommended method.

4.2. Keep an In-Memory Cache

This is slightly more complicated but also more effective. Here, the intention is to keep an in-memory cache of the hashed version of every piece of data you process that could throw a transaction warning. Essentially, whenever you create a loan, you cache a hash of the VIN and Display ID used. And whenever you create a customer, cache a hash of their email, address, SSN, and phone number.

Please note that it is crucial to cache only a cryptographic hash of the SSN and other sensitive data. A person’s identity can be stolen if this information is made easily available. A cryptographic hash (such as SHA-2 or BCrypt) will make it harder for attackers to receive that information.

There are a few downsides to this approach. One is that in-memory caches take up RAM, and potentially, a lot of it. The data is also not persisted, which means that if the machine or caching program restarts, the cache is lost. Another downside is that unless you use a distributed cache like Redis, the cache is only available on one machine. If you do choose to use a distributed cache, be advised that it can take a while to propagate changes.

4.3. Keep a Database

This technique is very similar to the in-memory cache method we listed above, but it requires you to use a dedicated database instead. While the difference seems subtle, it has a few advantages. The first advantage is that it uses the cheaper option of hard drive space instead of RAM. Databases also persist their data, so you can restart the database without losing data.

However, you still need to cache a cryptographic hash of the data. This is more important with a database since all data is persisted, and hackers can potentially access more data with more time to access it.


How did we do?


Powered by HelpDocs (opens in a new tab)