WS proxy retrieve filter value limitations
I have discovered that when using the WS Proxy Retrieve method in Salesforce Marketing Cloud, the Filter’s Value property has limitations. Let’s find out how we can mitigate them.
After dedicating a couple of hours to debugging, I found that half of the records are lost and never queried from the database using the WS Proxy Retrieve method. If you provide an array, as seen below, into the filter value, it will only take the first 5 records of the array, and the rest will be ignored. In my case, it was five automation CustomerKeys. However, when querying automations by statuses and providing a numeric array of statuses with a length of 10, all of them are taken into the filtering condition. So, it cannot be taking only the first five array items; there might be some character length restrictions that I have never encountered before.
What exactly is the limitation when using the WS Proxy Retrieve method in Salesforce Marketing Cloud SSJS? Let’s apply simple logic to figure out where I encounter record cuts. After several attempts, it appears that I can only retrieve a maximum of 5 automations by their customer key. Given that one customer key is 37 characters long, I printed an array of 5 customer keys using Write and Stringify to examine the total character length of such a stringified array. Lo and behold, my fancy Visual Studio for code showed me that my array gets cut after 200 characters.
How to mitagate it?
Now that I know what I’am fighting against I can apply simple fix to mitagate it. I loop my original array of 300 customer keys and split the ws proxy retrieve to only take five customers keys at the time. These I keep pushing to my final array of all automation objects.
var automations = ["91f649a9-475e-6979-8732-1b02b48ad76d","a426504b-ce0e-ce8f-135b-4867b25a3e51",...., 300 more],
filter={};
for(k = 0;k < automations.length;k += 5){
//fetch automations fith retrieve method
//filter should look like
filter = {
Property: "CustomerKey",
SimpleOperator: "IN",
Value: automations.slice(k, k + 5)
};
//retrieve using ws proxy and fetch all the data
//loop Results from ws proxy and push to your final array
finalArray.push(data.Results[i])
}You can see the example of advanced retrieves via ws proxy in the official documentation.
