Salesforce to Salesforce and Database Account Broadcast
home
Synchronizes online accounts from a Salesforce instance to many destination systems using the publish-subscribe pattern. Each time a new account or a change occurs, the integration polls for changes in the Salesforce source organization, publishes the changes to a JMS topic, and each subscriber is then responsible for updating the accounts in the target systems.
The application has two batch jobs consuming the JMS topic, one for migrating the changes to the second Salesforce org and the other for migrating changes to the database. During the Process stage, each Salesforce account is matched with an existing account in the Salesforce org B or the database by name. The last step of the Process stage groups the accounts and creates updates in Salesforce org B.
During the On Complete stage, the template logs output statistics data to the console.
Read more about the publish-subscribe pattern in this blog post.
License Agreement
This template is subject to the conditions of the MuleSoft License Agreement. Review the terms of the license before downloading and using this template. You can use this template for free with the Mule Enterprise Edition, CloudHub, or as a trial in Anypoint Studio.
Use Case
As a Salesforce admin I want to synchronize accounts from a Salesforce Org to another Salesforce Org and a Database.
This template serves as a foundation for setting an online sync of accounts from a Salesforce instance to many destination systems, using the publish-subscribe pattern. Every time there is new account or a change in an already existing account, the integration polls for changes in the Salesforce source organization, publishes the changes to a topic and each subscriber is responsible for updating the accounts in the target systems.
Requirements have been set not only to be used as example, but also to establish a starting point to adapt the integration to your requirements.
As implemented, this template leverages the Mule batch module. The batch job is divided in Process and On Complete stages. The integration can be triggered in two different ways: by a scheduler defined in the endpoints file or by an outbound notification generated by Salesforce. Either way, the application queries or receives newest Salesforce updates or creates matching a filter criteria and adds them to a JMS topic.
The application has two different batch jobs consuming this JMS topic, one for migrating the changes to the second Salesforce organization and the other one for migrating the changes to the Database. During the Process stage, each Salesforce account is matched with an existing account in the Salesforce organization B or the database by name. The last step of the Process stage groups the accounts and creates or updates them in Salesforce organization B.
Finally during the On Complete stage the template logs output statistics data in the console.
Considerations
To make this template run, there are certain preconditions that must be considered. All of them deal with the preparations in both source and destination systems, that must be made for the template to run smoothly. Failing to do so can lead to unexpected behavior of the template.
Database Considerations
This template uses date time or timestamp fields from the database to do comparisons and take further actions. While the template handles the time zone by sending all such fields in a neutral time zone, it cannot handle time offsets. (Time offsets are time differences that may surface between date time and timestamp fields from different systems due to a differences in each system's internal clock.) Take this in consideration and take the actions needed to avoid the time offset.
As a Data Destination
There are no considerations with using a database as a data destination.
Salesforce Considerations
- Where can I check that the field configuration for my Salesforce instance is the right one? See: Salesforce: Checking Field Accessibility for a Particular Field.
- How can I modify the Field Access Settings? See: Salesforce: Modifying Field Access Settings.
As a Data Source
If a user who configures the template for the source system does not have at least read only permissions for the fields that are fetched, then an InvalidFieldFault API fault displays.
java.lang.RuntimeException: [InvalidFieldFault [ApiQueryFault
[ApiFault exceptionCode='INVALID_FIELD'
exceptionMessage='Account.Phone, Account.Rating, Account.RecordTypeId,
Account.ShippingCity
^
ERROR at Row:1:Column:486
No such column 'RecordTypeId' on entity 'Account'. If you are attempting to
use a custom field, be sure to append the '__c' after the custom field name.
Reference your WSDL or the describe call for the appropriate names.'
]
row='1'
column='486'
]
]As a Data Destination
There are no considerations with using Salesforce as a data destination.
Run it!
Simple steps to get this template running.
See below.
Run On Premises
In this section we detail the way you have to run your template on your computer.
After your app is all set and started, there is no need to do anything else. The application polls accounts to see if there are any newly created or updated objects and synchronizes them.
Download Anypoint Studio and the Mule Runtime
If you are new to Mule, download this software:
Note: Anypoint Studio requires JDK 8.
Import a Template into Studio
In Studio, click the Exchange X icon in the upper left of the taskbar, log in with your Anypoint Platform credentials, search for the template, and click Open.
Run on Studio
After you import your template into Anypoint Studio, follow these steps to run it:
- Locate the properties file
mule.dev.properties, in src/main/resources. - Complete all the properties required per the examples in the "Properties to Configure" section.
- Right click the template project folder.
- Hover your mouse over
Run as. - Click
Mule Application (configure). - Inside the dialog, select Environment and set the variable
mule.envto the valuedev. - Click
Run.
Run on Mule Standalone
Update the properties in one of the property files, for example in mule.prod.properties, and run your app with a corresponding environment variable. In this example, use mule.env=prod.
Run on CloudHub
While creating your application on CloudHub (or you can do it later as a next step), you need to go to Deployment > Advanced to set all environment variables detailed in "Properties to Configure" as well as the mule.env.
Once your app is all set and started, there is no need to do anything else. Every time an account is created or modified, it will be automatically synchronized to Salesforce Org B and the Database as long as it matches the filtering criteria.
Deploy a Template in CloudHub
In Studio, right click your project name in Package Explorer and select Anypoint Platform > Deploy on CloudHub.
Properties to Configure
To use this template, configure properties such as credentials, configurations, etc.) in the properties file or in CloudHub from Runtime Manager > Manage Application > Properties. The sections that follow list example values.
Application Configuration
HTTP Connector Configuration
- http.port
9090
Batch Aggregator Configuration
- page.size
200
Scheduler Configuration
- scheduler.frequency
10000 - scheduler.startDelay
100
Watermarking Default Last Query Timestamp Such As 2019-12-13T03:00:59Z
- watermark.default.expression
YESTERDAY
Trigger policy(push, poll)
- trigger.policy
push|poll
Note: The property trigger.policy can assume only one of the values above as either push or poll.
JMS Connector Configuration
- jms.username
admin - jms.password
admin - jms.url
tcp://localhost:61616
Database Connector Configuration
- db.host
localhost - db.port
3306 - db.user
user-name1 - db.password
user-password1 - db.databasename
dbname1
If it is required to connect to a different database, provide the JAR for the library and change the value of that field in the connector.
Salesforce Connector Configuration for Company A
- sfdc.a.username
bob.dylan@orga - sfdc.a.password
DylanPassword123 - sfdc.a.securityToken
avsfwCUl7apQs56Xq2AKi3X
Salesforce Connector Configuration for Company B
- sfdc.b.username
joan.baez@orgb - sfdc.b.password
JoanBaez456 - sfdc.b.securityToken
ces56arl7apQs56XTddf34X
API Calls
Salesforce imposes limits on the number of API calls that can be made. Therefore calculating this amount may be an important factor to consider. The template calls to the API can be calculated using the formula:
- 1 + X + X / ${page.size} -- Where X is the number of accounts to synchronize on each run.
- Divide by ${page.size} because by default, accounts are gathered in groups of ${page.size} for each upsert API call in the commit step. Also consider that this calls are executed repeatedly every polling cycle.
For instance if 10 records are fetched from origin instance, then 12 API calls are made (1 + 10 + 1).
Customize It!
This brief guide provides a high level understanding of how this template is built and how you can change it according to your needs. As Mule applications are based on XML files, this page describes the XML files used with this template. More files are available such as test classes and Mule application files, but to keep it simple, we focus on these XML files:
- config.xml
- businessLogic.xml
- endpoints.xml
- errorHandling.xml
config.xml
This file provides the configuration for connectors and configuration properties. Only change this file to make core changes to the connector processing logic. Otherwise, all parameters that can be modified should instead be in a properties file, which is the recommended place to make changes.
businessLogic.xml
The functional aspect of this template is implemented in this XML file, directed by two different batch jobs consuming this JMS topic, one for migrating the changes to the second Salesforce organization and the other one for migrating the changes to the database. The several message processors constitute four high level actions that fully implement the logic of this template:
- The template queries Salesforce for all the existing accounts that match the filtering criteria.
- During the Process stage, each Salesforce account is matched with an existing account in the Salesforce organization B or the database by name.
- The last step of the Process stage groups the accounts and creates or updates them in Salesforce organization B.
- Finally during the On Complete stage the template logs output statistics data to the console.
endpoints.xml
This file contains three Flows:
- push flow - Contains an HTTP endpoint that listens for Salesforce outbound messages. Each is processed and thus updates or create accounts, and then executing the batch job process.
- scheduler flow - Contains the Scheduler endpoint that periodically triggers the sfdcQuery flow and then executes the batch job process.
- sfdcQuery flow - Contains watermarking logic for querying Salesforce for updated or created accounts that meet the defined criteria in the query since the last polling. The last invocation timestamp is stored by using Object Store component and updated after each Salesforce query.
The property trigger.policy defines from which endpoint the template receives its data. When the push policy is selected, the HTTP Listener is used for the Salesforce outbound messaging and the polling mechanism is ignored. The property can only assume one of two values push or poll any other value results in the template ignoring all messages.
errorHandling.xml
This file handles how your integration reacts depending on the different exceptions. This file provides error handling that is referenced by the main flow in the business logic.