CRM

Multithreading in CRM

For anyone doing batch processing in Microsoft Dynamics CRM; you’ve most likely experienced how painfully slow it is compared to custom-built solutions. At least that’s how I approached it when I first started working with CRM 2011. Always being performance centric I was curious to see exactly how fast I could get it to run. Said and done, I put together a few samples.

But first a few words…

There are several issues one needs to be aware of. For one, every object written to the database needs to be serialized to SOAP and then deserialized, before being stored in the database. That means the following steps has to happen under the hood:

  1. Establish http connection
  2. Serialize object to SOAP
  3. Send the object(s) over the wire
  4. Deserialize object to SOAP
  5. Establish database connection
  6. Store data in database.

As you can see, there are many steps, which means many opportunities for optimization.

First attempt

This simply uses the ExecuteMultipleRequest, which is a built-in feature. You simply write many objects to the database at once. Done deal, here’s some sample code:

using (XrmServiceContext context = new XrmServiceContext(new OrganizationService(new CrmConnection("connName"))))
{
    const int maxItems = 1000;
    for (int i = 0; i < maxItems; i++)
    {
        Contact contact = new Contact();
        contact.FirstName = "FirstName " + startIndex++;
        request.Requests.Add(new CreateRequest { Target = contact });
    }

    var response = context.Execute(request);
}

Second attempt

According to the CRM documentation you can have multiple threads running at once. I figured I’d give it a try. But, in order to not make the http connection back to the CRM server act as a bottle neck I figured I should create a unique context per thread. That way I can parallelize all the way back to the CRM server. Note the use of the ServiceConfigurationInstanceMode property on the context. Here’s the sample code:

const int maxItemsPerBatch = 1000;
const int maxBatches = 100;

Stopwatch sw = new Stopwatch();
sw.Start();

List<System.Threading.Tasks.Task> tasks = new List<System.Threading.Tasks.Task>();
for (int j = 0; j < maxBatches; j++)
{
    var t1 = System.Threading.Tasks.Task.Factory.StartNew(() =>
    {
        using (XrmServiceContext context = new XrmServiceContext(new OrganizationService(new CrmConnection("connName"))))
        {
            Timeout = TimeSpan.FromMinutes(5),
            ServiceConfigurationInstanceMode = ServiceConfigurationInstanceMode.PerRequest
        })))
        {
            ExecuteMultipleRequest request = new ExecuteMultipleRequest
            {
                Settings = new ExecuteMultipleSettings
                {
                    ContinueOnError = false,
                    ReturnResponses = false
                },
                Requests = new OrganizationRequestCollection()
            };

            for (int i = 0; i < maxItemsPerBatch; i++)
            {
                Contact contact = new Contact();
                contact.FirstName = "FirstName " + i++;
                request.Requests.Add(new CreateRequest { Target = contact });
            }
            var response = context.Execute(request);
            response.LogErrorsIfRequestFailed();
            }
    });
    tasks.Add(t1);
}
System.Threading.Tasks.Task.WhenAll(tasks).Wait();

Some code pointers:

  • Line 10: create a bunch of threads in a for-loop
  • Line 12: create a CRM context per thread
  • Line 18: create an ExecuteMultipleRequest for use by one thread
  • Line 28: Add items to the ExecuteMultipleRequest
  • Line 34: Insert all items in the ExecuteMultipleRequest
  • Line 38: Add the task to a list
  • Line 40: wait for all tasks to finish.

The sample code above uses:

  • The ExecuteMultipleRequest inside each thread loop
  • Multiple threads
  • One CRM context per thread

By utilizing all three concepts above I have managed to do 100 000 inserts in just under 7 minutes.


Posted

in

,

by

Tags:

Comments

Leave a Reply