Testing Data Migration with Dynamics 365 On-Premise

I recently had a requirement to perform some Data Migration, but I needed a test system while I built up my SSIS process as I didn’t want to use the real online environments. Luckily, I have my own on-premise Dynamics 365 Version 9 Server which I have always used for testing and messing around with solutions and compatibility.

The main issue I have always had with using on-premise servers however is that these days, we are all developing with online environments, and solutions and what not from online cannot be imported into an on-premise environment. In the past, I have often just resorted to hand modifying solutions to get them in to the system. This would often involve editing the solution.xml file within the solution zip file, to alter the version tags at the top, and to remove the crmonline tag. Quite often, this would also require a few modifications to the customizations.xml file to remove any tags that do not exist in version 9 of Dynamics.

Its a hack, but for testing purposes, this was always something that worked.

These days though, you more often than not will face all manor of other issues, such as dependencies on other solutions, and its become a lot harder to achieve this.

I needed a new plan.

Here comes a solution

I thought about the issue, and came to the conclusion that all I really needed were the entities and the attributes, considering I am exploring data only. Views, Forms, Workflows and plugins are not really a priority at this stage.

So I wondered about the possibility of having some sort of utility that would clone an online Dynamics environment into an on-premise environment, but without exporting and importing solutions. Something that would literally take the “schema” of one environment, and recreate it in another. You know, a bit like what you can do with an SQL database where you can export the structure into a script that literally can rebuild it from scratch.

So I developed one.

For a while now, I have been working on a Dynamics Command Line Interface, a good old fashioned command prompt tool where you can type commands, and do things with Dynamics. Think “XrmToolbox”, but in a command prompt. This tool can do things such as :

  • Connect to environments, and maintain a list of connections
  • Export, import, pack and unpack solutions
  • Manipulate solution version numbers
  • Compare solutions between environments
  • Query the metadata of entities and attributes
  • Activate and deactivate SLA’s, workflows, and plugins
  • List, find, view, edit and add records
  • Run predefined scripts containing commands

This means that this tool lends itself as a useful utility to get quick information from Dynamics, make quick changes, backup solutions etc, but also can be used within a DevOps Pipeline as well.

For this particular post, I want to cover the additions I made to this utility to support my Data Migration problem. It now has the ability to “clone” the entity and attribute schema from one environment, to another.

It does this first by looking at all the global option sets, and making sure they all exist in the target environment, and all have the correct options and labels.

Then, one by one, it recreates the basic entities that exist in one environment, into the new environment. It does not worry about any advanced customisation’s, just the basics, so you get the entity name, the main properties of the entity, primary key, descriptions etc, but no icons or web resources. Currently it does not work with many to many connection entities, but I am sure it will soon.

Once all of the entities physically exist in the target system, it then goes through each entity, and creates all of the custom attributes, so string, money, option sets, lookups etc.

The resulting entity will be a duplicate of the original source entity bar a few specific points:

  • Relationships may not be created with the exact same name
  • Some entity level properties will not be copied, such as Access Teams
  • Web Resources and Entity Images will not be copied

The result will at least be a collection of entities and attributes that match the original environment, meaning the same data queries, inserts and updates performed on the source environment should also work on the cloned environment.

This should obviously never be used in any form of production scenario. It is purely for testing purposes as there are no concepts of solutions, publishers or managed customisation’s. Everything that is created via the clone command is un-managed, and not tied to a solution or publisher prefix (although all of the field names and option set values do match the source environment).

I will be releasing my Dynamics Command Line Interface in the coming weeks, so keep checking back on this blog for when its released.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s