Power Platform and Dynamics 365 Integration
Author:
Paul Smithdale
Published: April 16, 2024
14 minutes to read
Businesses are constantly exploring process efficiencies, collaboration and, new and meaningful ways to connect with customers. Power Platform and Dynamics 365 provides the tools to rapidly implement new enterprise grade, business capabilities that match and surpass what bespoke applications can offer.
Your business wasn’t created yesterday. Any new system must integrate with other line of business systems. Failure to integrate will contribute to process inefficiencies, hinder collaboration and, create data inconsistencies across the business.
When embarking on a new Power Platform or Dynamics application, we need to ask the question How will data flow into and out of our new application?
Understanding your business data
Before connecting Power Platform to our business systems, we need to understand what our data looks like do decide if the tools are capable of meeting our requirements. We can provide data requirements in terms of:
- Volume: Will we have 100 records or 1 million? How many records do we expect to have in 5-10 years?
- Size: Will each record contain a few text fields (small) or, complex spatial data (large)?
- Source: Will the application create and manage the data, reference the data from another system (read-only) or, modify and contribute to another system?
- Timeliness: If the system is collaborating with another system, how quickly must data be conveyed? Monthly, daily or near real-time?
- Shape: Does the structure of data within one system match the structure in other systems? Will data need to be transformed and validated before being communicated?
Communicating with business systems
Once an understanding of business data has been established, we need to investigate the business systems involved in the communication and how they will present information to each other. The following items must be known to define communication.
- Protocol: Defines the communication technology. This may include internet (HTTP/S), file transfer (FTP), streaming (websockets), proprietary connections (TCP). It can also include manual delivery e.g. an email or physical file upload.
- Message Type: Business systems may expose data in common formats including CSV, JSON, XML. Many systems do not natively support export of data, direct access may be required via a database connection.
- Authentication: Can the business system securely identify other business systems and grant permissions to read or write data? Authentication will be defined using protocols such as OIDC, OAuth, SAML, Kerberos, Basic Auth, MTLS and API Keys. For authentication to work, both systems must support the same authentication protocol.
- Network: Systems within a private network may not be accessible from Power Platform (public network / Internet). Understanding network boundaries will help plan for appropriate networking infrastructure and security controls to enable network boundary traversal.
Integration Technology Selection
Once data and system analysis are complete, we can investigate how to implement integration with Power Platform. Power Automate is a natural first choice for developing integrations for Power Platform. It is tightly integrated into the platform and excels when responding to data and user events within an application.
As data or business system complexity increases, alternative integration technologies may be required. E.g. Power Automate is a poor choice for importing millions of rows into Dataverse as it is difficult to manage batching, upload recovery and API limits. Alternative technologies can bridge the capability gap, ensuring benefits across areas including reduced development time, performance increases, reliability and supportability.
The following technologies collaborate with Power Platform to help address varying integration business requirements:
- Power Automate: Design-led, low code and AI enhanced process automation engine, tightly integrated with Power Platform and D365.
- Logic Apps: Design-led platform for creating and running complex orchestration workflows.
- Azure Functions: Code-first, event-driven compute platform to solve complex orchestration problems.
- Service Bus: Broker messages across multiple services with a high degree of reliability.
- API Management: Securely expose services and abstract implementations of diverse APIs across the internal and external systems.
- Microsoft Fabric: All-in-one analytics solution for enterprises that covers everything from data movement to data science, Real-Time Analytics, and business intelligence including:
- Data Factory
- Synapse
- OneLake
When we take into consideration our business systems, integration technologies and new applications built in Power Platform and D365, our technology landscape may start to resemble the below diagram.
It is essential to establish required technologies before embarking on Power Platform development! An emergent Enterprise Architecture will be swayed by project pressures. Perceived shortcuts will result in poor technology choices which inevitably will result in longer implementation times and possible remediation.
Power Platform Integration Patterns
Profiles of business data and business systems will align themselves with integration patterns. Understanding these profiles and patterns will accelerate decision making processes and help ensure the right solution is selected to ensure success of the project. The following sections provide common patterns and, data and system profiles where the pattern may be applicable.
User Triggered Events
Scenario: When a User interacts with forms within Power Apps, Canvas Apps or D365 we may need to create or modify data in another table or other O365 system such as SharePoint or Office.
| Data & System Attribute | Profile |
|---|---|
| Volume | Low. Individual records created by the user. |
| Source | Power Platform. E.g. user interaction with a Power App record. |
Integration: Power Automate by default, runs under the context of the invoking User account. Power Automate can call other services on behalf of the current user. This provides the benefit of capturing the person responsible for a change in a related systems e.g. Power Automate can create a row in Dataverse on-behalf of the current user. The created by field will contain the person that invoked the flow, rather than the identity of the underlying system.
Data Migration
Scenario: D365 is replacing a legacy business system. Existing data must be migrated into the new D365 system to ensure users can continue working on current information and have access to the past 12 months of data. It is projected that 2 million rows need to be migrated into Dataverse to support migration of business processes to D365.
| Data & System Attribute | Profile |
|---|---|
| Volume | High. 2 million rows must be migrated into Dataverse tables. |
| Source | Legacy Business system. |
| Shape | Complex joins and queries are required to transform from the source system to the target Dataverse table structure and relationships. |
| Timeliness | Once off migration. Will need to be executed multiple times in non-production to support testing. |
Integration: Data Factory is optimised to work with sets of data, rather than individual records. Built-in functionality for data mapping and batch configuration optimising write performance into Dataverse.
Data Factory tooling can also orchestrate writes to multiple Dataverse tables, allowing maintenance of table relationships when inserting into a hierarchy. Highly complex scenarios may require the use of tools such as Databricks to stage and manipulate data between writes to Dataverse.
Data Analytics
Scenario: The business needs to provide a self-service reporting model in Power BI based on data from Dataverse and other line of business systems
| Data & System Attribute | Profile |
|---|---|
| Volume | Hundreds of rows daily. |
| Source | Dataverse. |
| Shape | Data sanitisation and denormalisation to analytics structure. |
| Timeliness | Once per day |
Integration: Azure Synapse Link provides native functionality to extract Dataverse changes into a Delta Lake within Azure Data Lake or Fabric OneLake. Once located in storage, analytics tools such as Azure Synapse and Databricks can format data ready for consumption by Power BI.
The emergence of Microsoft Fabric is dramatically changing the data analytics landscape. The recommended practices are expected to change as new capabilities become generally available.
Hybrid Connectivity
Scenario: Power Apps requires integration with data centre hosted critical business systems. Connectivity needs to ensure that operational, performance and security best practices are adhered to, protecting the integrity of critical business systems.
| Data & System Attribute | Profile |
|---|---|
| Volume | Thousands of rows transferred individually throughout the day. |
| Source | Bi-directional between Power Platform and data centre systems. |
| Shape | Significant transformation and business logic requirements. |
| Authentication | Kerberos and Basic Authentication. |
Integration: Power Apps can reach into private networks through the use of On-premises Data Gateway. This provides direct access to business systems however; integrations require intimate knowledge of how each system functions to read and write data (tight coupling). Tight couplings can become fragile when business systems change, causing system instability.
Abstracting business systems from consumers such as Power Platform allows the implementation of multiple controls that can make integration easier for consumers and ensure the reliability of business systems. Controls can be implemented at multiple points including:
- Custom Connectors: Data loss prevention rules can be applied to connectors only allowing traffic between trusted Business connectors.
- Application Gateway: Provides a Layer 7 Web Application Firewall (WAF) that protects applications against common exploits and vulnerabilities.
- API Management: Provides a standard API language for Power Automate to consume and send data. Abstracts direct knowledge of business systems and internal integrations. Policies on API definitions can inspect traffic and validate tokens before reaching sensitive systems. Throttling rules can reject high or abnormal loads to protect impacts against availability and, subscription keys can be used to provide authentication to external consumers.
- Logic Apps / Function Apps: Provides implementation of business logic and message transformation. Deployments can be aligned with business systems instead of Power Platform solutions keeping change close to the source.
Hybrid connectivity adds considerable complexity to a solution. The decision to implement various layers should be balanced by criticality of systems, potential for reuse, incompatibility between Power Platform and business systems technologies and the need to provide abstraction.
Complex Business Logic
Scenario: Power Platform needs to insert data into a business system. There are business rules enforced by the systems interface however, there are minimal controls to enforce or validate these rules in the system’s storage.
| Data & System Attribute | Profile |
|---|---|
| Volume | 10s to hundreds of rows transferred individually throughout the day. |
| Source | Power Platform. |
| Shape | Significant transformation and business logic requirements. |
Integration: Power Automate can support business rules through the use of Conditions and expressions within Actions. When a rule requires multiple, nested conditions or complex expressions, the Power Automate interface can be difficult to navigate for creation or support of the workflow.
In these complex instances, custom code can provide a simpler solution with improved supportability. Technologies such as Function Apps support multiple languages to provide authoring of business logic. Onus is placed on the developer to ensure code is well structure, documented and supportable.
Unit testing frameworks supports writing scenarios for the business logic to ensure against breaking changes e.g. We can assert that if given a combination of user criteria, the code will return a specific security role. Unit tests can replace the need for manual regression testing when combined with Continuous Integration (CI) tools such as GitHub or Azure DevOps which will run the tests each time the code is checked in.
Conclusion
Power Automate is a highly capable integration tool available to Power Platform and Dynamics 365 that will address many of your integration needs. As data and system requirements increase in complexity and volume, the feasibility of Power Automate decreases. Persevering with inappropriate tooling may result in a solution that is more complex than a code-based solution or introduce security, reliability and performance risks to the solution.
Before embarking on a new project, model the systems that the solution will integrate with (Conceptual Architecture). Catalogue the data and system requirements to understand limitations and, select appropriate tools and provision environments for connectivity.
Platform Engineering
It can be daunting when a solution requires building capability beyond the core features of the selected SaaS platform. A major investment is required to establish new infrastructure, development processes and alignment with existing practices including security and observability.
Insight accelerates delivery time and reduces the overheads of implementing best-practice platforms via Platform Engineering. Insight have invested in establishing Data and Integration platforms based in infrastructure as code (IaC), process automation and, aligned to Microsoft best-practices such as the Azure Well-Architected Framework.
If you are embarking on a new Power Platform or D365 solution or looking to connect your successful solutions to the rest of your business, reach out to Insight to see how we can accelerate your journey.