Integration approaches and methods

Integration between enterprise systems and applications can take many forms, with various degrees of complexity. When approaching a potential integration during the design process, it is important to consider what types of integration may be possible. For example, an external system may include a REST API, may submit data to a database, and provide a Python-based SDK for querying its API. These details provide different approaches to integration which may be better suited to different workflows or requirements for your own system. This section describes integration guidance in two main approaches:

• The high-level way you achieve the integration
• The methods you use, including the technical components that support the integration

Integration approaches

There are several typical approaches to integration that can guide further design decisions, as described in the following sections.

Bring data and capabilities into ArcGIS

This approach involves querying data from another system, database, or API to display it alongside ArcGIS-hosted data, usually in a map or tabular interface. Data may also be combined with spatial data from ArcGIS to support new visualizations or reporting that can only occur when the data is combined. This approach might leverage OGC-based services like WFS or WMS, or other standardized geospatial data formats that can be used to integrate but may be successful using simple data formats like a web-enabled CSV endpoint, which can be added to a web map in ArcGIS.

Examples of integrations using this approach include:

• An ArcGIS Maps SDK for JavaScript application that queries an API from an asset management system to show work orders and status for facilities which have locations stored in the enterprise GIS system
• ArcGIS Pro connecting to an OGC service that displays boundaries to overlay and enrich an existing map
• A query layer connection to an externally managed data warehouse, published as a map service, which summarizes website activity by zip code to display on a map interface in ArcGIS Enterprise

Provide data and capabilities to other systems

In this approach, other systems that include server software, applications, or data storage, can query, and interact with ArcGIS through the ArcGIS REST APIs and features of both ArcGIS Online and ArcGIS Enterprise. This might include querying data from feature layers, displaying imagery from image services, or submitting jobs to geoprocessing tools to run an analytic or process. Many examples of the location services system are built for this purpose, where services primarily support other applications, including non-ArcGIS systems.

Examples of integrations using this approach include:

• A CRM customer entry interface that calls an ArcGIS geocoding service to provide coordinates for user-entered addresses
• An ArcGIS routing service that is used inside a larger parcel tracking and delivery management system
• An ArcGIS-designed and hosted basemap, provided as vector tiles, used across an organization’s mapping applications through a variety of web mapping SDKs or tools

Integrate through workflows

To integrate based on a workflow or series or steps generally involves reliance on actions being taken in one system, then moving the user, data, or workflow to another system to complete the workflow. This approach can be the “lightest” integration approach in that usually neither system is customized to support the integration, but rather some orchestration or automation between the systems keeps things in sync or moves workflow steps between systems.

Examples of integrations using this approach include:

• Field data collection in ArcGIS Survey123 or ArcGIS Field Maps that triggers an asset management system job through Microsoft Power Automate when a certain record type is submitted.
• An editing workflow which initiates a new process through a database trigger when edits are made through an ArcGIS feature service.
• An imagery request process that allows users to initiate an imagery request, identify providers that can capture imagery, and procure the data, launching acquisition tasks in the individual providers’ systems when run.

Create focused business apps

To integrate ArcGIS capabilities into a specific application, which also references other business systems, use the ArcGIS REST APIs and SDKs to access dynamic geospatial content, tools and functionality. This approach focuses on integrating ArcGIS services and capabilities with other services, endpoints or tools through one or more custom applications.

Publish to shared data stores

Some organizations use a shared data store, such as a data warehouse or data lake, as a common location to bring data together from different enterprise systems. ArcGIS systems can contribute to these shared data stores, including foundational geospatial layers that are then used for analysis, combined with other data sources, and referenced across many different enterprise applications. ArcGIS also supports reading and analyzing data from a variety of shared data store providers.

Integrate security systems or identity

ArcGIS integrates with a variety of 3rd party identity systems, providers or patterns, including SAML, OpenID Connect, LDAP and Active Directory. These patterns are described in more detail in the Authentication models and providers topic in the Security pillar. In addition, ArcGIS Enterprise deployments in Azure or AWS can natively integrate with security models including AWS Identity and Access Management (IAM) roles and Azure Managed Identities.

Integration interfaces or methods

The technical methods or interfaces used for migrations are usually situation-dependent and may depend on which apps or tools are already deployed. During a design process, these are the technical components that should be considered and compared with each other to identify what the best method or interface is for integrating and achieving the desired experience.

Application tier integration

Application or presentation-tier integration focuses on bringing data or services into a specific user interface or experience. This is often the shallowest level of integration, but can also be the most efficient, effective, or inexpensive, as it focuses on making data or services available specifically in one application or set of interfaces. This may require customization, or build on a custom interface, but can also be supported in off-the-shelf applications or configurations of ArcGIS and other systems. Examples of presentation-tier integration include:

• Embedding applications through <iframe> or <embed> tags, so they appear within a larger app or experience. This is commonly used with ArcGIS Hub and ArcGIS Enterprise Sites to embed other ArcGIS applications or external interfaces. With this method, communication between the ‘parent’ and embedded application is usually limited.
• Apps can be linked together through popups, attribute-based links, and related records to allow users to jump between existing applications or systems to access the requested data or document.
• Custom apps build with the ArcGIS Maps SDK for JavaScript can make API calls or requests to remote APIs, manipulate the results, and then display them in an interface as map-based or tabular information. For example, an app could call a remote API to return customer order information from a CRM, and then display it on a map for analysis of order density.
• Other apps and interfaces can also be customized to make requests to ArcGIS REST endpoints, such as a healthcare management system using ArcGIS-based routing service requests to efficiently route medical staff for home visits.
• Also included in this method or interface are workflow automation platforms like Zapier, Power Automate or Make.com. These applications integrate through REST requests, where a workflow can be initiated, or can call out to external services, and tie together multiple parts of a workflow or user groups.

Services tier integration

Integrating at the service level generally integrates data through web services, which then makes the data available to a variety of both ArcGIS-based and external applications. While there are many potential examples of this method, the most relevant examples include query layers, custom data feeds and server object extensions or interceptors.

• Integrations built with the ArcGIS Enterprise SDKs include Custom data feeds are a powerful feature of ArcGIS Enterprise, where developers can create read-only feature services from virtually any data source. Examples of these data sources can be a query to an API, connecting to databases, or even files. Since the resulting services are read-only feature services integrated into ArcGIS, they can be served to web clients, desktop apps, and field apps. Source data of Custom Data Feeds can remain in the native format and is read directly through ArcGIS Enterprise without utilizing ETL workflows. Custom data feeds are useful for scenarios where ArcGIS does not natively support a specific data source. Other use cases and data source samples can be found in the documentation. Creating a custom data feed requires developer resources and expertise including:
  • A development environment with installations of the ArcGIS Enterprise SDK, NodeJS, and a JavaScript IDE to create the custom data provider package.
  • A deployment of ArcGIS GIS Server with the custom data feeds runtime installed, which will host the published feature service.
  • Further guidance on development and configuration for custom data feeds can be found in the documentation.
• Another Enterprise SDK pattern is the development of Server object extensions (SOEs) and Server object interceptors (SOIs) are customizations of individual geospatial web services in an ArcGIS Server site. Extensions generally add new functionality (creating new REST endpoints for resources or methods) and interceptors operate within existing methods such as /query or /exportImage to interact with and modify the request or response as it is processed. SOEs and SOIs can be used to integrate other data sources, such as a query to a different endpoint or to data on disk and can also be used to integrate other security providers, to apply row-level security or group-based access to layers in a service.
• The ArcGIS REST APIs can also be used for services tier integration, called from other systems or used in an enterprise service bus pattern where APIs and feeds are collected into a central catalog or system that links them together and provides discoverability for users and developers.

Data tier integration

Integration can also be accomplished at the level of data storage or persistence. This usually takes the form of data migration, extract, transfer, and load (ETL) and similar processes, which move data between systems. Some databases support connectivity to other sources (such as the foreign data wrapper in PostgreSQL, or linked databases in SQL Server), but generally data-level migration involves an automated, repeated data movement between systems. Popular patterns include:

• ArcGIS Data Pipelines and ArcGIS Data Interoperability, along with other tools in this category that can move data along with transforming or processing the data, such as changing values, enriching with geometric information, or changing format.
• Query Layers are authored and published in ArcGIS Pro, where a connection is first made to an external relational database or data warehouse, to pull in either tables or a view of data from that database. Importantly, these databases can be fully external to ArcGIS, without any enterprise geodatabase objects or configuration, and may or may not include spatial data using native spatial types from those systems. Query layers can be used to view individual rows in a transactional system, to view a summarization or analytic result, or to view a modified, simplified version of data through a view or definition of specific columns. This flexible approach is made available to users by publishing the map containing the query layer as a map image layer, which then enables any configurable ArcGIS application or ArcGIS SDK-based application to query the data through a familiar REST interface.
• Python scripting is often used, either in stand-alone scripts or Python Notebooks, to automate data movement between systems, including quality control, manipulation, geo-enrichment, and data mergers between different sources.
• Additional ETL processes are highlighted in the Data pipelines and ETLs Architecture Practices topic

All data-level integrations should consider several topics carefully during the design phases of an architecture process:

• Update frequency - if data is integrated from another system of record, understand how often the other system is updating, and how long of a delay is expected or acceptable before updated data is visible in the ArcGIS system.
• Data quality - when a system relies on external data, the quality of that data is paramount - understanding the tradeoffs and choices that led to the data source configuration can inform the use of those data in the ArcGIS system.
• Data resilience - when relying on another system, ensure that the update process provides a fallback option when the remote system is unavailable or provides incorrect or corrupted data - using a static copy of a dataset or a previous version (with appropriate disclaimers) can help to bridge a short gap or outage.

Recommended integration strategies

Some successful strategies that contribute to successful integrations during the architecture design phase include:

Take a strategic approach

Integrating enterprise systems changes the way an organization functions, providing new envelopes of time by reducing once expensive processes or tasks to repeatable, inexpensive activities.  Leveraging enterprise integration in a priority strategic context can enable an organization to realize highly valuable outcomes by integrating processes, applications, or data to enable improved coordination in the production and delivery of a portfolio of products and services.

Apply this value proposition as a guidepost to defining initial requirements, develop scope and cost estimates, and make resources commitments for your integration effort.

Integrate at workload-appropriate and data-appropriate system tiers

Enterprise integration is commonly achieved through orchestrating human and automated processes, including components embedded in the applications people use, providing access to the digital content or analytics produced by people and processes in other systems, or a combination of these approaches via application programming interfaces (APIs).  It is also common to leverage shared systems and processes for enterprise identity and security within the technical landscape of enterprise integration.

Resource enterprise integration efforts effectively

Enterprise integrations can be technically complex, often including multiple system tiers and detailed requirements for performance, security and availability. These projects often require software and systems engineering disciplines and experience that may reside outside of a traditional GIS team or project team. Allocating the right resources and team members from various parts of the organization is essential to ensuring that integration functionality performs as expected and allows users to focus on their work rather than the technology.

Ensure that data accessed across multiple systems is appropriate for use

Integration between systems often brings data together that would not otherwise intersect, which introduces potential compilation concerns related to confidentiality, appropriateness and relevance. Information resources can be interpreted differently by enterprise or external users, so it is important for development teams to have a clear understanding of the meaning and scope of use for the various forms of digital content integrated between systems. 

Misinterpretation of content types, fields, values, and so on, can lead to negative impacts that also degrade the value of enterprise integration investments. Strong data governance can contribute to this area by ensuring that developers and users understand the standards applied to datasets.

Implement appropriate network and information security

Safeguarding sensitive information resources and systems is an important requirement for every organization.  Network security ensures that appropriate personnel are authenticated and authorized to access and apply information resources. Information security measures ensure that the digital content resources are made available in an appropriate manner for every given audience.

Network and information security constraints for all forms of information may require multiple data processing stages to produce the right form of content that is appropriately suited to an intended purpose for a given audience. This can cause integration at data and application tiers to become complex and process automation is routinely required to transform or otherwise process data assets in motion from one system to another. Additional considerations related to security can be reviewed in the security architecture pillar.

Retire systems, data, and integrations that are no longer needed

All enterprise systems should operate with a clearly-define life cycle. While evolution of these systems may be slow, change is inevitable and without clear life cycle planning many organizations struggle to govern their portfolio of systems, solutions, and integrations. Integrated enterprise solutions are dependent on the stability of the data and technology landscapes, and changes to these environments can interrupt the use of applications and associated workflows, impacting organizational productivity. As systems and their digital contents change, keep track of these dependencies so that enterprise integrations may be evolved, and when no longer needed, retired.

Conclusion

While many of these concepts are relevant to any enterprise information system, the integration of enterprise geographic information systems includes additional considerations such as geospatial data correlation and support for mapping visualizations and interfaces. Enterprise integration of ArcGIS systems with other business systems enables people across an organization to work together, with improved coordination, to apply geography more fluently and effectively.