Optimize performance in ArcGIS

To deliver a similar level of performance for users who are accessing a service, it is important to understand there are different methods of optimization, depending on the geospatial service type. The options that are available may depend on what the backend data source is, or what software deployment pattern is used. Understanding different types of caching and tiling, along with other forms of optimization, is an important part of ensuring a positive user experience. This section briefly covers a variety of performance optimization techniques, across different tiers of ArcGIS architecture.

Network design for performance

The design of the network where ArcGIS components are deployed is a key contributor to performance (both good performance and poor performance). Enterprise systems are often global in nature, serving users across a wide array of network conditions and connectivity, and there are important differences between ArcGIS components and workflows in terms of their reliance on network connectivity. The breadth of network considerations for ArcGIS workflows is broader than this topic, but some initial recommendations and considerations include:

  • In general, network latency is more impactful to user experience than network bandwidth for day-to-day GIS operations. For activities like downloading data, copying files, or viewing large imagery datasets, bandwidth is also important.
  • Traffic from ArcGIS Pro directly to databases is latency-sensitive, and many small requests are made while viewing and editing data or working with and changing schema.
  • Traffic from ArcGIS Server to databases, file or object storage also requires low latency and generally high bandwidth. This allows the Server process to access the data, transform it to match user requests, and send back an optimized response to users.
  • Clients such as browsers accessing web applications are less sensitive to network constraints than desktop clients, as the Maps SDK for JavaScript optimizes requests to reduce excessive traffic.
  • Many modern networks blur the distinction between on-premises and cloud resources through network peering and other technologies, which can result in very low latency between cloud resources and on premise resources or clients. In general, it is recommended for clients, server processes, storage and databases to all be hosted in the same “network zone”. For example, hosting them all in the cloud, or all on-premises, but in some scenarios, it is acceptable for clients to be separated from servers, or servers from databases. How suitable this design may be for your system is dependent on your network design and structured testing of the implications.

Optimize web apps

Web applications are delivered from a web server, which is often either an ArcGIS Online SaaS service, a web server hosting an ArcGIS Web Adaptor, or a web server dedicated to hosting custom web applications. Methods for hosting optimized web applications are described below.

Build process

Prior to deploying an application, a build process (also including or known as minification or bundling) can be used to reduce the size and complexity of the application’s static assets, such as JavaScript, CSS, and HTML files, or remove unneeded information from these files. While there are many valid approaches to establishing a successful and suitable build process, these processes are often closely tied to the application framework used to author the web application.

HTTP response caching

Another method used to optimize web application performance is to enable HTTP response caching for the static assets of the web application. This configuration provides the end client with a response header from the web server, indicating that the static asset can be cached in the client and how long it can be cached for. This allows the client browser to avoid a full round-trip request on a reload of the application, as the contents of that file are already stored locally to the client machine and are re-used as-is until the cache expiration time is reached. HTTP response caching can be enabled selectively for certain file types, either through a web server or another reverse proxy or application acceleration software layer.

Content delivery network (CDN)

Web applications hosted in ArcGIS Online already make use of these two optimization patterns, as they are built before deployment and hosted in a SaaS environment that makes use of Content Delivery Network (CDN) caching of static app responses.

CDNs can also be used with ArcGIS Enterprise but be careful to make a distinction between static assets that are suitable to caching in a CDN, such as HTML, CSS, and JavaScript files, versus dynamic REST API responses that should not be cached to avoid unexpected user behavior. Also take particular care in considering the use of a CDN as it relates to upgrades, as the CDN cache should be forcefully invalidated when the software is upgraded or an application changes, to ensure users do not receive outdated information.

Optimize web services

There are many different types of web services that can be used in an ArcGIS environment – some of which will display or respond based on dynamic data sources. There is no one-size-fits-all approach to caching and optimization of these services, but instead a variety of patterns that should be used with consideration and planning. Common methods to optimize web services include:

  • Raster tile caches
  • Vector tile caches

Both methods take input data (from any source), which might otherwise be accessed dynamically through a map, feature, or image service, and cache it into tiles. These tiles are predefined packets of data, organized in a standard, multi-level grid. Cached tiles allow clients to request multiple pre-defined tiles at the same time for a given scale, which are “pre-rendered” in the tile form, so there is no request to the backend data, and no dynamic drawing of symbology or features.

  • Raster tile caching refers to data that is cached into raster tiles. These tiles are images that display a pre-rendered view of data, which is then drawn on the client as a set of multiple, stitched-together tiles for each display extent. Raster tiles can be created from map services, hosted feature services, or image services, with slightly different considerations for each.
  • Vector tile caching is similar in concept, but the tiles are compressed binary files (based on protocol buffers) which store features as geometry and attributes, cut into individual tiles which are then re-assembled and rendered on the client.

Related resources:

Strategies for raster tile caching

Raster tile caching is an effective method to use when delivering relatively static data that should be viewed in one primary cartographic representation. While raster tiles can be highly performant, they do not provide the user with the ability to query for attributes, identify features, or change symbols, turn off labeling layers, and so on.

While raster tiles were used for many years to delivery cartographic basemaps, these services most often use vector basemaps today for a variety of performance, map quality, and flexibility reasons. Raster tiles are still used to serve many imagery basemaps or services based on satellite or ortho-photography, as these services aim to present a single, carefully balanced layer of imagery as a background for applications or tools.

Strategies for vector tile caching

Vector tile caching is a newer, but broadly supported method for caching data, where a raster tile might be limiting or unsuitable for other reasons. Vector tiles can be faster to generate and smaller in size, which has benefits for storage of the tile set as well as in the speed at which they are transferred to a client over a network connection.

Vector tiles also allow for dynamic rendering through the use of a vector tile style, which can be configured or updated entirely on the client, such as a browser or native application, and can dynamically show or hide different layers, filter based on attributes, and display multiple different cartographic versions of the same data, all using the same single vector tile layer.

Both raster tile and vector tile caching have a wide variety of configurations, options, and choices that need to be made during the optimization of a service. Consult additional Esri documentation, resources, and conference proceedings for additional guidance in these areas.

Feature tiles

Another related optimization approach is a newer capability of ArcGIS referred to as feature tiles. Feature tiles are not pre-generated by a planned process (like vector tiles or raster tiles) but are generated by functionality-aware clients requesting standardized extents of data for standard zoom levels. Since these requests are identical between different clients, the features can be cached by an intervening CDN or proxy, and reused or re-sent to different clients. This functionality is exclusively available for use by clients that are designed to use this feature, primarily through the ArcGIS Maps SDK for JavaScript, though also through ArcGIS Pro and native applications built with the ArcGIS Maps SDKs. The use of feature tiles relies on several key assumptions:

  • Feature tiles are only used when making requests to feature layers (based on a map service or feature service) that support this capability, which varies by release. ArcGIS Online-based hosted feature services have supported this functionality for several years.
  • Usually, feature tiles are automatically requested by the clients from services that support them. While ArcGIS Online provides several layers of caching for these feature tiles, ArcGIS Enterprise-based layers have additional considerations for making best use of this feature.
  • The object store functionality of ArcGIS Data Store is designed to support the caching of feature tiles, so that users that request a feature tile that was already generated for another user request can receive that existing tile, providing a rapid response and caching the result for a certain period of time. This functionality requires enabling caching on individual layers in the hosted feature layer using a REST API request.

Hosted feature services in ArcGIS Online include an option to enable optimized layer drawing, which can generate reduced-resolution data in the database which is used at smaller or more zoomed-out scales of view. This reduced resolution data is automatically queried by the feature tile generation process when using those scales, which in turn, improves response performance.

Data sources and storage

Beyond application and service optimization, an equally important area for performance improvement is the storage and characteristics of the data behind a specific service. Most data used in web services is tabular in nature (with the exception of image services), and most often involves both:

  • Geometry – A data column with a geography or geometry data type
  • Attributes – Columns with other supported data types

Both geometry and attributes can be optimized to support improved service performance.

Optimize attributes

You can optimize the use of attributes through attribute reduction. Put simply, this entails the process of removing or hiding columns or attributes that may exist in a table but are not needed for your application or use case. This can be accomplished by removing columns when they are not needed, or un-checking them for display in the map when authored for publishing, for example, for map image layers.

Index attributes

Attribute indexing is another important consideration, primarily when columns will be used to either filter or query in an application. Indexing allows for faster, more efficient queries when the request filters are based on a known attribute or is querying for a text, numeric or date-based attribute, and adding attribute indices is an excellent way to make use of already-built optimization technology in relational databases.

Optimize geometry

Geometry columns can also be optimized in several important ways – depending on the use case and requirements for the data. One option is spatial generalization, often also referred to as simplification, which is a process of simplifying, smoothing, or reducing the level of detail of a geometric representation, such as a line or polygon, to reduce the size of the feature and the storage and processing requirements. This process can be completed on a permanent basis, if the original data is overly precise or overly detailed in terms of the vertices included, or could be completed on a read-only copy of the data that is used for visualization, while retaining full detail in the origin data.

Closely related is the idea of reducing spatial precision, which refers to the geometric or geographic data type, which usually stores a decimal value for the actual coordinates of each vertex of a feature, such as 124999.24541512. This value represents a linear unit, which varies, depending on the coordinate system of the dataset, which could be as large as decimal degrees or as small as feet or meters. In any case, ensuring that the spatial precision captured reflects the actual use case of the data can be important for both functional and performance reasons.

For data in a coordinate system with a meter-based linear unit, a geometry stored as (123456.789101112, 234567.891011121) has 9 decimal digits of precision, equivalent to nanometer-level precision. If the data is captured by a GPS unit with 1m accuracy (on average), then this precision is overstating the actual precision of the measurement by a significant factor. Reducing the spatial precision of the dataset by removing the decimal portion is not only more accurate but will reduce the size of the geometry when it is transferred in a network request by 50%, which helps with everything from serialization processes to bandwidth constraints. Using appropriate spatial precision is especially important with highly detailed polygon features, as the number of vertices can quickly reach thousands of records, and the resulting impact on service performance is significant.

Beyond attribute and geometry adjustments, other higher-level data storage approaches may help to optimize performance, including:

  • Use database views or materialized views to store a commonly accessed query at the database level, which can then be published into a map service for use in ArcGIS applications
  • Consider whether a file geodatabase may benefit from being compacted – this operation can improve performance by re-organizing the internal storage of a frequently-edited file geodatabase
  • Enterprise geodatabases include a variety of other performance considerations, some of which may be database-specific, but it is important to consider the Compress and Analyze Datasets operations when working with versioned data, as well as a wide variety of performance considerations when using a multi-editor workflow.

Mosaics and image services

Image services are usually built from mosaic datasets, which are geodatabase objects that follow many of the previous optimization suggestions above. In addition, there are several image service-specific optimization techniques to consider:

  • Mosaic settings, including the maximum number of rasters per mosaic and the maximum image size, each have an impact on service performance, by ensuring that the system is not searching for too many images to build the resulting mosaic, and that the resulting image is not so large that building it and sending it back to the client overwhelms the network access the user has to the mosaic.
  • Mosaic dataset overviews exist primarily to improve service performance at smaller scales (more zoomed out) – overviews represent pre-rendered views of a mosaic dataset, either where data does not change frequently or where the specifics of images are not needed at that small scale. Creating overviews, and updating them as needed, can improve the display performance of the image service significantly for users.
  • The format and configuration of the source images is also critical, as there are more, or less, optimized image formats that can impact performance, as well as properties like image pyramids, which can be calculated with an up-front compute cost but benefit all future users of that image.