A test study is a comprehensive examination and evaluation of a system’s functionality, performance, and end-user experience to determine whether it meets specified requirements and operates as expected. Test studies are usually developed using a particular reference architecture as a starting point, and then built upon it by:
The intent of providing test studies is to help organizations design and implement well-architected systems. Specifically, test studies aim to provide an example of:
Test studies do not represent the recommended system design for all possible systems of that type. Following a structured and comprehensive design process is critical, and these examples can serve as useful references to help guide that process.
Test studies are a brand-new addition to the ArcGIS Architecture Center. There are currently three available:
More to come soon!
Test studies presented in the ArcGIS Architecture Center have four primary elements:
The overview section introduces the parameters of the test study. It also outlines some of the key capabilities the system delivers, the software components included to enable those capabilities, the respective software versions that were used, and an overview of the data that were tested.
The tested workflows section outlines the specific steps each user persona conducted for each of the defined workflows in the reference architecture. Workflows in this context refer to a series of tasks that a user would perform using the system to achieve a specific business process or objective.
Developing a workflow as a detailed, stand-alone set of steps allows them to be repeated, iterated on to tweak operations, or adjusted to improve performance or user experience. Once each workflow is clearly defined, it is used consistently throughout the remainder of testing to establish the performance baseline and conduct load tests.
The physical architecture section is where you will find a diagram of the physical design. It illustrates the core software components of the system and how they were implemented within the deployment pattern used (e.g. Windows/Linux, Kubernetes). The diagram also defines the specific physical characteristics, such as machine counts, machine types, and disk size that are allocated to each system component.
Accompanying the diagram is a high-level description of the resourcing and configuration considerations applied to the design, organized around the six pillars of the ArcGIS Well-Architected Framework.
The test methods and results section explains the testing approach taken, including the test scope, workflow pacing, and testing tools used. Additionally, it includes the test results and an evaluation for multiple test scenarios, such as for:
Conducted workflow results are also shared to give context for what end-user experience is like under each load scenario and to help inform your design based on relative load each workflow places on the system.
These test study resources are an exciting addition to the ArcGIS Architecture Center. We hope they help you successfully design and implement business systems with ArcGIS. We will continue to publish more test studies, so stay tuned for new additions to the site, and let us know what you’d like to see next through the “Tell us what you think” links at the bottom of each page!