Test results on ArcSOC configuration impacts

In addition to an assessment of database resources, the second set of tests was focused on identifying the optimal ArcSOC configuration to support an 8x design load on our system and its workflows, assuming sufficient database resources.

Test methods and results

Three tests were performed with different ratios of ArcSOCs : vCPU:

• 2:1, or two ArcSOCs per vCPU on the ArcGIS Server instances
• 3:1, or three ArcSOCs per vCPU on the ArcGIS Server instances
• 4:1, or four ArcSOCs per vCPU on the ArcGIS Server instances

We also performed this test with a 1:1 ArcSOC per vCPU ratio, which you can see in the large database instance size test as described in the Impact assessment of database resources.

We ran several load tests, systematically varying the ratio of ArcSOC instances to vCPUs to observe and measure the performance and user experience impacts. All other aspects of the system were held constant to achieve meaningful results.

Performance metrics like ArcSOC use and availability, service wait times, system resource utilization, and error rates were monitored to evaluate each configuration.

Tests were performed at 8 times (8x) the design load of the original system test study and the ArcGIS Enterprise server resources were cut in half to ensure there was enough load to impact the system. JMeter was used to simulate the user workflows and measure system performance under different loads.

Because ArcGIS is a multi-tier system, tests were conducted across client, service, and data storage tiers, as well as the underlying infrastructure itself. In this test study, JMeter was used to simulate the user workflows and measure system performance under different loads.

2:1 ArcSOCs : vCPU ratio

In this run, we configured two ArcSOCs per vCPU on the ArcGIS Server instances. In this case, 16 running ArcSOCs to 8 vCPUs. As in the previous charts, percent utilization of CPU is orange, disk is gold, and memory is purple.

In the chart below, CPU for all machines is generally less than 60%. However, you can see that memory utilization on the UN GIS Server peaks at over 80%. This is due to the additional running ArcSOCs as compared to a 1:1 ratio. Services on the UN GIS Server enable versioned database editing. While the system appears to be running smoothly, memory will need to be monitored closely to avoid problems. The concurrent request chart shows concurrent view requests (red) steadily opening and closing, while averaging 35.

The chart below shows ArcSOC usage for the Hosting Server, where 16 ArcSOCs are running (the blue line is covered by the green line), the maximum in use (busy) is 14. The UN GIS Server (not shown) had a maximum of 7 busy ArcSOCs, so it did not take advantage of the additional service instances, where 16 ArcSOCs were running, but were mostly idle. Because this server showed memory utilization in excess of 80%, reducing the service instances to min/max 8 on the UN GIS Server might relieve some of the memory pressure and make this system optimal for these workflows and loads. Any change in workflows or loads could impact the balance.

3:1 ArcSOCs to vCPU ratio

In this run, we configured three ArcSOCs per vCPU. In this case, 24 running ArcSOCs to 8 vCPU. Once again, CPU utilization (orange) is generally below 60% across all machines. Unfortunately, memory utilization (purple) on the UN GIS Server is maxing out, with the dips to 95% occurring as part of the cleanup process. Concurrent view requests (red) show they are steadily opening and closing, with an average of 36. This system appears to be handling the load, but it’s not sustainable due to a memory shortage.

Further, by looking at the ArcSOC chart below, you can see that with a 3:1 ratio the UN GIS Server has 24 ArcSOCs running (the blue line is covered by the green line). However, the maximum busy is only 18, which consume memory even when not in use. This is an example of a poor configuration. Our workload doesn’t require all of the ArcSOCs that are available. The six that are not needed (24 running minus 18 that are busy) consume resources (memory) unnecessarily. Increasing memory on the UN GIS Server may improve this situation, but it could also move the problem to the CPU or to the database. Testing and observation are required to make apt configuration and design choices to support the system.

4:1 ArcSOCs to vCPU ratio

In this run, there were four ArcSOCs configured per vCPU. In this case, 32 running ArcSOCs to 8 vCPU on the UN GIS Server. CPU utilization (orange) is growing, with two peaks above 80% on the UN GIS Server. However, the biggest issue is near 100% memory usage (blue) on that instance, where even the cleanup process is struggling to keep pace.

Concurrent view requests (red) at the bottom show they are still opening and closing steadily, with the same average of 36, like with the 3:1 ratio. This shows that the additional ArcSOCs are not providing any benefit to the system’s performance or end user experience. Rather, they are merely consuming GIS server resources.

This is further validated by the ArcSOC chart below that shows a maximum of only 16 busy ArcSOCs. We can see clearly here how adding additional service instances only consumed unnecessary server resources, without providing any performance or user experience gains. Increasing UN GIS Server CPU and memory may improve results, but this could move the problem to the database server CPU. Testing and observing is key.