Test study design

The test environment configurations were consistent aside from differences in compute capacity of the virtual machines. These tests were planned across 4-core, 8-core, and 16-core environments. With each test, a secondary objective was to identify performance differences in Windows and Linux operating systems.

Key configurations are provided.

Test harness

This real-time performance test framework provides end-to-end validation of the capacity of ArcGIS Velocity to ingest and process real-time data at production scale. A central test orchestration service coordinates each run by configuring the simulated data sources, executing the test according to a defined profile (warmup, measurement, and cool-down phases), and aggregating results into a single pass/fail verdict by comparing expected throughput targets.

The test framework emitted features at controlled rates and volumes. Each feature emitted was a uniform 200-byte payload with point geometry. This was intentional to eliminate payload variance as a variable and isolate throughput behavior based purely on feed configuration. The 200-byte size was selected based on prior research which indicated that this is a good representative feature size used in ArcGIS Velocity with ArcGIS Online production environments. See the feature schema below.

The test framework drives traffic to ArcGIS Velocity through a suite of purpose-built data source simulators that mirror Kafka, HTTP Poller, and HTTP Receiver feeds. While the test runs, an independent monitoring service continuously observes the platform’s outputs and records the actual feature throughput observed across every monitored item. Comparing the simulated input rates against the measured output rates produces an objective measure of the platform’s reliability, accuracy, and performance.

Raw test data

The payload used across all load generation tests is a JSON record modeled after flight telemetry data. Each record is fixed at 200 bytes and contains 20 fields that include:

• Positional data (latitude, longitude, altitude)

• Flight metadata (airline, aircraft type, departure and arrival airports)

• Additional fields that could be used for filtering and analysis. Geometry was constructed from the longitude and latitude values.

The record below is shown in JSON format for readability. The actual transmitted payload is a delimited text record at a fixed size of 200 bytes.

{
  "unique_id": "A1",
  "timestamp": "1710028800",
  "longitude": "-180.0000",
  "latitude": "-90.0000",
  "pseudo_altitude": "2000",
  "pseudo_heading": "0",
  "pseudo_speed": "120",
  "pseudo_arrival_airport": "LAX",
  "pseudo_departure_airport": "YGW",
  "pseudo_airline_name": "Alpha Airlines",
  "pseudo_aircraft_type_name": "Bennie-A2A",
  "pseudo_aircraft_type": "A21NA",
  "pseudo_record_type": "Satellite",
  "pseudo_has_landed": "TRUE",
  "pseudo_in_transit": "FALSE",
  "filter_alpha_beta": "Alpha",
  "divide_int": "20000",
  "random_int": "100000012345",
  "random_string": "Esri data stream near Redlands California.",
  "filler_200": "xxxxxxxxxxxxxxxxxxxxxxxxxxx"
}