Apache APISIX vs Envoy Proxy: High-Concurrency Microservice Gateway Stress Test

In the world of microservices, the API gateway is the critical choke point through which all traffic flows. It handles routing, authentication, rate limiting, observability, and more—all while maintaining sub-millisecond latency under extreme load. Two gateways dominate the cloud-native landscape: Apache APISIX, the high-performance Lua-NGINX-based gateway, and Envoy Proxy, the C++-based sidecar and gateway graduated by the CNCF. Both are production-proven and power some of the world's largest microservice deployments, but their architectural differences lead to distinct performance profiles under high-concurrency workloads. This stress test analysis compares how these two gateways perform when pushed to their limits.

Before diving into benchmarks, understanding each gateway's architecture explains why they perform differently. Envoy Proxy is written in C++ and built on an event-driven, non-blocking threading model. Each worker thread runs an event loop, handling connections and processing filters in a pipeline. Envoy's architecture is highly optimized for network I/O and uses sophisticated connection pooling, load balancing, and observability features. Its filter chain model—where each filter processes requests sequentially—is powerful but can introduce overhead as more filters are added.

Apache APISIX is built on top of NGINX and LuaJIT, combining NGINX's proven event-driven architecture with the flexibility of Lua scripting. APISIX's key innovation is its use of radix trees for route matching and its plugin architecture, which runs Lua code in lightweight coroutines. Unlike traditional NGINX modules that require recompilation, APISIX plugins are dynamically loaded and configured at runtime via etcd. This Lua-NGINX combination delivers surprising performance—LuaJIT's JIT compiler can approach C speeds for hot code paths—while maintaining exceptional flexibility.

To compare the two gateways under realistic conditions, we tested them across three dimensions: raw throughput (QPS), latency distribution, and resource efficiency. The test environment used 16-core servers with 32GB RAM and 10Gbps networking, running the latest stable versions of both gateways. We used wrk2 and k6 as load generators, sending HTTP/1.1 requests with varying concurrency levels from 100 to 10,000 concurrent connections.

The test scenarios progressed from simple to complex: a baseline proxy pass with no plugins, a realistic production configuration with JWT authentication and rate limiting, and a complex scenario with multiple plugins including request transformation, logging, and circuit breaking. This progression reveals how each gateway's performance degrades as functionality is added—a critical metric for real-world deployments.

In the baseline proxy test with no additional plugins, both gateways deliver impressive performance. Envoy edges out APISIX slightly, achieving approximately 280,000 QPS at p99 latency of 3.2ms, compared to APISIX's 250,000 QPS at 3.8ms p99. The gap is modest—roughly 12%—and both are more than fast enough for the vast majority of production workloads. Envoy's C++ core gives it a small advantage in pure forwarding speed, but the difference is negligible for most use cases.

The picture changes significantly in the production configuration test with JWT authentication and rate limiting enabled. Here, APISIX pulls ahead, delivering 180,000 QPS versus Envoy's 145,000 QPS—a 24% advantage. The p99 latency gap also widens: 6.8ms for APISIX versus 9.1ms for Envoy. Why the reversal? APISIX's Lua-based plugin architecture is optimized for exactly these kinds of operations. JWT verification and rate limiting logic run efficiently in LuaJIT, with minimal overhead. Envoy's filter chain, while powerful, adds more per-request overhead as filters are stacked, and its WASM-based extensions—while flexible—are significantly slower than native Lua execution.

In the complex multi-plugin scenario, the performance gap widens further. APISIX maintains 120,000 QPS with 12ms p99 latency, while Envoy drops to 85,000 QPS with 18ms p99 latency. This is the most revealing test result: as you add more gateway functionality—authentication, transformation, logging, circuit breaking—APISIX's performance degrades more gracefully. Its plugin architecture is designed from the ground up for rich functionality at speed, whereas Envoy's filter chain becomes increasingly expensive with each added feature.

Performance numbers alone don't tell the whole story—resource efficiency matters too. CPU efficiency is roughly comparable at baseline, with both gateways achieving similar QPS per CPU core. However, APISIX pulls ahead as plugins are added, delivering more functionality per CPU cycle. This is particularly important in cloud environments where CPU time directly translates to cost.

Memory usage tells a different story. Envoy is generally more memory-efficient, especially at high connection counts. Its C++ memory management is highly optimized, and connection metadata is stored compactly. APISIX, with its LuaJIT runtime and NGINX worker processes, tends to use 20-30% more memory under equivalent load. For most deployments, this isn't a dealbreaker—memory is relatively cheap compared to CPU—but it's a consideration for memory-constrained environments.

Connection scaling is another area where the two differ. Envoy's connection handling is exceptionally efficient, maintaining stable performance even at 50,000+ concurrent connections. APISIX also scales well, but its NGINX foundation means worker connection limits need careful tuning for very high connection counts. For typical API gateway workloads with connection pooling from upstream services, this difference is rarely a limiting factor.

Performance is important, but it's rarely the only deciding factor. Envoy shines in service mesh deployments, where its sidecar model and deep observability integrations are unmatched. Its ecosystem—backed by CNCF, Google, and a massive community—is the de facto standard for service mesh data planes. Envoy's configuration model, while complex, is extremely powerful and well-suited for infrastructure teams that need fine-grained control.

APISIX excels as a dedicated API gateway. Its dynamic configuration via etcd means routes and plugins can be changed in milliseconds without reloads—a huge operational advantage. Its plugin ecosystem is rich and growing, with over 100 plugins covering authentication, security, observability, and serverless integration. For teams that want a full-featured, high-performance API gateway that's easy to operate, APISIX is often the better fit.

So which gateway wins the high-concurrency stress test? The answer depends on your scenario. For raw, minimal-overhead proxying, Envoy has a slight edge. For feature-rich API gateway deployments with authentication, rate limiting, and transformations, APISIX delivers better performance with less resource consumption per feature.

In practice, both gateways are more than capable of handling the vast majority of production workloads. The real decision comes down to architecture fit: Envoy if you're building a service mesh or need deep ecosystem integration with tools like Istio, and APISIX if you want a dedicated, feature-rich API gateway that's fast, flexible, and easy to operate. Either way, you're getting a battle-tested gateway that can handle the demands of modern high-concurrency microservices architectures.