Node handlers run concurrently, SPOT handlers run serially

The same three requests are handled differently in each place

Node handlers run concurrently, SPOT handlers run serially The same three requests are handled differently in each place req A · Node handlers — concurrent (worker pool) req A worker 1 · handling · Node handlers — concurrent (worker pool) worker 1 handling req B · Node handlers — concurrent (worker pool) req B worker 2 · handling · Node handlers — concurrent (worker pool) worker 2 handling req C · Node handlers — concurrent (worker pool) req C worker 3 · handling · Node handlers — concurrent (worker pool) worker 3 handling req A · SPOT handlers — serial (single queue) req A req B · SPOT handlers — serial (single queue) req B req C · SPOT handlers — serial (single queue) req C single queue · SPOT handlers — serial (single queue) single queue A → B → C · one at a time, in order · SPOT handlers — serial (single queue) A → B → C one at a time, in order Node handlers — concurrent (worker pool) SPOT handlers — serial (single queue)

What differs

  • • A node handler's requests are spread across the worker pool, so all three run at once.
  • • A SPOT handler's requests pass through one queue and run one at a time in arrival order, so no lock is needed inside.