nrtb-core team mailing list archive

[Rick Stovall <fpstovall@xxxxxxxxx>]

Aron asked about situations with multiple consumes where all attached consumer threads may not be used. First, let's let's have a little background. When a thread enters pop(), one of two things happen. 

In case 1, there is data in the queue and it pops the top one off and returns immediately. 

In case 2, the queue is empty and the thread blocks and is put in an OS managed wait queue waiting on the next signal.

If any threads are blocked the signal queue, the next call to push() will release one to pick up the new queue item and proceed. Each call to push() causes the release of the next thread in the OS queue.

Given the above, let's look at two extremes.. first, where the queue is loaded before any consumer threads are attached, and second, an empty queue with many consumers waiting which gets a sudden rush of data.

In the pre-loaded queue model, case 1 dominates. As each new consumer thread is attached and started, it calls pop(), finds data in the queue and immediately returns with the data to process. No one blocks until the queue is empty, so the work is as broadly distributed as possible.

In the empty queue with many consumers model, we have a number of consumers blocked on the wait queue. As each new item is pushed on the queue, it signals the OS to release the next thread in the wait queue. If we insert 100 items and we have 10 consumers, all the consumers will be released by the first 10 messages, and once released each runs under case 1 until the queue is exhausted again.

Obviously, if you never have more than n messages in queue, you'll never run more than n threads at a time. This is okay.. a waiting thread does not consume CPU cycles. The important point to remember is that it will use all the threads as needed to process the load as it's presented, and no more than that. Cycles are not wasted polling for work, nor is there the typical 0.5(polling interval) start-up latency you find with polling models. Cheaper and faster.. what more could you ask for?






-- 
https://code.launchpad.net/~fpstovall/nrtb/circular_queue/+merge/78527
Your team NRTB Core is subscribed to branch lp:nrtb.