we’re using Couchbase in live production for about 5 years now.
We have different clusters with different tasks.
One of those is session storage.
We store about 10M of sessions, and update them at 1K requests/second/node.
One of problems we’re facing is that we need to do some actions if session is abandoned.
Generally speaking, to do some thing with a session, when we feel it’s time to do that.
Our current approach is to
- accumulate all actions we plan to do at a certain one second in future and stuff IDs of those in one APPEND operation to a special $BUCKET.TIMEOUT, where number of second (since EPOCH) works as a key.
- scan timeline, one second at a time, and process a block of 1000 events from each second at a time. GETL, extract 1000 events, CAS whatever events left, repeat. This works from several client nodes.
There are problems with this approach, will spare everyone from detailing.
Upon 2 weeks of investigations into internals, we feel that the following approach may be convenient.
I don’t feel this is ready for jira issue yet – wanted to discuss it first.
I welcome everybody’s comments. Especially those of dear Creators, but not limited to!
Add SDK/memcached protocol a new command
TIMER key time
It will add that information to metadata.
Once a certain time, a special pager will visit all entries (like exp_pager_stime infrastructure), and…
generate an event
$bucket $key $time happened
and put that event to… some external message queue broker.
IP of that broker to be configurable via web.
Client application could subscribe to those events.
MINUS: these timers would not tick very precise, only not oftener than pager will reach an item
MINUS: integration with external piece of software is required, and different people might prefer different protocols
PLUS: we continue to have one storage=Couchbase “all in one place, data and scheduled events inside” and do not have to invent a supporting timer-infrastructure.