Key timeout expiration behavior and stats

[bdw429s]

Hello, I'm using the Java Client and CouchBase 2.0.1 community edition (build-185)
I'm writing a provider to allow CFML developers to use CouchBase inside a cache aggregator called CacheBox (part of the ColdBox platform). I'm trying to get the correct metadata from the CouchBase keys for the reporting portion of the library. Data I would ideally like to get for all keys in the bucket is:

• Key Name (retrieved via view)
• Hits (Not sure this is possible)
• Timeout (time before item expires regardless of hit count)
• IdleTimeout (time before item expires counted since last hit-- I don't think CouchBase support this))
• Created (time key created -- This does not appear to exist in key stats)
• Last Accessed (Time key was last "hit")
• Status (active/expired)

I been playing with the stats that are available for a bit but haven't been able to nail down some specific answers after a bit of Googling so I wanted to confirm here.

Can I determine from the key stats what the initial timeout of a key was set to when it was first set into the bucket?

Also, can I see when a key was created?

The next question: is there a date that stores the last hit to a key? The key_last_modification_time key stat appears to change to the current time every time I access the stats, however the expiration date doesn't change. So it basically always looks like my keys were just touched. Is this a bug since viewing stats doesn't seem like it should count as "modifying" the key.

And finally, I think I saw an answer to this floating around Google, but to confirm, are there key-level hit stats available?

Thanks for your help.

~Brad

[tgrall]

First of all sorry for this inconvenience with the "staging" server; a typo in the message. However you found your way

[bdw429s]

Is there any additional information I can provide to help someone answer these questions for me?

[bdw429s]

Am I not going to be able to get an answer out of the community for these questions? :/
Would it help if I asked separately for each question?

[perry]

Hey Brad, I'll do my best to answer your questions. Some of this is going to be available, some of it not, some of it may be in the future


In many cases, Couchbase is actually trying to reduce the amount of metadata that it imposes on a per-key basis due to RAM and disk size requirements, so doing this across the board would actually be to the detriment of some of our customers.


I think for many of these, your best bet is going to be to include one or more timestamps within your own JSON values and then you can have a view off of that.


To your questions:
Can I determine from the key stats what the initial timeout of a key was set to when it was first set into the bucket?
[pk] - No, the TTL that is recorded with a key is it's current expiration time


Also, can I see when a key was created?
[pk] - Not at this time. One challenge here would be dealing with keys that get deleted and recreated..."created" time might be inaccurate. I believe we are looking into a change that would persist the last written time, but that's not there yet.


The next question: is there a date that stores the last hit to a key? The key_last_modification_time key stat appears to change to the current time every time I access the stats, however the expiration date doesn't change. So it basically always looks like my keys were just touched. Is this a bug since viewing stats doesn't seem like it should count as "modifying" the key.
[pk] - Yes, that's a bug and if I recall correctly, that stat is being removed altogether to save space. As I mentioned above, it's likely that we will write the "last written" time into the document's metadata on disk. Storing the "last hit" time is likely not going to be something we add due to the performance overhead.


And finally, I think I saw an answer to this floating around Google, but to confirm, are there key-level hit stats available?
[pk] - Sort of but not really. We do a bit of tracking the more frequently accessed keys but while that's per-key for the time that it's in the most frequently accessed list, all the stats are not kept around for all keys.


I just realized that I basically said no to all your questions, I was hoping I'd have a different result, sorry


I think our best recommendation would be to do this within your application. The "hit" stats get a little tricky, but maybe you could do something interesting with the setting of the TTL using "get and touch" that you can then use to determine when it was last hit, or how many times, etc...something like "encoding" those stats within a Unix timestamp that is sufficiently far into the future that the key wouldn't ever expire? I'm just grabbing a wild guess there, but might be something interesting...


Perry

[bdw429s]

Thanks for the reply Perry. Don't worry about saying no to everything. I think all the questions that were a "yes" I'd already figured out. This was basically the list of stuff I didn't think was possible but wanted to make sure

We had the same thought as you which was to wrap the object being stored in a JSON doc that had a little metadata about itself.

Yes, that's a bug and if I recall

Should I enter a ticket or is there one already?

[perry]

I just confirmed that it has already been removed.

[bdw429s]

Ahh-- I guess I should go delete some code that tries to use it