What's the magic number TTL on a doc?

SQL++
1

I am wondering what the magic number for TTL is for documents. I recall i saw there was a doc that mentioned that any nbr above X will have to be in form of timestamp and not in sec offset.
Like when i test it via
UPSERT INTO default (KEY, VALUE) VALUES ("k01", {"id":"k01"}, {"expiration":2592});
it works as expected, but when i do a
UPSERT INTO default (KEY, VALUE) VALUES (“k01”, {“id”:“k01”}, {“expiration”:25920000});
the doc disappears before i can retrieve it…

A link to the doc or specs would be helpful

2

Hi @makeawish,

Basically a TTL less than 30 days (in seconds) is relative but after that it becomes absolute - yet it can get much more complicated with a TTL on a bucket. I believe you are setting the TTL about 50 years in the past.

Please read Expiration or TTL ... second or millisecond - #7 by jon.strabala and see if that answers your questions ( and directs you to the proper documentation).

Best

Jon Strabala

3

https://docs.couchbase.com/java-sdk/current/concept-docs/documents.html#setting-document-expiration

4

Yes that’s what I was looking for the magic 30 days as there is also docs which refer to the max value of expiry of int 32… It s good and bad, because if I set a month it will be over that number if the month is 31 days it will fail. I also saw some discussion back in 16 that there might be a change in works to pass an object like value and unit type to be able to set larger offsets based on sec’s. Also will there be a change in couchbase 7 which will effect the expiry or will it be like current where a N1QL update will wipe out the expiry value and to preserve i have to get it and update with original value.
I would think the smarter approach would have been to not touch expiry, if user wants to reset or remove the meta expiry one could do that. So if I want to only update one key in doc I will always have to set expiry with old timestamp which is a bit counter productive.

5

the magic 30 days

This causes so much confusion. If we had a time machine, this is he first thing I’d change :slight_smile:

Unless you’re setting a very short expiry (like, on the order of seconds where clock drift between the client and server could be an issue) you might as well always pass the expiry as an epoch second.

max value of expiry of int 32

Yes, unsigned int 32, so the max expiry value is 4294967295 which corresponds to 2106-02-07T06:28:15Z.

I also saw some discussion back in 16 that there might be a change in works to pass an object like value and unit type to be able to set larger offsets based on sec’s.

I don’t know about N1QL, but if you’re using SDK 3 with the KV service you can pass a (let’s say Duration) of whatever length and the SDK will honor your intent.

will there be a change in couchbase 7 which will effect the expiry or will it be like current where a N1QL update will wipe out the expiry value and to preserve i have to get it and update with original value.

With SDK 3 and the KV service, Couchbase 7 will let you specify preserveExpiry=true for KV ops that would otherwise reset the expiry value. For N1QL, I’m under the impression it will still be necessary to use the technique described in the blog post vsr1 linked to above.

Thanks,
David

6

N1QL by default reset expiration. If you need to preserve you must set expiration like below.

UPDATE default AS d SET d.comment = “xyz” , META(d).expiration = META(d).expiration;

In statement if you refer META().expiration, It needs to get current expiration using SUBDOC API with full document. This require extra overhead (if query qualifies large documents you can see difference )

7

That’s why I thought it would be wiser and less overhead if the default action would be to maintain the expiry to avoid the subdoc overhead if you don’t make changes on the expiration

8

Thanks, in many cases the issues and confusion come from lack of documentation as some is in docs, some in blogs etc. and in case of the expiry quite outdated.
Where can I find more docs for NodeJS SDK 3 and Duration part ? Lack of features like " preserveExpiry=true in N1QL would be nice and since a N1QL query at the end is a fancy KV operation it should be to hard to keep it consistent.

9

Track MB-45871

1 Like
10

Sorry, I spoke out of line. The node.js SDK just takes a number and sends it directly to the server, so you do need to be careful about the 30 day threshold.

What I should have said is that the SDKs that support specifying the expiry using a Duration-like type will do the right thing.

Thanks,
David

11

No worries, I fixed it for now like this… I pass to my function a object which has unit of sec or timestamp and check the value.

  let expiresec: number = 0;
    if(option && option.unit === 'sec')

        if(option.value <= 2592000){
            expiresec = option.value;
                } else {

            let currentTimeStamp:number = new Date().getTime() / 1000
            let newTimeStamp: number = currentTimeStamp + (option.value)
            expiresec = newTimeStamp
        }


    else if(option && option.unit === 'timestamp'){
        expiresec = option.value;
    }
1 Like