How to get NTP time from 64 bit HLC or CAS?

Hi,
There is a need to retrieve the NTP time from Couchbase DCP packet.
I found XDCR Opcode, Set with Meta Request, contains additional CAS in the extras.


Does this one means the HLC? referred to https://github.com/couchbase/goxdcr/blob/master/docs/sherlock_xdcr_lww_design.md.
After got some knowledge from Logical Physical Clocks and Consistent Snapshots in Globally Distributed Databases .

We propose the following scheme for combining l and
c and storing it in single 64 bit timestamp. This scheme
involves restricting l to track only the most significant 48
bits of pt in the HLC algorithm presented in Figure 5.
Rounding up pt values to 48 bits l values still gives us
microsecond granularity tracking of pt. Given NTP synchronization levels, this is sufficient granularity to represent NTP time. The way we round up pt is to always take
the ceiling to the 48th bit. In the HLC algorithm in Figure 5, l is updated similarly but is done for 48 bits. When
the l values remain unchanged in an event, we capture
that by incrementing c following the HLC algorithm in
Figure 5. 16 bits remain for c and allows it room to grow
up to 65536, which is more than enough as we show in
our experiments in Section 5

I am confused about the CAS.
The first 48 bit in the CAS does not conform to the NTP time when the key were mutated, the first 32 bits of CAS should starts with “e06”, however, it starts with “159”
Any help would be appreciated.

@ingenthr @drigby any idea?

Hello @sfxu,

It would be good to know what problem is being solved by using the time from the mutation. I ask because this is an internal implementation detail that can change without warning between versions. As a result there might be a better solution.

It should be noted that the Couchbase Server uses the time provided by the operating system (OS), which might not always be NTP. We do recommended that OS is configured to use NTP.

Thanks a lot @pvarley,
I just want to retrieve time information, which may be unix timestamp, NTP time or some other from the CAS in the extras.
Moreover, to calculate XDCR time delay in seconds or microseconds, other than the changes left from web console.
Logical Physical Clocks and Consistent Snapshots in Globally Distributed Databases told me may the most significant 48 bits of CAS just conform to the format of NTP, but indeed it seems incompatible.
So I wanna to get some knowledge about the algorithm which is used to encode time into CAS and what’s the meaning of the time.

CB 6.0 has provided the subdocument API, which looks like this:

// Insert 10 documents, the old way
for (String id : keys) {
    JsonDocument doc = JsonDocument.create(id, content);
   bucket.upsert(doc);
   System.out.println(bucket.lookupIn(id).get("$document", new SubdocOptionsBuilder().xattr(true)).execute());
}

it prints such information:

DocumentFragment{id='javaDevguideExampleBulkGet_ak_9',
 cas=1555559906546548736, mutationToken=null}
[GET($document){value={"exptime":0,"deleted":false,
"CAS":"0x1596757e488c0000",
"seqno":"0x0000000000000003",
"datatype":["json"],"vbucket_uuid":"0x0000f4df42f895e1",
"flags":33554432,"value_crc32c":"0x0d51c812",
"value_bytes":32,"last_modified":"1555559906"}}]

It seems like the last_modified derives from cas, but cas has got 9 more numbers than last_modified, does the 9 more numbers mean extra precision in nanoseconds?

Does the cas mean the timestamp when the document has been mutated for the last time in nanoseconds?@daschl

May the output above come from Statistics(opcode 0x10) response?

The CAS in the output above seems to be the same as the CAS in extras of Set with meta(opcode 0xa2) request, still confused about what does this CAS mean and how it’s encoded.

Looking forward to your replies @pvarley @ingenthr @drigby

I made it sophisticated.Indeed it’s simple.
As for the version of Couchbase CE 4.1.0, take CAS of 0x15a4aae0d37a0000 for example,
there is no HLC in the CAS, it’s totally just about the physical time of the system.
In a big endian way, so 0x15a4aae0d37a0000 just means 1559559253574418432,
which is the unix timestamp in nano seconds.
OMG, that paper misled me.

In newer versions of Couchbase Server CAS is not the unix timestamp in nano seconds, it is a HLC.

Yeah,I am not familiar with the newer version.
But as for the HLC, I guess the most significant 48 bits of CAS still related with the unix timestamp in nano seconds, and “16 bits remain for c and allows it room to grow
up to 65536, which is more than enough as we show in
our experiments in Section 5”, quoted from Logical Physical Clocks and Consistent Snapshots in Globally Distributed Databases