Use startkey and endkey directly on buckets instead of views


#1

Hello to all,

I would like to fetch all documents from a bucket (with paging) starting with a pattern.

For instance to return all comments from service 1:
startkey = ‘service:1:comment:’, endkey = ‘service:1:comment://ueff’ .

I am trying to avoid creating a view since they are expensive and not consistent.

Can you please help me.


#2

Hi,

The aggregation requires a view, or something external that does something view like. You cannot wildcard pattern match against Key Values Directly without some form of aggregation taking place. Using Reference Docs as pointers and maintaining all of the sequence information is possible, but would take a lot of overhead.

Thanks

Todd


#3

Thanks for the quick reply,

I was hoping to use something similar to the one used on the couchbase manager:

I noticed that you are using the docs endpoint with the startkey and endkey, and that’s what I need, but it is not referenced in your REST API documentation.

Other databases based on the work of CouchDB have this capability and it seems a good way to avoid creating views.

I don’t understand why we need aggregation to lookup for keys since we already have the primary index constructed. (the idea is to avoid the construction of another index).

Can you explain in more detail what would be the overhead in this approach and if is there any way to use the same feature you are using in your maintenance application.


#4

Unfortunately the internal aggregate view your’e referring to is not supported publicly, and is no longer part of the product in the latest version (4.0). Have you considered using pattern matching, through the query api and N1QL that is part of 4.0? This type of wildcard search is well supported and could provide the flexibility you’re after.

Thanks

Todd


#5

Thanks Todd,

When you first reply that was the approach that I was going to. Something like:

var query = N1qlQuery.fromString('SELECT * FROM default where meta.id() like '‘service:1:comment:%’);

but them i notice that is only available in the 4.0 version, and for now we are using the community edition.

Can you tell me if this query use the primary index under the hood?

Looking forward to see this version in production.


#6

The indexing in 4.0 is completely in memory, and user defined. Primary indexes and secondary are both supported, and query engine (cbq engine) that runs n1ql queries has built in explain plans and optimization which make it really easy to determine which indexes are being utilized.

Thanks

Todd