Couchbase view using "WHERE" clause dynamically


#1

I have Json Documents in the following format

Name :
Class :
City :
Type :
Age :
Level :
Mother :
Father :

Now I have a map function like this

function(doc,meta)
{
    emit([doc.Name,doc.Age,doc.Type,doc.Level],null);
}

Now What I can do is give “name” and filter out all results but what I also want to do is give “age” only and filter out on that. For that couchbase does not provide functionality to skip “Name” key So I have to create a new map function which has “Age” as first key. But I also have to query on only “Level” key also so like this I would have to create many map functions for each field which obviously is not feasible so is there anything I can do apart from making new map function to achieve this type of functionality.


#2

This is a limitation of views and exactly why N1QL was built. Have you looked into it? It’s available from Couchbase Server version 4.0 onwards (there’s also 4.1 and an upcoming 4.5 versions).


#3

n1ql takes a lot of time. I have approximately 10-15 crore documents. Can anything be done using a combination of views and n1ql? to speed up this method?


#4

have you tried creating indexes in N1QL? what do you mean by “a lot of time”? 10-15 core documents sounds like a small number that shouldn’t cause any problem…


#5

No I mean 100 million to 150 million documents. For my testing I tried my query using N1QL with 50k documents and it took around 10-12 seconds for N1QL to execute


#6

ah yeah that makes much more sense :smile:
perhaps @geraldss or @keshav_m have some insight on performance in such a configuration.


#7

Yes, @keshav_m, please share your insights. With proper indexing and configuration, N1QL performs well with these scales and much bigger.


#8

I have tried indexing with N1QL but still takes a lot of time :frowning: . Any ideas on how can I get about this problem. and the number of fields in original documents is around 50 and I can have filter on any field as well as any number of fields and not to forget number of documents are in range of 100 million to 150 million


#9

Out of the 50 fields, (1) how many fields do you need to query on, and (2) how many fields do you need to return in the initial result (before the user selects a specific document)?

Did you use EXPLAIN to verify your use of the indexes you created? did you create single-field or multi-field indexes?


#10

Out of 50 fields I can query on any field(as well as multiple fields together also) and I need to return all fields on document as result.
I didnt use N1QL myself. Someone else from my team used it and found very bad performance even with indexing. Anyways I will ask him and let you know how the indexing is done. But I am trying to find a solution using map-reduce and views because they are a lot faster than N1QL