Concept and indexing

I have a document that looks like this:

{
“id”: “{c1cd6c7a-b0b2-241f-f4b8-8351f230ce26}”,
“fromTime”: “2013-05-23T02:00:00”,
“value”: 3,
“timeserieId”: “{09a398cd-ed4d-4a66-afba-d86de34e2214}”,
“type”: “timevalue”
}

In my database there are about 15 million documents of this type.

I have created an index that looks like this:

function (doc, meta) {
if(doc.type == “timevalue” && doc.timeserieId && doc.fromTime ) {
emit([doc.timeserieId, doc.fromTime], null);
}
}

I want to make a query based on timeserieId and fromtime.
With the C# library i use this code:

object[] startKey = new object[] {timeserieId, from}; object[] endKey = new object[] {timeserieId, to};

var view =
client.GetView(“timevalues”, “by_fromtime_timeserieid”)
.StartKey(startKey)
.EndKey(endKey)
.Descending(false);

var timevalues =
client.Get(view.Select(v => v.ItemId))
.Select(d => JsonConvert.DeserializeObject(d.Value as string))
.ToList();

This works and it takes about 5 seconds to get data for about 10 years from 1 timeserieId. (~80k documents)

My question is if this is the right way to do it or am i missing something?
I think i read somewhere that the secondary indexes should not be so large, but it is not clear to me how i should do it if i need to query on other attributes than the key.

Any help is appreciated

Regards
Chris

Hello,

You have the proper approach.

I have you tried with a query that returns “small amount of data”? and/or at least use pagination? ( http://blog.couchbase.com/pagination-couchbase )

I have the feeling that returning 80k document is one part of the problem.

Regards
Tug
@tgrall

Most of the times i will query a more “normal” number of documents.
However there are times where this amount may be necessary.
With Oracle it retrieves the data just under a second.

Am i right to assume that couchbase would perform very well on a high number of requests rather than retrieving a large recordset for a single request?

chris