Same View Different Query Time

Couchbase Server
1

It is wierd! i have the same problem my data set consists of 3500 json documents and each has 10-30kb sized Json documents.

Everytime i run a query Which uses a very easy map function (emits just meta.id and doc itself) it takes 0.6s then 10s and 0.8s 1.5s so on for fetching 50 Json documents.
all tests were executed with just one client that retrieves 50 json documents from default bucket.
I have been using .Net sdk 1.2.4. My application is .net windows form application.
If I run this software 5 clients at the same time, retrieval time is getting longer. This software easily executes a query on a view then enumerates the result and adds a gridview to show

My server has 2 nodes total 6gigabytes of ram for bucket and 9terra bytes of disk space (6 terrabytes for data files and 3 terrabytes for index files each are on different scsi disks)
2x8 core processor.

I have only one bucket which is default bucket for this simple test nothing else!!!

Why query times for same view differs so much each time I execute???
Why couchbase can not handle this easy test data retrieval operation?

2

We develop our institutional project. We use couchbase for data management.We try same query many times from couchbase .Thats are same query but executing times are different.First 0.2 seconds second 1.7 third 0.6 …How can i solve this problem.Because now couchbase has test data.It is only 3000 record.If we will use it in realtime, it will 6.000.000 record and it will increase linear.
Anybody help us?

3

Hello,

Could you also give use more information about:

  • How many nodes do you have in your cluster?
  • Which SDK are you using to call the view?
  • Which parameters are you using when calling the view?
    This will help the community to answer your question

Regards

4

I think I have explained all information you need in detail my previous post.

  • I have 2 nodes.
  • .NET SDK 1.2.4 is used
  • List MyList;

var view = client.GetView(“designname”,“ViewName”,false).Limit(50);
IView data = view;
MyList = data.ToList;

Regards.

5

  1. As far as I remember in .net sdk default value for stale parameter is “update_after”. So if you run your tests with stale “update_after” your view index will be updated after query, and if there is no timespan between your queries, io and cpu load will not be equal and one query will be slower than other.

  2. Also view indexes are updated periodically so some of your test queries could arrive at same time when couchbase performs view update. So in this case query times can differ too.

  3. If your app creates doesn’t use connection pool and creates new connection on first query (or each query) also makes query time bigger.

  4. If nodes doesn’t located in one LAN with minimal latency, network issues can make query slower too.

6

Hello,

Skella you are right the default value for stale is “update_after”.

You should try to run your queries from the admin console and look at the full URL, and see how it behaves.

Regards
Tug

7

Hi,

I resolved the case. Initially I thought that pulling documents one by one was not efficient way to do, but i was wrong somehow it is!!! Normally when you do kind of work, you execute your query and get bulk data and show the users. But here it is different. After you execute your query you get a view that includes your keys then for each row you get values by using these keys “ne by one(!)”. It is wierd but it is the most efficient way to do this for minimum latency.

This is good way for latency:

var view = client.GetView(“DesignName”,“getDocumentsAll”).Limit(somelimit);

foreach(var row in view){
YourClass yourclass = client.getJson(row.ItemId);
somelist.Add(yourclass);
}
dataGridView1.DataSource = somelist;

This is bad way for latency:

var view = client.GetView(“designname”,“ViewName”,false).Limit(50);
IView data = view;
MyList = data.ToList();
dataGridView1.DataSource = MyList;

my latency values are decreased from maximum 5s to 0.8 seconds for 50 documents (minimum 40Kbytes)
Good Luck

Regards…