Some errors with python client under concurrency


#1

I am a new guy to couchbase.I try to use couchbase and couchbase’s python client in my python-base web APIs.But I find some errors when I use ab(a tool to test the web api performence) to test the api with some concurrecny http requests. The errors of the python clinet throwed bellow here.I want know why these errors occoured,and how to solve this problem.

The errors


uwsgi: src/server.c:666: lcb_server_purge_implicit_responses: Assertion `c->cmd_log.nbytes’ failed.
python-couchbase: self->nremaining == 0 at src/oputil.c:77. AbortTraceback (most recent call last):
File “api.py”, line 111, in falcon.api.API.call (falcon/api.c:1389)
# startkey=start, endkey=end,
File “api.py”, line 96, in falcon.api.API.call (falcon/api.c:1210)
# end = ‘/’ + adid + ‘/’ + end
File “api.py”, line 174, in on_post
query_results = couchbase_category.get_multi(keys, quiet=True)
File “/usr/local/lib/python2.7/dist-packages/couchbase/connection.py”, line 1053, in get_multi
return _Base.get_multi(self, keys, ttl=ttl, quiet=quiet, replica=replica, no_format=no_format)
couchbase.exceptions._TimeoutError_0x17 (generated, catch TimeoutError): <Key=u’com.htc.fm’, RC=0x17[Client-Side timeout exceeded for operation. Inspect network conditions or increase the timeout], Operational Error, Results=34, C Source=(src/multiresult.c,286)>

My python code

couchbase_category = Couchbase.connect(
bucket=‘Category’, host=‘127.0.0.1’, timeout=5)
class Category:

def on_post(self, req, resp):

    t1 = datetime.datetime.now()
    content = {}
    # try:
    params = json.load(req.stream, 'utf-8')
    keys = []
    for app in params['apps']:
        keys.append(app['bundle_id'])

    unknown_apps = []
    result = {}
    result['known_bundle_ids'] = []
    result['unknown_bundle_ids'] = []
    result['apps'] = []
    result['categorys'] = []

    query_results = couchbase_category.get_multi(keys, quiet=True)
    for key in query_results:
        if query_results[key].success:
            result['apps'].append(query_results[key].value)
            result['known_bundle_ids'].append(key)
            result['categorys'].append(
                query_results[key].value['categoryName'])
        else:
            result['unknown_bundle_ids'].append(key)
    result['categorys'] = list(set(result['categorys']))
    for item in params['apps']:
        if item['bundle_id'] in result['unknown_bundle_ids']:
            unknown_apps.append(item)
    timestamp = time.time()
    pipe = r.pipeline()
    for app in unknown_apps:
        pipe.zadd('uncached_apps', timestamp, app)
    pipe.execute()
    content['is_ok'] = True
    content['result'] = result
    t2 = datetime.datetime.now()
    content['total_time'] = (t2 - t1).total_seconds()
    # except Exception, err:
 #   content['is_ok'] = False
    # finally:
    resp.status = falcon.HTTP_201
    resp.body = json.dumps(content)

#2

Looks to me like your Couchbase Server is responding slow, and therefor the client gets a timeout which is uncaught and therefor bubbles up to show up in the console. I guess there is some configuration setup so your server maybe runs out of memory for the bucket you are using and therefor swaps to disk? Or your server has a bad connection to the Couchbase Server which causes the timeouts? This is just a first guess, hope I can help more with more details on the setup.