Support for newer gevent versions (1.3.x)


#1

Hello,
I plan to use gevent in my project and as described in the documentation I need to use gcouchbase module.
Per documentation gevent versions 0.13 and 1.0.0 are tested (supported?).
Since gevent 1.0 is from Nov 26, 2013 which is really old, I’ve been testing versions 1.1.2, 1.2.2 and 1.3.5 (latest - 2018). Note that those newer versions also contain many bug-fixes and security updates.
I am using Couchbase community edition version 5.0.1, python-sdk versions 2.3.2 (but also tested with 2.4 which is the latest) with Python 2.7.13 on Ubuntu 16.04.

My findings:

  1. gevevnt 1.1.2 using gcouchbase seems to work fine.
  2. gevevnt 1.2.2 using gcouchbase seems to work fine.
  3. gevevnt 1.3.5 using gcouchbase fails right at the start when trying to create a connection. The error I’m getting is:
    <RC=0x17[Client-Side timeout exceeded for operation. Inspect network conditions or increase the timeout]

So my questions are:

  1. Are gevent versions 1.1.2 and 1.2.2 officially supported and the documentation is just not updated?
  2. Is it safe to use gevent versions 1.1.2 or 1.2.2 (i.e. do you know about any major issues)?
  3. When are you planning to add support for the latest gevent versions (1.3.x)?
  4. Do I need any special libcouchbase dependency for working with gcouchbase (for example libcouchbase2-libevent must be installed)?

Thanks in advance!


#2

Hi, we haven’t done much testing of gcouchbase with newer gevent versions, so can’t yet guarantee support above 1.0. but my own tests verify what you have seen. Hopefully in SDK3 we will rework async support to make it even easier to retrofit different async APIs and keep things up to date.

Do you have a specific use case?

Many thanks,

Ellis


#3

Hi Ellis,
Thanks for the fast response!
Our use case is this:
We have a REST API server (Flask application running on Gunicorn with gevent workers - i.e with --worker-class gevent) which performs CRUD operations against Couchbase.
In order to get a non-blocking behaviour, we use the gcouchbase module and see that Couchbase successfully returns data for multiple short queries while there is a long query executed in parallel (i.e. the long query doesn’t block the short ones). So we can get really good performance and responsiveness from a small number of Gunicorn workers.

Our concern, based on the usage of old gevent versions, was that the gevent support (gcouchbase) wouldn’t be maintained in the future. Am I correct in assuming that is not the case (i.e. you are still committed to supporting gevent)?
Do you have a rough estimate when v3 of the SDK will be released?

Thanks!


#4

Hi, I believe SDK3 is scheduled mid-next year, but is subject to change.

As far as I know the current status of all the async APIs is supported, but I’ll discuss gevent with our product managers. Certainly, we are committed to supporting asynchronous operation.

Twisted is used extensively in house so will continue to be supported, and asyncio is the de-facto Python 3 async framework now.

Both of these look like they can integrate with Flask. I’m preparing for an imminent release this week, but can have a look at the issues with gevent on Monday. I do remember from earlier benchmarking that gevent did pretty well, so hopefully it’s not a complex problem to fix.

The good news is most of the async wrapper code for the various async frameworks is in the Python world and fairly lightweight, so considerably better placed for user contributions/debugging.

Hope that helps,

Ellis


#5

Hi Ellis,

Thanks a lot for looking into these issues.
We prefer Gunicorn with gevent workers and gcouchbase because the implementation is really straightforward and easy to understand. Unlike other other frameworks, gevent brings ‘automagically’ non-blocking behavior with a really good performance while the code changes are really minor.
So if gcouchbase will be maintained in the future it sounds like very good news for us.

Thanks,
Denis