Pass txid to Cluster.query

[dmaarouf]

Is it possible to pass the txid when performing queries through the Python SDK?

Using CB server 7.0.2 and SDK 3.2.2

[vsr1]

Checkout GO thread , it may help you and provide useful info.

N1QL transactions with Go SDK and lost updates

The docs state that N1QL transactions can be used at any place N1QL can be used. I am using the Go SDK which doesn’t have explicit support for transactions but can use the REST API to communicate with the query service. The docs also state that CAS value will be used to make sure updates are not lost in case of concurrent transactions that touch the same document. var wg sync.WaitGroup for i := 0; i < 100; i++ { wg.Add(1) go func() { results, err := cluster.Query("START TRANSACTION;",…

[dmaarouf]

I tried passing the txid with “raw”, the SDK seems to not be passing it to the server

c = Cluster('couchbase://localhost', authenticator=auth)

q = "START TRANSACTION"

r = c.query(q)
rows = [row for row in r]
txid = rows[0]["txid"]
print(txid)

q = "COMMIT TRANSACTION"

opts = QueryOptions(raw={"txid": txid})
r = c.query(q, opts)

for row in r:
    print(row)

Logs:

496ms [Ibee9c47e23504bd9] {32674/103} [INFO] (connection - L:159) <localhost:8091> (SOCK=be5f44c841515f2f) Connected established
496ms [Ibee9c47e23504bd9] {32674/103} [DEBUG] (connection - L:106) <localhost:8091> (SOCK=be5f44c841515f2f) Successfully set TCP_NODELAY
496ms [Ibee9c47e23504bd9] {32674/103} [DEBUG] (connection - L:106) <localhost:8091> (SOCK=be5f44c841515f2f) Successfully set TCP_KEEPALIVE
497ms [Ibee9c47e23504bd9] {32674/103} [DEBUG] (lcbio_mgr - L:369) <localhost:8091> (HE=0x7feee153bcc0) Received result for I=0x7feee154e490,C=0x0; E=0x0
497ms [Ibee9c47e23504bd9] {32674/103} [DEBUG] (lcbio_mgr - L:329) <localhost:8091> (HE=0x7feee153bcc0) Assigning R=0x7feee15183f0 SOCKET=0x7feee154e4f0
497ms [Ibee9c47e23504bd9] {32674/103} [DEBUG] (ioctx - L:94) <localhost:8091> (CTX=0x7feee1564530,unknown) Pairing with SOCK=be5f44c841515f2f
500ms [Ibee9c47e23504bd9] {32674/103} [DEBUG] (ioctx - L:139) <localhost:8091> (CTX=0x7feee1564530,mgmt/capi) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1
502ms [If7aec5665e89ed0d] {32674/103} [TRACE] (http-io - L:394) <localhost:8093> POST  http://localhost:8093/query/service. Body=111 bytes
502ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (lcbio_mgr - L:501) <localhost:8093> (HE=0x7feee206f160) Creating new connection because none are available in the pool
502ms [If7aec5665e89ed0d] {32674/103} [TRACE] (lcbio_mgr - L:413) <localhost:8093> (HE=0x7feee206f160) New pool entry: I=0x7feee206dfe0
502ms [If7aec5665e89ed0d] {32674/103} [INFO] (connection - L:497) <localhost:8093> (SOCK=34a5da2c359bc633) Starting. Timeout=75000000us
503ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (connection - L:262) <localhost:8093> (SOCK=34a5da2c359bc633) Created new socket with FD=6
503ms [If7aec5665e89ed0d] {32674/103} [TRACE] (connection - L:362) <localhost:8093> (SOCK=34a5da2c359bc633) Scheduling I/O watcher for asynchronous connection completion.
503ms [If7aec5665e89ed0d] {32674/103} [TRACE] (n1qlh - L:426) (NR=0x7feee1610bd0) execute query: {"client_context_id":"c9fdd0c9ce53fb32","metrics":false,"statement":"START TRANSACTION","timeout":"75000000us"}, idempotent=false, timeout=75000000us, grace_period=0us, client_context_id="c9fdd0c9ce53fb32"
503ms [If7aec5665e89ed0d] {32674/103} [INFO] (connection - L:159) <localhost:8093> (SOCK=34a5da2c359bc633) Connected established
503ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (connection - L:106) <localhost:8093> (SOCK=34a5da2c359bc633) Successfully set TCP_NODELAY
503ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (connection - L:106) <localhost:8093> (SOCK=34a5da2c359bc633) Successfully set TCP_KEEPALIVE
503ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (lcbio_mgr - L:369) <localhost:8093> (HE=0x7feee206f160) Received result for I=0x7feee206dfe0,C=0x0; E=0x0
504ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (lcbio_mgr - L:329) <localhost:8093> (HE=0x7feee206f160) Assigning R=0x7feee206df30 SOCKET=0x7feee206e0d0
504ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (ioctx - L:94) <localhost:8093> (CTX=0x7feee206d0f0,unknown) Pairing with SOCK=34a5da2c359bc633
507ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (ioctx - L:139) <localhost:8093> (CTX=0x7feee206d0f0,mgmt/capi) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1
507ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (lcbio_mgr - L:554) <localhost:8093> (HE=0x7feee206f160) Placing socket back into the pool. I=0x7feee206dfe0,C=0x7feee206e0d0
90d967af-f500-42f3-a29c-196573dfd373
507ms [If7aec5665e89ed0d] {32674/103} [TRACE] (http-io - L:394) <localhost:8093> POST  http://localhost:8093/query/service. Body=112 bytes
507ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (lcbio_mgr - L:492) <localhost:8093> (HE=0x7feee206f160) Found ready connection in pool. Reusing socket and not creating new connection
507ms [If7aec5665e89ed0d] {32674/103} [TRACE] (n1qlh - L:426) (NR=0x7feee1544ea0) execute query: {"client_context_id":"6b5da2f68f3fa216","metrics":false,"statement":"COMMIT TRANSACTION","timeout":"75000000us"}, idempotent=false, timeout=75000000us, grace_period=0us, client_context_id="6b5da2f68f3fa216"
507ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (lcbio_mgr - L:329) <localhost:8093> (HE=0x7feee206f160) Assigning R=0x7feee156bb00 SOCKET=0x7feee206e0d0
507ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (ioctx - L:94) <localhost:8093> (CTX=0x7feee154e490,unknown) Pairing with SOCK=34a5da2c359bc633
510ms [If7aec5665e89ed0d] {32674/103} [TRACE] (confmon - L:319) Refreshing current cluster map (bucket: (null))
510ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (ioctx - L:139) <localhost:8093> (CTX=0x7feee154e490,mgmt/capi) Destroying context. Pending Writes=0, Entered=true, Socket Refcount=1
510ms [If7aec5665e89ed0d] {32674/103} [DEBUG] (lcbio_mgr - L:554) <localhost:8093> (HE=0x7feee206f160) Placing socket back into the pool. I=0x7feee206dfe0,C=0x7feee206e0d0
Traceback (most recent call last):
  File "cb2.py", line 31, in <module>
    for row in r:
  File "/Users/dmaarouf/dev/couchbase-python-client/couchbase_core/__init__.py", line 281, in __iter__
    next_item = next(parent_iter)
  File "/Users/dmaarouf/dev/couchbase-python-client/couchbase_core/n1ql.py", line 585, in __iter__
    raw_rows = self.fetch()
  File "/Users/dmaarouf/dev/couchbase-python-client/couchbase_core/n1ql.py", line 590, in fetch
    return self.raw.fetch(self._mres)
couchbase.exceptions.HTTPException: <RC=0x41D[LCB_ERR_HTTP (1053)], HTTP Request failed. Examine 'objextra' for full result, Results=1, C Source=(src/pycbc_http.c,203), OBJ=ViewResult<rc=0x41D[LCB_ERR_HTTP (1053)], value={'requestID': '1dc9d251-6702-42f3-ba63-7aa9551e076b', 'clientContextID': '6b5da2f68f3fa216', 'errors': [{'code': 17002, 'msg': 'COMMIT statement is not supported outside the transaction'}], 'status': 'fatal'}, http_status=0, tracing_context=0, tracing_output=None>, Context={'first_error_code': 17002, 'http_response_code': 500, 'first_error_message': 'COMMIT statement is not supported outside the transaction', 'statement': 'COMMIT TRANSACTION', 'client_context_id': '6b5da2f68f3fa216', 'query_params': '', 'http_response_body': '', 'endpoint': 'localhost:8093', 'type': 'QueryErrorContext'}, Tracing Output={":nokey:0": null}>

[graham.pople]

Hi @dmaarouf
Indeed, that does look to be the case. Paging our Python SDK expert @jcasey on that one.

However, you’ll find another issue when trying to use the SDK for this purpose: the same query node must be used for all statements within the same transaction, and for this you will need to use REST directly.

We are in the process of working on full transactions libraries for other languages, including Python, which will take care of that detail - as well as providing a nice API, and full error handling that will retry the transaction attempt where necessary. So that will be the recommended solution once it is released.

[jcasey]

I will need to do some investigation into the QueryOptions. Will look to get feedback today or tomorrow.

Concerning a Python transactions library, while timelines for availability are not concrete at the moment, I can say that we are actively working to provide this feature to users.

[jcasey]

Sorry for the delayed response, but I have created a ticket to track the fix the Python SDK needs in order to propagate the raw option in QueryOptions correctly. Should be able to get it the SDK for the 3.2.4 release.