Hello,
I’m working on couchBaseServeur 7.0 Entreprise and i have an issue when i want to deploy my Event function.
The error is the next :
“Code : 52 in info which says tha the SyncGareway is enabled on : , deployment of source bucket mutating handler will cause Intra Bucket Recursion”
You can see below my settings.
This error is only when i need a bucket in read/write.
Any idea ?
Hi @jlefray,
SGW uses XATTRs which look like mutations to Eventing (it a hard problem to solve trust me), thsi is an issue you don’t bring up. Your issue in ability to share the same source bucket as SGW with Eventing in a r+w Eventing mode.
When using Eventing with sync gateway there are a few work arounds depending on what you are doing.
Case 1
I will start with the first use case. Suppressing duplicate mutations (the source bucket is read only in Eventing) The work around in this case is to use crc64 and is documented in Language Constructs | Couchbase Docs
Case 2
The second work around (the source bucket is read+write in Eventing) is a bit more complex …
The work around it requires an extra bucket (or collection your choice - but I will discuss buckets here) and two (2) Eventing Functions. We do have the ability to enable cross bucket recursion (or cross collection it is the same), but it doesn’t appear that there is a setting to disable the SGW on source bucket restriction .
curl -X POST -u $CB_USERNAME:$CB_PASSWORD http://localhost:8091/_p/event/api/v1/config \
-d ‘{“allow_interbucket_recursion”:true}’
Note, the above disables safety checks and allows you to shoot yourself in the foot, so use with great caution.
To avoid infinite recursion you can using a staging bucket (or collection) along with a second helper Eventing Function as follows (you would need to run the above cURL command (to set “allow_interbucket_recursion”:true) to allow the two functions to coordinate and deploy):
- SGW & your “primary_sgw_recursion_7X” Eventing Function listen to bucket ‘primary’ (with Eventing alias ‘primary_bkt’)
- Anytime the “primary_sgw_recursion_7X” Eventing handler wants to update the bucket binding alias ‘primary_bkt’ bucket it writes to a helper bucket ‘helper’ (with Eventing bucket binding alias ‘helper_bkt’)
- The second “helper_sgw_recursion_7X” Eventing function has a source bucket of ‘helper’ (with bucket binding alias ‘helper_bkt’) and can write to itself and the bucket ‘primary’ (with Eventing alias ‘primary_bkt’). Anytime a mutation is seen by the “helper” Handler it merely writes the item to the alias primary_bkt and deletes it from the alias helper_bkt.
- I added a tracking field to count loops but that is not necessary, in fact after I got rid of the recursion it was useless
- Now please understand that I am not super very familiar with SGW but I do know that it sends a duplicate mutation, your question was different, i.e. on how to avoid infinite recursion. To this end here I have provided you with two toy functions for you to experiment with.
The functions a) suppress both the SGW duplicate mutations and b) also suppress the infinite recursion loop. I am pretty sure this will work for your use case.
In order to suppress loops we track two CRC’s in the helper bucket ( to keep the helper bucket data manageable documents can auto expire based on an optional bucket TTL if you set one - or alternatively you could set a TTL on each document itself in the helper bucket via Eventing itself )
- The input CRC of our doc to suppress SGW duplicates
- The CRC of all docs we write to the ‘helper’ bucket (which get written to the ‘primary’ bucket) to prevent infinite loops
Then we merely check to see if they exists (if so we short circuit further mutation processing)
For more details read the comments inside the updated “primary_sgw_recursion_7X.json”
Refer to the two attached Eventing functions (in sgw_recursion_7X_two functions.zip):
helper_sgw_recursion_7X.json
primary_sgw_recursion_7X.json
Attached …
Make two buckets (or collections)
primary
sgw_recursion_7X_two functions.zip (2.7 KB)
helper
(of course a bucket or collection your eventing storage called metadata)
Attach SGW to the bucket primary
Import the two functions and deploy them your setting screens should similar to the following:
Add a test document with any key you want to the the “primary_sgw_recursion_7X” function’s source bucket “primary”, note it only keys off of field “category”
KEY ATST::1
{ “type”: “ATST”, “id”: 1, “category”: “for_eventing”, “data”: “other …” }
Yes I said that this solution Case 2 was a bit complicated, hopefully I have given you enough to work with.
Best
Jon Strabala
Thanks a lot Jon. Sure it’s a bit complicated but it works!