_bulk_docs returns 400 - Bad JSON :: CBLite Push Replication Fails

Our production users are getting failed push replications more and more often and I cannot diagnose this error.

Server: 5.0.1-5003-enterprise
Sync Gateway 1.5
CouchbaseLite: 1.4.1

Our production users are starting to get sync failures for unknown reasons. We have not changed anything related to our document schemas, and have no issues saving documents into CBLite. However when certain users sync (push replicate), The _bulk_docs api fails with this obscure error. I am using CBLReplicator to sync, so this _bulk_docs is all under the hood of the CBLLite library.

  1. How can I increase logging in CBLite iOS to see what the request body looks like in this API request?
  2. How can I increase logging in SyncGateway to understand what the “Bad JSON” source is?
  3. What else might throw this error? Is it possible the request body is TOO LARGE for the endpoint to handle in Sync Gateway?

Regards,

Andrew

I have a vague memory of this (sorry, it’s been years since I worked on 1.x.) IIRC there was a bug in CBL-iOS that caused invalid JSON to be generated in some rare cases, which got fixed in some 1.4.x release (where x > 1).

Excuse me, we are running CBLite 1.4.4, the last release. Iappears that CBLite is sending broken chunks of JSON. They appear to be very large request, but not sure why this would start occurring last week, we’ve been syncing very large data sets for years.

Any chance you could find a PR or commit with this fix?

It seems as though the size of the request in the batch bulk_docs may be too large of a body. I think it’s his could be caused by the size of some of our documents. We store embedded arrays in our photo documents, which can get quite large.

Thanks

You can try setting the log level to debug, and enable the “*” log key.

I’m not sure how much information you’ll get logged about the document body. It certainly won’t log the full JSON body, but the error might give you some further clues as to what’s going on.

@pasin do you remember anything about this issue?

I have tried to search couchbase-lite-ios’s issues, PR, and CBSE tickets but I couldn’t find anything exactly like this. There was one SG issue that was related the size of the request or rev id size (not sure though) that might be related but I couldn’t find the ticket.

@adamf have you recognized any issues that might be related to this?

Found one issue that is related to BAD_JSON but that was fixed in 1.3.


From the Sync Gateway side, it looks like this would only result from malformed JSON being sent to the _bulk_docs request. Have you already checked the SG logs? Looking at the SG 1.5 code, there should be a warning logged like “Couldn’t parse JSON in HTTP request:” that may have additional error information.