How to store a blob into a document using couchase-lite-core


I have found the answer. I was not formatting the JSON correctly. I had {"_attachments":{“attached”: [ and it should simply be {“attached”: [

Not complaining any more!



To be precise, the problem was that you had the value of _attachments.attached as an array; removing the outermost pair of square brackets would have fixed it.

But the _attachments property is a holdover from CouchDB and has a more strict schema; you should avoid using it, so your current schema is better.


@jens I’m running into a new problem.

I’m able to store a blob into the database. However, it does not seem to be replicated correctly. In the couchbase-server UI I can see a binary document and inside the document data I can see the blob being reference but the length is 0.

When I add the document with the attachment, the replicator breaks with the following error:

18:31:36.947513| [Sync]: {Push#5} Got response for 1 local changes (sequences from 10)
QCBLWebSocket:disconnected "" "/db/_blipsync" QAbstractSocket::UnknownSocketError 
"Unknown error"
18:31:37.291698| [WS] WARNING: {C4SocketImpl#8}==> litecore::repl::C4SocketImpl wss://sg- @0x7ff15d4e21d0
18:31:37.291743| [WS] WARNING: {C4SocketImpl#8} WebSocket closed abnormally with status 1006
18:31:37.300727| [Sync]: {Repl#4} Connection closed with WebSocket status 1006: "Unknown error" (state=2)
18:31:37.301527| [Sync] ERROR: {Repl#4} Got LiteCore error: WebSocket error 1006 "Unknown error"
Replication error occured, aborted "Unknown error"

On the sync gateway side there is this:

sync-gateway_1      | 2019-05-16T17:45:03.008Z [TRC] Bucket: GetRaw("_sync:att:sha1-720tj641+M0XPDyGofv0cNIFbyE=") [363.897µs]
sync-gateway_1      | 2019-05-16T17:45:03.008Z [DBG] Sync+: [590f7ae3]     Asking for attachment "blob_/blob" (digest 
sha1-720tj641+M0XPDyGofv0cNIFbyE=). User:ff4fdde2-02fd-41fd-9386-94842f156c5a
sync-gateway_1      | 2019-05-16T17:45:03.008Z [DBG] WSFrame+: [590f7ae3] Queued MSG#3
sync-gateway_1      | 2019-05-16T17:45:03.008Z [DBG] WS+: [590f7ae3] Push MSG#3
sync-gateway_1      | 2019-05-16T17:45:03.008Z [DBG] WS+: [590f7ae3] Sending frame: MSG#3 (flags=       0, size=   78)
sync-gateway_1      | 2019-05-16T17:45:03.153Z [DBG] WS+: [590f7ae3] Received frame: RPY#3~ (flags= 1001001, 
sync-gateway_1      | 2019-05-16T17:45:03.153Z [INF] WS: [590f7ae3] ERROR decompressing frame: inputLen=4090, 
remaining=0, output=1, error=unexpected EOF
sync-gateway_1      | 2019-05-16T17:45:03.154Z [INF] WS: [590f7ae3] Error decompressing frame RPY#3~: unexpected 
EOF. Raw frame = <03496200000000ffff1c5a0554dc5ad74d3233998cbb6383438152da525aea46dd5fdddddddd85fa7b757737ead485b6144a694b711d60dc7d2693497ebe7fdd7557b2e

At this point, the websocket closes and everything gets corrupted.

I’m wondering if this is the socket that is failing to transmit the data that is asked.

My interpretation is that sync-gateway is asking the blob, the blob data is sent, the data is corrupted and it fails.

Any ideas? I’ve been here for hours without any luck.


Best regards,



@borrrden @jens

Yesterday I have spent 8 hours trying to understand why the replicator socket breaks when sync gateway asks for a attachment that I add to a document. I couldn’t figure it out.

To ensure that the problem was not from my side I have made several things:

  • Tried another branch from couchbase-lite-core (2.5.0) - No success!
  • Tried to disable the SSL websocket layer using civetwebsocketfactory - No success!
  • Tried to isolate the operation in a isolated project - No success!

Everytime I add an attachment to a document, it is successfully inserted in the local database but not correctly uploaded to the couchbase server, sync-gateway crashes upon requesting the attachment. The feedback is always the same: Error decompressing frame RPY#3~: unexpected
and connection is closed by peer .

Do you have any insights of what might be happening? I’m sure this is not a couchbase-lite-core problem but something I’m doing wrong. I just can’t seem to find…


Best regards,


Are you able to reproduce this in a clean project or create a repro case to show? The SG logs and your other logs don’t match up with the digests and you said you changed the way you are storing them into the documents so it’s likely something else is going on.

  • Do the C4Tests pass on your system?
  • Can you reproduce the problem without introducing unknown variants into the equation? (QT CBL wrapper)
  • Can you write a project that any of us can quickly run and verify?

@borrrden I will try to isolate everything in a C program. I will get back here when I have it.

Can you tell me how to call C4Tests?



@borrrden test results:

09:29:19.237612| [DB]: Deleting database file /private/var/folders/bw/lynl848s6dg1rfpvsv0frh9m0000gn/T/Litecore_C_Tests/cbl_core_test.cblite2/db.sqlite3 (with -wal and -shm)

All tests passed (392017 assertions in 113 test cases)



I have created a public repo with a test project:

It has already the code to add an attachment together with the document.

but curiously I can’t even create a document and sync it to the db. I can’t find the reason why…

Sync gateway is currently public and allowing guest connections.

What am I missing here?

I have been morning to write this example…



There are two errors in your test project:

  1. You forgot to encode the JSON (c4db_encodeJSON) before you tried to save the document
  2. Your JSON is not good because the entire “blob” dictionary is in quotes. Furthermore, the length is also in quotes.

@borrrden thanks for your valuable insights. You were completely right. I have updated the program on github.

I’m now able to set the document on the database but for some reason the attachment doesn’t seem to be recognized because the the property _attachments is empty and I can’t see binary data on the database.

I have triple checked the JSON format and everything appears to be correct. Blocked again… :confused:

Thank you very much for your help!

Best regards,



@borrrden I discovered the reason for the attachment not being uploaded. It was a copy/paste typo on the digest. It was not being dynamically set.

I ended up with the problem I was describing earlier. There is the Error decompressing frame error unexpected EOF followed by the dump of the raw frame.

Although the document is created on the database, the digest from the attachment doesn’t match and the length is 0:

attachments": {
/attached/0”: {
“content_type”: “text/plain”,
"digest": “sha1-2jmj7l5rSw0yVb/vlWAYkK/YBwk=”,
"length": 0,
“revpos”: 1,
“stub”: true
“attached”: [
@type”: “blob”,
“content_type”: “text/plain”,
"digest": “sha1-iypXuV4DbskNO4roX+oGsR+fcik=”,
"length": 95862

I have updated the test again:

I don’t have Qt in the middle now.

Thanks in advance!




The error in question means that for some reason the content of the frame is not being properly compressed on the client side, or is failing to send the entire thing. I’m not that familiar with this area but maybe @jens has an idea of what is going on. It could be something we take for granted in Lite that I’m not remembering at the moment or it could be some flaw in CivetWebSocket since we don’t use it for actual SG connections in production.


Furthermore I am not able to reproduce this locally. I started up a Sync Gateway on my own machine with several different configurations and versions and your test program worked fine. What is the content of your sync gateway config file?


@borrrden awkward that it doesn’t fail on your side. I hope this is a simple fix because it is preventing me from moving forward… :frowning:

On the client side I have been using the release/iridium branch

My sync-gateway config file is like this:

"logging": {
	"console": {
		"log_level": "debug",
		"log_keys": ["*"]
"interface": ":4984",
"databases": {
	"db": {
		"server": "http://couchbase-server:8091",
		"bucket": "couchbase_qt",
		"username": "DB_USER_NAME",
		"password": "DR_USER_PASSWORD",
		"enable_shared_bucket_access": true,
		"import_docs": true,
		"num_index_replicas": 0,
		"users": {
			"GUEST": {"disabled": false, "admin_channels": ["*"] }

Sync-gateway Dockerfile is like this:

FROM couchbase/sync-gateway:2.1.2-community

COPY config.json /etc/sync_gateway/config.json

And docker-compose is like this:

  image: couchbase
  restart: always
    - traefik.backend=cb-dev
    - traefik.enable=true
    - traefik.frontend.entryPoints=http
    - traefik.port=8091
  - couchbase-data:/opt/couchbase/var

  build: sync-gateway
    - traefik.backend=sg-dev
    - traefik.enable=true
    - traefik.frontend.entryPoints=http
    - traefik.port=4984
    - 4985:4985

This sounds like a problem in the compression part of the BLIP protocol we use in the replicator (it’s a layer on top of WebSockets), but it’s unclear whether the problem is on the LiteCore or the SG side. It would be great to find a reproducible case. Could you please file an issue against LiteCore, including a link to your test code?



I have done that this afternoon. Check couchbase-lite-core issues.



@jens @borrrden

I have found the culprit…

Since Jim said it didn’t had any problems running my example locally I decided to do the same. It worked.

So then I have been debugging my whole network architecture do find where the problem laid. It seems that the culprit is Traefik.

I have Traefik to allow me to expose certain urls to the network rather than access via IP address. It seems that Traefik is not handling this case very well. And it seems I’m not the only one complaining:

I’m currently using v1.7 of Traefik and using the latest version doesn’t fix the problem. I didn’t try version 2.0 because it has breaking changes and I don’t have time to learn about it right now.

In the mean time, I just wanted to confirm that the bug isn’t in couchbase-lite-core side. I will move forward with my work but I need to find a way of exposing sync-gateway url in a proper way. Any suggestions? Can you ask around inside your team if they know anything about alternatives to do this with Traefik or another kind of proxy service?


Best regards,



@sinosoidal Thanks for getting back and confirming that this is not and SG issue!

I don’t know that anyone here is familiar with Traefik. I believe that most of our customers just expose the IP address and close down any unnecessary ports. If you can say more about what you mean by “exposing the url in a proper way”, I can ask around.


@blake.meike sorry if I’m using using the proper language to describe this.

Traefik is a router. Look at this docker-compose snippet to see how it works:

 build: sync-gateway
   - traefik.backend=sg-dev
   - traefik.enable=true
   - traefik.frontend.entryPoints=http
   - traefik.port=4984 

With Traefik I can say any request to the url will be forwarded to the docker service sync-gateway at port 4984. This allows me to expose only port 80. Even better is that I can than use Cloudflare to serve that url as https and have a SSL communication. This was working perfectly for documents only. For some reason, it is not working with attachments upload. The websocket is closed with the message:

Error when copying from client to backend: websocket: close 1006

Who’s fault? I don’t know. Uncharted terrain for me! The fact is that without Traefik, it works as it should be.

Maybe around your team of web developers, full stack developers, backend developers and docker wizards, someone knows alternatives to Traefik or even know about this problem that I’m having in details.

Thank you very much,

Best regards,



Yeah… I figured that.

This is going to depend on the specifics of Traefik. In order to hold a websocket connection open, a router has to do some clever stateful trickery. I’m not surprised to hear that it fails sometimes.

Lemme ask around.