Sync gateway showing android couchbase lite requesting deleted document multiple times

I’m monitoring a replication between our mobile client and couchbase. We have sync gateway setup in front of couchbase to handle versioning, authentication, etc. (sync gateway enterprise 1.0.3, android couchbase lite v1.0.4) I can see in the sync gateway logs that the mobile client is often requesting the same document multiple times, even if the document is marked as deleted by sync gateway. Example from the sync gateway logs:

20:50:34.970522 HTTP:  #12625: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:34.971837 HTTP:  #12626: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:34.972920 HTTP:  #12627: GET /todos/0db7632facb34dcfc89b97dca7145abd?rev=2-993ad5a34a6168cf27cb0968c8438e3a&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.507124 HTTP:  #12628: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.509473 HTTP:  #12629: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.544635 HTTP:  #12630: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.559156 HTTP:  #12632: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.563395 HTTP:  #12631: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.590365 HTTP:  #12633: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.591582 HTTP:  #12634: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.619931 HTTP:  #12635: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.658208 HTTP:  #12636: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.665245 HTTP:  #12637: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.689424 HTTP:  #12638: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.690668 HTTP:  #12639: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.711098 HTTP:  #12640: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.723650 HTTP:  #12641: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)
20:50:35.742823 HTTP:  #12642: GET /todos/b430f711b861041e9c02ddfff08d3562?rev=2-97114b8c462aee1abf236163efd5fb8c&revs=true&attachments=true  (as b31aa5d6-2ac4-4822-b365-7021a8133299)

The document with id b430f711b861041e9c02ddfff08d3562 is marked as deleted though. So my main questions are:

  1. Does couchbase lite remove documents locally that have been sync gateway deleted, and is it supposed to not replicate them down from the server?

  2. Why is couchbase lite requesting the same document and revision multiple times for every document? I have the following SG log flags set: Enabling logging: [CRUD REST+ Shadow Access]
    So if it is requesting, failing, and requesting again just the failure isn’t getting logged please let me know how I can adjust my logging flags to log the failure.

@alexegli

Document revisions that represent a deletion are still replicated to mobile clients, which will hold the revisions locally.

I would recommend that you upgrade to Couchbase Lite and Sync Gateway 1.1 and retest and let us know if the problem persists.

Andy