My team is investigating a migration to from Elastic Search(inherited from legacy) to Couchbase as our primary data store. The team is having trouble warping our heads around a few things but the one sticking point that consistently holds us back is how to do separation of domain on documents in CouchBase.
We are in a micro service architecture and would like to effectively have a service own a document and not let other services touch that document. This is for the sake of good separation of domain as well as a fear of someone writing a bad query and affecting data outside of the services domain. In a relational database you could accomplish this with new Databases but it doesn’t look like it would be a good idea to spin up bucket after bucket in CouchBase to mirror this.
One solution that we have started to explore is to use sync gateway for all queries and set users permissions on each document. Something is rubbing me the wrong way about this approach though. It feels weird to put the extra overhead on sync gateway just to enforce separation of domains. It also feels like it could get complicated to manage over time.
Has anyone else run into this issue? Of course, we could be over worried but it seems like a legitimate concern on the face of it.
When you say that “a service owns a document”, do you mean that a document will belong only to a single service for the entire life of the document? Or will a document be sometimes owned by one service, and sometimes by another?
As you note, buckets are quite heavyweight and generally don’t scale past 10. How many micro services do you want to support? If it is 10 or less then you can use buckets. At some future time Couchbase will support vast numbers of lighter weight collections, but not quite yet.
If you will have more than 10 services, one pattern that has been traditionally used is to put all the documents in a single bucket, but to use different document ID prefixes for different types of documents. E.g., have all the documents for “Service 1” have their keys prefixed with “Service_1_”. Then, make sure your queries (and indexes) have an appropriate filter on the key, e.g.:
where subsstring(meta().id,0,10) = “Service_1_”
Of course, it would be still possible for a malicious service to write a query that didn’t include that clause, for the complete solution to separation will come with support for collections (real soon now).
We are currently working on a feature called “collections”. If Couchbase buckets correspond to “databases” in most relational systems, collections will correspond to tables. These collections will allow more fine-grained control over visibility.
Collections will have limited support (developer preview only, not for production) in 6.5, due out in 2019, and will be fully supported across the system in 7.0, due out in 2020.