Good morning. Having spent 25 years with SQL databases I am new to NoSQL. This question is not directly related to Couchbase but it would be great to receive some advice. The answer to this question will answer a ton of other questions for document design.
Question: If I have a document and I want to “attach” multiple images/files to that document should there be an array of files inside the document OR have another document with the images and a pointer to the original document OR have multiple documents that contain the files with them all pointing to the original document?
Thank you very much.
Brian
Hi @brianjolive,
I will try to answer your question, but it’s really an “it depends” type of situation 
I’m also going to assume you’re asking about Couchbase Server and not Couchbase Mobile.
- Do you mean to store the actual binaries within the documents, or just metadata about them (e.g. file system location, file size, date, etc). If you mean to store binaries, Couchbase Server may not be the best choice for that.
- As you stated, you can model it both ways in JSON. So how you decide to depends on a few factors: is this list going to be relatively unbounded in size? Do you plan to read/write the whole document most of the time, or do you plan to access the individual images? Are you going to use key-based access, N1QL (SQL) queries, full-text search, or some/all of the above to retrieve data?
Thank you for responding. First question is why would Couchbase not be a good choice to store the file? I was thinking of using some of the .NET methods for converting to byte array or similar type to store as a string.
I would plan on having like a description for each file and then retrieving all of the documents that have a certain criteria and displaying just the description and then when clicking on an item it would display the image/file.
However, I am using Couchbase Lite and now that I am learning more about how syncing works that means it would have to pull down all of the images which would not be beneficial.
I guess this is my main question is best practices for syncing with Couchbase Lite. In SQL, I would run a query, display the results in a list and the user chooses an option, then go back out to the database to retrieve the file/image. I am trying to understand using channels or hopefully you can suggest a better method for this?
An example would be if I had a project management app and I wanted to assign a task to someone I want that task to show up on both of our devices so I am trying to understand out how that synchronization would work.
How would I setup, let’s say, to sync if I want all of my tasks that I created and any that are assigned to me?
@brianjolive,
I would say that Couchbase Server is suited best for JSON. This doesn’t mean you can’t store binaries in Couchbase or even encoded in JSON. But individual document size in Couchbase is limited, so we tend to recommend that you store it in the file system instead. That being said, there are certainly customers who do it anyway
As for syncing data, I think you’re missing a big piece of the puzzle here: Couchbase Sync Gateway, which is designed to do exactly what you’re describing: syncing data between Lite and Server. I’ll tag @hod.greeley and @priya.rajagopal on this too: they can handle any questions about syncing and mobile better than I can.
Hi @brianjolive,
How large are the images you’re considering? For large images, we often recommend storing them separately. Here’s a blog post that goes into the considerations: https://blog.couchbase.com/large-objects-in-a-database/
Typically one of the key points with mobile is to give a good experience offline. So, you’ll need to think about the trade offs between pulling all the images down (presumably smaller ones) vs. not having them when network connectivity is poor. Otherwise, you would treat the retrieval and display just as you described with SQL.
Sync Gateway and channels are very powerful for what you want in terms of tasks. We have a To do list type example application that could help: https://github.com/couchbaselabs/mobile-training-todo/tree/feature/2.0 (make sure you’re looking at the feature/2.0 branch, not master, since the 2.0 API has changed significantly).
Channels will do just as you suggest. There’s a lot of flexibility. You could, for example, have two fields in your document, say “owner” and “assigned”. Have each user have a channel. In the sync function, check the owner and assigned fields and assign the document to a user’s channel if they’re listed in either. The easiest is to have everything based off a consistent ID. So the user logs in under that id, it’s used as the channel name, and used in the owner/assigned fields.
The sync function can also be used for validation. Let’s say you want an owner to be able to change anything, but someone to whom the task is assigned to only change a progress field. You would, of course, typically build that into the application itself. But you can also have the sync function reject a document based on the authenticated user at the time of the sync, the previous version of the document, and the new version being presented.
With channels, the on-device database will only see the documents a user has access to, so you won’t be downloading the complete set of images to every device.
A separate remark on design. How you choose to refer to something like image files depends on a number of factors. We often talk about 1:1 vs 1:few vs 1:many. If you have one image with associated information, you’d typically embed it. Same if you have a relatively bounded number. If you have an unknown and possibly ever growing number of items, it’s often better to split the documents. The classic example given is a blog post. You’d typically put the post in one document, and store follow-up comments (assuming you have a popular blog!) in a separate doc or docs. I’ve found this video by Microsoft to be a pretty good introduction to design considerations: https://www.youtube.com/watch?v=IUxT7ZRHlZ4
Thank you very much for all of you super helpful information! I guess that is the catch with mobile devices. If the file is stored on the server file system and the application is just storing the link to the file then if the device is offline the user would have to wait to view the file until they were back online BUT if trying to store the file in something like a BASE64 format within the JSON document you are possibly looking at slow sync. Does that sound about right?
Is there a way with Couchbase Lite to query the data with something like N1Q1. Basically doing a query to pull a JSON document down that does not involve the syncing process?
No, not really. CBL is built around the sync from the cloud, query locally model. You could probably do it by talking to Couchbase Server with one of our SDKs, but I’ve never tried it and it probably won’t work well without a solid connection.
The sync function on Sync Gateway is really intended to give you anything you need to filter and have the device only receive what’s needed. I’m speculating from your question that you’ve seen sync functions are written in Javascript, and you’d prefer SQL/N1QL? It’s an interesting idea. I can pass that on to our product management team.
You are correct. The Javascript sync functions and ASP.NET applications we are testing as well. Basically, the ASP.NET can retrieve a single document (and Couchbase Lite can as well but locally) but I may be thinking too narrowly with this specific requirement with the files. I just come from a heavy SQL background and this concept is just new to me and I am grateful for your help.
Happy to help. For mobile, especially if you expect the mobile user to make changes, the approach needs to be really different from a typical server app. Consistency when you have many writers and potential delays involved has some very tricky edge cases.
@matthew.groves is the one to go to with any ASP.NET questions you might have.