For anyone hosting in the cloud (EC2 or Azure), you know that page blob storage and Couchbase aren’t exactly a match made in heaven We have an application that relies heavily on views and with CB 4.0 now released, we only see our index usage increasing over time with SQL for Documents.
Azure provides temporary local storage as both SAS disks and SSD, both of which have much lower latency than page blob. We are very interested in configuring the index path on the temporary storage, but we don’t know how Couchbase will react if the indexes don’t exist after a reboot (which is the case during a dealloc/alloc event on the Azure side).
With the Azure Linux Agent, I can ensure that the appropriate directories and permissions are assigned, and I can even create empty files in that would help. Has anyone tried this, or does anyone know if it’s on the Couchbase development roadmap to make indexes semi-permanent (i.e. auto-rebuild if missing when the couchbase service starts)?
Speaking in terms of probability/uptime, the “temporary” storage really isn’t very “temporary”. It only removes the temp storage upon a de-allocation/allocation event, or hardware failure. Both of which will force a reboot of our couchbase nodes anyway.
If you really wanted to extend this functionality, the couchbase services could have a nightly (scheduled) job that persists the temp index storage to page blob storage. Then during the Couchbase service start-up sequence, if the index files were missing it could automatically restore from persistent copies, saving time by only re-indexing what changed the last backup.
Thoughts? The only other option is premium storage which is great, but considerably more expensive. Storing indexes on temp storage would be a great use of existing resources that you already get for free with each provisioned VM.