Can we store a large video file in couchBase with streaming capability and storing as chunks

Couchbase Server
1

Hi All,

I want to store a large size video file mp4 (1 GB or more) into couchbase DB in chunks and retrieve the video file. Also looking for the streaming of video instead of downloading the entire file.

Can you please help me how to implement this in couchbase.

I am looking similar to “gridfs in MongoDB”

2

Have you considered storing the meta-data which would include the location, id, and access information necessary to retrieve and stream this video in Couchbase yet store and maintain the video file itself in a streaming-optimized cloud-service?

Let Couchbase store and maintain how to fetch the video resource and let an AWS or Azure cloud-based service manage the streaming. Best of both worlds.

JG

3

I agreed with your point @The_Cimmerian

But what is the maximum file size couchbase supports ? As per the below link it says 20 mb, link is little bit old one 2016.

I want to know how we can store a large video file in couchbase.

4

The max files size remains 20 mb. If you are merely storing the meta-data and identifying information in Couchbase and the actual media data in a media-streaming cloud service, the max file size is not an issue.

There are a number of considerations one weighs when modeling data stored in a distributed architecture like Couchbase. A recommended minimum node count in a cluster is 3. This is to establish a quorum to resolve conflicts in the case of simultaneous updates. The 3rd node serves as a tie-breaker when deciding which request is considered “last in” and stored.

At any rate, each document is stored and replicated across the network, node-to-node and memory first. Unnecessarily large data-objects could really hurt your responsiveness and performance.

My recommendation is NOT to store large objects of any kind in Couchbase or any database technology, regardless of whether they will fit or not.

I hope you find this helpful.

JG

5

An old project which might show some of what you’re looking for is in cbfs. It’s not officially supported in any way. IIRC, Couchbase was mainly a metadata store for it, but it has some of the same ideas in chunking files.

6

Was curious about this and tested with some data sets. It appears like couchbase accepts files larger than 20MB and logs a warning with OverThresholdRequestsRecordedEvent tag. At what size will couchbase reject the input?

7

Hi @PShri ,

I’m surprised to hear that. I ran a little test just now and confirmed Couchbase Server 7.0.0 rejects values larger than 20 mebibytes (20,971,520 bytes). Can you double-check the results of your experiment?

The OverThresholdRequestsRecordedEvent message is about some requests taking longer to execute than others; it’s referring to a time threshold, not a size threshold.

Thanks,
David