Hi,
I’m trying to evaluate Couchbase capabilities.
Is Couchbase capable of storing multiple lists, each holds between 100,000-100,000,000 records?
The records are stored in a “data series” fashion (or delayed queue) and queried accordingly.
Example
List dataset structure:
- id
- list_id # the list the record belongs
- next_check timestamp
- status
- some other fields…
Typical use case:
Select All records that have next_check in the past and a specific status.
SELECT * FROM RECORDS
WHERE next_check < now()
AND status = X
limit, offset
Then I can perform several actions:
- Update the record with a new next_check/status values.
- OR delete the record and insert a new one.
Questions
What I’m trying to understand is this:
- If Couchbase can handle such huge dataset
- Huge lists of 100,000 and up to 100,000,000.
- We don’t have a lots of parallel reads, max 10 reads per list (for palatalization).
- Once a record is read, its going to be updated with a new status/next_check or completely deleted.
- What is the best way to store and query such structure?
- and finally, is there any Couchbase limitation I need to pay attention to (i.e don’t use more then 1000 buckets)?
Thanks!