What about views on HUGE datasets


#1

Howdy y’all!

I’m thinking about using this view:

...

On a dataset of doc like this one:

{...}

But with a few billions of documents like this one.

I’m just thinking about the performances and disk usage of such view? Does anybody has experience with this use-case ?

Any comments?

Thank you!


#2

I have a much more complex view over almost 2M items, no problems. It’s less about the complexity and more about how fast the data changes. Performance is good for either stale queries or if the rate of change is something your system can keep up with.

As far disk space usage is concerned, the storage is snappy compressed, so that should help a little. It doesn’t help a lot with small records though. You should be able to calculate what’s needed from the size of the objects as an upper bound.