I would like to ask about the performance implications of having many scopes, collections, and indexes in Couchbase 7.0 Beta.
Our applications initialize their own scope in the target bucket lazily when the Data API is acquired and an operation (K/V or query) is called for a specific JVM class. The Data API also checks for indexes and registers new indexes when required. Moreover, the Data API waits for all scopes/collections/indexes to be online before proceeding with the first operation.
Due to this design, in our tests, we create many scopes/collections/indexes. We observed, that tests would slow down and start to fail, as create/get scope/collections/indexes operations become slower.
In one of our applications, for a single bucket having 100 scopes, 30 collections in each scope, and 60 indexes in each scope turned out to be so detrimental to performance, that create/get scope/collections/indexes required minutes to go through. When an application is getting initialized, many instances of the Data API start to acquire collections/indexes for their own use in parallel.
Why having a lot of indexes/collections (mostly passive ones) turn out to be so bad for performance? Is this performance degradation only appears in Community edition or in Enterprise edition as well?