Move document from one collection to another

Couchbase Server
1

Hello,

In a bucket I’ve created a lot of documents (in the default/default scope/collection) and now I want to add them into a dedicated scope/collection on the same bucket.

How can I do that ?
I didn’t see anything about this kind of action.

Regards

2

Easiest /fastest way is use eventing function @jon.strabala, @shivani_g

1 Like
3

You can also insert into scope/collection select from default/default.

1 Like
4

Hi @dmihura as @vsr1 indicate Eventing can do what you want in fact you could split up all your “types” into different collections via the eventing function.

A small dataset using N1QL as @shivani_g points out can be quite simple the use of Eventing might be more appropriate if you have 100’s of millions of rows of data and/or don’t want to use indexes and/or don’t have query/index nodes. The eventing solution uses only DCP feeds and KV.

I will show a concrete example because if you are not familiar with Eventing things can be a bit confusion.
If you load the test sample “travel-sample” in ‘travel-sample’._default._default there are 31,591 documents with several types.

Say we want to split up just the airports and routes (type=“airport” and type=“route”) into a new bucket named ‘new-travel-sample’ into the scope ‘split’ collections ‘new_travel-sample’._split.airports and ‘new_travel-sample’._split.routes.

  • Make a new bucket ‘new-travel-sample’

  • Add a scope ‘splits’

  • Add a collection of ‘airports’ to the above scope

  • Add a collection of ‘routes’ to the above scope

  • Make the Eventing Storage of meta. (we will use meta._default._default) if you don’t already have an Eventing metadata bucket/collection storage area.

  • Create and Eventing function via “Add Function”

  1. Set the source to 'travel-sample’._default._default
  2. Eventing storage scratch pad collection meta._default._default
  3. Name our function to something like reorgTS
  4. If you want more performance expand the settings and then up the # workers from 1 to say 8
  5. Under bindings make a bucket alias of airports_col to ‘new_travel-sample’._split.airports in mode r+w
  6. Under bindings make a bucket alias of routes_col to ‘new_travel-sample’._split.routes in mode r+w

  • Hit “Next Add Code”

      function OnUpdate(doc, meta) {
    if (doc.type === "route") {
      // optional
      // delete doc.type;
      // store to our new collection
      routes_col[meta.id] = doc;
    }
    if (doc.type === "airport") {
      // optional
      // delete doc.type;
      // store to our new collection
      airports_col[meta.id] = doc;
    }
  }

  • Save the code

  • Deploy the Function

  • When done (on 31591 success for the deployment stats) you are done undeploy the Function.

You will process 31,591 documents and end up with 24024 items in the ‘new_travel-sample’._split.routes collection and 1968 items in the ‘new_travel-sample’._split.airports collection.

Best

Jon Strabala

3 Likes
5

Hi all,

Thank you for your answers. I’ll check that out and get back to you.

Regards,
David

6

Hi,

Thank you very much! It works well. Currently, I don’t have 100’s of millions of rows, but I tried the @jon.strabala detailed solution as it’s a nice example of how eventing can be used (and it was a good practice). My issue was to keep data in the same bucket, but I managed to do it.

It could be a good example to show in official eventing tutorials.

Thanks all and have a good day :slight_smile:
David

7

Hi @dmihura,

Thanks for the suggestion, this is now part of the Scriptlets in the Eventing docs as ConvertBucketToCollections. Hopefully it will help others with similar needs.

Best

Jon Strabala
Principal Product Manager - Server‌

3 Likes
8

Hi @jon.strabala

Thanks a lot
Regards,