As the following error message suggests:
Type mismatch: expected value of type object, but got the value of type array
It looks like a “WHERE” statement filter issue. Some field in your filter is expected to be a JSON object but it was a JSON Array in one or more documents. As mentioned earlier, every time that document is encountered in CB Server 6.0.x, it will cause ingestion to restart from a certain point. If your application is expecting this field to be always an object, best thing to do is to fix the source data in the Data Service. The Analytics Service can help you identify data issues by creating a dataset without any filters then running queries similar to this:
SELECT meta().id FROM some_dataset WHERE IS_OBJECT(suspected_field_name) = false;
This will give you the IDs of those documents then you can fix their data in the Data Service. After that, you may create the dataset in Analytics with the proper filter.
Another option would be to add the condition IS_OBJECT(expected_to_object_field_name) at the beginning of your filter to ensure that the field won’t be accessed unless it is of the expected type. However, I don’t recommend this solution unless you can’t fix the source data since it will cause some overhead and depends on the execution order of the conditions in the “WHERE” statement.
You don’t encounter this issue in your tests when your data size is small most likely because your sample data is all well-formed. As mentioned earlier, the behavior of restarting ingestion in such failures has been improved in CB Server 6.5.x and now ingestion will continue and the IDs of documents that fail to be ingested are logged with the cause of the failure.
Hope this helps.