Data source couchbase.kv does not support streamed reading

Spark Connector
1

Hello,

I was looking at the latest documentation of Spark Connector: Spark Structured Streaming Support | Couchbase Docs. When tested the examples on an Azure Databricks environment (Spark 3.2.1, Scala 2.12, com.couchbase.client:spark-connector_2.12:3.2.1), I got the following error when I tried to retrieve a stream from my Couchbase bucket:

UnsupportedOperationException: Data source couchbase.kv does not support streamed reading.

Could you please advise what could be going wrong ?

Thank you very much.

Best regards

2

@Ahmed_KAMAL can you show us the full code you are using? (of course *** out credentials)

3

Yes sure. It is exactly the one put as in example in the link I mentioned:

import com.couchbase.spark._

val spark = org.apache.spark.sql.SparkSession
.builder()
.master(“local[*]”)
.appName(“Structured Streaming Example”)
.config(“spark.couchbase.connectionString”, “”")
.config(“spark.couchbase.username”, "“)
.config(“spark.couchbase.password”, “”")
.config(“spark.couchbase.implicitBucket”, “””)
.getOrCreate()

val sourceDf = spark.readStream.format(“couchbase.kv”).load

4

@Ahmed_KAMAL hm, I’ve seen some funky stuff with databricks recently but this one did not come along yet. Can you try and run it standalone on your dev machine (just spark itself) and see if it works fine? I Want to make sure we figure out if it’s in the connector or within databricks…