Hello,
I try to use com.couchbase.spark.japi.CouchbaseSparkContext -> couchbaseView
My Simple program:
SparkConf sparkConf = new SparkConf().
setAppName(“my_app”).
setMaster("local[]").
set(“spark.couchbase.nodes”, “localhost”).
set(“com.couchbase.bucket.myBucket”, “1111”).
set(“com.couchbase.connectTimeout”, String.valueOf(100 * 1000)).
set(“com.couchbase.keepAlive”, String.valueOf(20 1000));
SparkSession spark = SparkSession.builder().config(sparkConf).getOrCreate();
JavaSparkContext jsc = new JavaSparkContext(spark.sparkContext());
CouchbaseSparkContext csc = couchbaseContext(jsc);
long a = csc.couchbaseView(ViewQuery.from(“my_view”,“my_view”).limit(10).reduce(false)).count();
However, it got NullPointerException:
Exception in thread “main” java.lang.NullPointerException
at org.apache.spark.scheduler.DAGScheduler.getShuffleDependencies(DAGScheduler.scala:431)
at org.apache.spark.scheduler.DAGScheduler.getOrCreateParentStages(DAGScheduler.scala:380)
at org.apache.spark.scheduler.DAGScheduler.createResultStage(DAGScheduler.scala:367)
at org.apache.spark.scheduler.DAGScheduler.handleJobSubmitted(DAGScheduler.scala:850)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1677)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1669)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1658)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:630)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2022)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2043)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2062)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2087)
at org.apache.spark.rdd.RDD.count(RDD.scala:1158)
at org.apache.spark.api.java.JavaRDDLike$class.count(JavaRDDLike.scala:455)
at org.apache.spark.api.java.AbstractJavaRDDLike.count(JavaRDDLike.scala:45)
I use the following dependencies:
org.apache.spark
spark-core_2.11
2.2.0
org.apache.spark
spark-sql_2.11
2.2.0
com.couchbase.client
spark-connector_2.11
2.2.0
Thanks!