Couchbase Cluster Connectivity is not getting established and getting com.couchbase.client.core.error.InvalidArgumentException error

rishabh_katyal · July 8, 2022, 2:37am

Hello,

I am trying to connect to the Couchbase Cluster

public class SalesCouchbaseDao {

    @Autowired
    @Qualifier("salesBucket")
    private Bucket bucket;

    @Resource
    private Cluster cluster;

My Couchbase Config Class is as below.

@Configuration
public class CouchbaseConfig {

    private static final Logger LOG = LoggerFactory.getLogger(CouchbaseConfig.class);

    @Autowired
    private CipherHelper cipherHelper;

    /** Common */
    @Value("${couchbase.cluster.host}")
    private String host;

    /** sales bucket */
    @Value("${couchbase.cluster.bucket.item}")
    private String salesBucket;

    @Value("${couchbase.cluster.query.timeout}")
    private Long queryTimeout;
    @Value("${couchbase.cluster.view.timeout}")
    private Long viewTimeout;

    @Value("${couchbase.cluster.username}")
    private String clusterUserName;

    @Value("${couchbase.cluster.password}")
    private String clusterPassword;

    /**
     * couchbaseCluster.
     *
     * @throws BadPaddingException
     * @throws IllegalBlockSizeException
     * @throws NoSuchPaddingException
     * @throws NoSuchAlgorithmException
     * @throws InvalidKeyException
     */
    @Bean
    protected ClusterEnvironment getEnvironment() {
        return ClusterEnvironment
                .builder()
                .timeoutConfig(TimeoutConfig.builder()
                        .queryTimeout(Duration.ofMillis(TimeUnit.SECONDS.toMillis(queryTimeout)))
                        .viewTimeout(Duration.ofMillis(TimeUnit.SECONDS.toMillis(viewTimeout))))
                .build();
    }

    @Bean
    public Cluster couchbaseCluster(final ClusterEnvironment clusterEnvironment)
            throws InvalidKeyException, NoSuchAlgorithmException, NoSuchPaddingException, IllegalBlockSizeException, BadPaddingException {
        return Cluster.connect(host, ClusterOptions
                .clusterOptions(clusterUserName.trim(), cipherHelper.decrypt(clusterPassword.trim()))
                .environment(clusterEnvironment));
    }

    @Bean
    @Qualifier(value = "salesBucket")
    public Bucket saleBucket() {
        try {
            Cluster couchbaseCluster = couchbaseCluster(getEnvironment());
            return this.getBucket(couchbaseCluster, saleBucket);
        } catch (InvalidKeyException | NoSuchAlgorithmException | NoSuchPaddingException
                 | IllegalBlockSizeException | BadPaddingException e) {
            throw new BeanInitializationException("Failed to init data source.", e);
        } catch (Exception ex) {
            LOG.warn("Unable to get connection to couchbase sales buckets", ex);
        }
        return null;
    }

    private Bucket getBucket(final Cluster couchbaseCluster, final String bucketName) {
        Bucket bucket = couchbaseCluster.bucket(bucketName);
        bucket.waitUntilReady(Duration.ofSeconds(10));
        return bucket;
    }

The CouchbaseConfig class is in a separate module and the SalesCouchbaseDao is in a separate module

I am getting below exception when i am trying to connect to the cluster.

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘couchbaseCluster’ defined in class path resource [as/ds/fs/as/fd/CouchbaseConfig.class.

Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.couchbase.client.java.Cluster]: Factory method ‘couchbaseCluster’ threw exception; nested exception is com.couchbase.client.core.error.InvalidArgumentException: Specifying 8091 or 18091 in the connection string “stg-xxxx-xxx-xxx-xx.xxxx.xxx.xxxx:8091,xxx-xxxxxxx.xx-xx-xx-xx.xx.xx.xx:8091” is likely not what you want (it would connect to key/value via the management port which does not work). Please omit the port and use “xx-xxx.xx-xxxx-xxx-xxx.xx.xx,xx.xx-xx-xx-xx.xx.xx.xx” instead.

How can i fix this issue ?

daschl · July 8, 2022, 5:14am

Well, the error message you copied in here already provides one clue. What is the “host” you pass in, and it seems to contain :8091 … please omit that as a starting point.

rishabh_katyal · July 8, 2022, 5:33am

There is a String array of host. I will try after removing the port

rishabh_katyal · July 8, 2022, 7:42am

@daschl I was able to connect to the couchbase cluster after removing the port . However, i am not able to connect to the bucket. Below is the code.

    @Qualifier(value = "salesBucket")
    public Bucket salesBucket() {
        try {
            Cluster couchbaseCluster = couchbaseCluster(getEnvironment());
            return this.getBucket(couchbaseCluster, salesBucket);
        } catch (InvalidKeyException | NoSuchAlgorithmException | NoSuchPaddingException
                 | IllegalBlockSizeException | BadPaddingException e) {
            throw new BeanInitializationException("Failed to init data source.", e);
        } catch (Exception ex) {
            LOG.warn("Unable to get connection to couchbase salesBucket buckets", ex);
        }
        return null;
    }

To get bucket

    private Bucket getBucket(final Cluster couchbaseCluster, final String bucketName) {
        Bucket bucket = couchbaseCluster.bucket(bucketName);
        bucket.waitUntilReady(Duration.ofSeconds(60));
        return bucket;
    }

I initially tried the bucket.waitUntilReady for a duration of 10 seconds and then 60 seconds but still its getting timed out

I am getting below error:

15:19:56.741 [main] WARN j.c.r.a.r.b.t.config.CouchbaseConfig - Unable to get connection to couchbase salesBucket buckets
com.couchbase.client.core.error.UnambiguousTimeoutException: WaitUntilReady timed out {“bucket”:“xxx-sales”,“checkedServices”:[“KV”,“MANAGER”,“VIEWS”,“QUERY”],“currentState”:“ONLINE”,“desiredState”:“ONLINE”,“services”:{“kv”:[{“last_activity_us”:2434081,“namespace”:“xxx-sales”,“state”:“connected”,“id”:“0x6b13911b”,“remote”:“xxx-xxxxxxxxx101.xx-xxx-xxx-xxx.xxxxx.xxxx.xxxxxxx:11210”,“local”:“xxx.xx.xx.xxx:xxxxx”},{“last_activity_us”:3466,“state”:“connected”,“id”:“0x0bed7d2b”,“remote”:“xxx-xxxxxxxxx101.xx-xxx-xxx-xxx.xxxxx.xxxx.xxxxxxx:11210”,“local”:“xxx.xx.xx.xxx:xxxxx”},{“last_activity_us”:2430072,“namespace”:“xxx-sales”,“state”:“connected”,“id”:“0xeb9737d9”,“remote”:"xxx-xxxxxxxxx102.xx-xxx-xxx-xxx.xxxxx.xxxx.xxxxxxx:11210,“local”:“100.67.68.148:38826”},{“last_activity_us”:2399,“state”:“connected”,“id”:“0x60f6b769”,“remote”:“xxx-xxxxxxxxx102.xx-xxx-xxx-xxx.xxxxx.xxxx.xxxxxxx:11210”,“local”:“100.67.68.148:38820”}]},“state”:{“current_stage”:“CONFIG_LOAD”,“current_stage_since_ms”:60026,“timings_ms”:{},“total_ms”:60026},“timeoutMs”:60000}
at com.couchbase.client.java.AsyncUtils.block(AsyncUtils.java:51)
at com.couchbase.client.java.Bucket.waitUntilReady(Bucket.java:229)
at xx.xx.xxxxxxx.xxxxx.xxx.batch.template.config.CouchbaseConfig.getBucket(CouchbaseConfig.java:161)
at xx.xx.xxxxxxx.xxxxxxx.xxx.batch.template.config.CouchbaseConfig.salesBucket(CouchbaseConfig.java:149)

daschl · July 8, 2022, 8:07am

@rishabh_katyal can you share the full logs somewhere, ideally debug? You need to make sure that 1) the bucket exists with the name and also 2) your user has the correct permissions to access it.

Looking closer, it looks like config loading timed out at 60 seconds, which can also point to a network issue. Make sure that the client has network access to all needed ports against all servers of the cluster.

rishabh_katyal · July 8, 2022, 8:15am

@daschl

The bucket name is correct.
Even the User is having the correct permissions to access it.

I will try to see if i can get debug logs

daschl · July 8, 2022, 8:16am

@rishabh_katyal please double check the network access for the ports as well, this looks like a timeout/network issue.

rishabh_katyal · July 8, 2022, 8:23am

@daschl Sure, I will . The port being used is 8091. Will try to see the network access to the ports as well.
Thanks.

rishabh_katyal · July 8, 2022, 8:27am

Actually, i can see in logs that the cluster nodes are getting connected.

15:18:56.767 [cb-events] INFO com.couchbase.node - [com.couchbase.node][NodeConnectedEvent] Node connected {“coreId”:“0x6bcf2e7800000001”,“managerPort”:“8091”,“remote”:“xxx-xxxxxxxxxxxx.xx-xxx-xxx-xxx.xxxx.dcnw.xxxxxxx”}
15:18:56.767 [cb-events] INFO com.couchbase.node - [com.couchbase.node][NodeConnectedEvent] Node connected {“coreId”:“0x6bcf2e7800000001”,“managerPort”:“8091”,“remote”:“xxx-xxxxxxxxxxxx.xx-cps-xx-xxx.xxxx.xxx.xxxxxxx”}
15:18:57.069 [cb-events] INFO com.couchbase.node - [com.couchbase.node][NodeConnectedEvent] Node connected {“coreId”:“0x6bcf2e7800000001”,“managerPort”:“8091”,“remote”:“xxx-xxxxxxxxxxxx.xx-cps-xx-xxx.xxxx.xxxx.xxxxxxx”}

daschl · July 8, 2022, 10:18am

@rishabh_katyal those events are only logical connect events, there is more for each individual service in the logs.

rishabh_katyal · July 11, 2022, 1:34am

@daschl Hello.

I got some logs.

2022/07/11 10:04:36 [INFO] [START] SalesSyncJob
2022/07/11 10:04:36 [INFO] [SalesSyncJob] START JOB (job parameters : [  ])
10:04:39.216 [main] INFO  j.c.r.a.r.b.f.xxxBatchApplication - The following profiles are active: stg
10:04:42.213 [main] DEBUG c.c.c.c.d.i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
10:04:42.219 [main] DEBUG c.c.c.c.d.i.n.u.ResourceLeakDetector - -Dcom.couchbase.client.core.deps.io.netty.leakDetection.level: simple
10:04:42.219 [main] DEBUG c.c.c.c.d.i.n.u.ResourceLeakDetector - -Dcom.couchbase.client.core.deps.io.netty.leakDetection.targetRecords: 4
10:04:42.223 [main] DEBUG c.c.c.c.d.i.n.u.ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: com.couchbase.client.core.deps.io.netty.util.ResourceLeakDetector@173511ff
10:04:42.248 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - -Dio.netty.noUnsafe: false
10:04:42.248 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - Java version: 8
10:04:42.249 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
10:04:42.250 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
10:04:42.251 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - java.nio.Buffer.address: available
10:04:42.251 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - direct buffer constructor: available
10:04:42.252 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - java.nio.Bits.unaligned: available, true
10:04:42.252 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9
10:04:42.252 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent0 - java.nio.DirectByteBuffer.<init>(long, int): available
10:04:42.252 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent - sun.misc.Unsafe: available
10:04:42.253 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
10:04:42.253 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
10:04:42.254 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent - -Dio.netty.maxDirectMemory: 477626368 bytes
10:04:42.254 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1
10:04:42.255 [main] DEBUG c.c.c.c.d.i.n.u.i.CleanerJava6 - java.nio.ByteBuffer.cleaner(): available
10:04:42.256 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent - -Dio.netty.noPreferDirect: false
10:04:42.268 [main] DEBUG c.c.c.c.d.i.n.u.i.PlatformDependent - com.couchbase.client.core.deps.org.jctools-core.MpscChunkedArrayQueue: available
10:04:42.288 [main] DEBUG c.c.c.c.d.i.n.u.i.NativeLibraryLoader - -Dio.netty.native.workdir: /tmp (io.netty.tmpdir)
10:04:42.289 [main] DEBUG c.c.c.c.d.i.n.u.i.NativeLibraryLoader - -Dio.netty.native.deleteLibAfterLoading: true
10:04:42.289 [main] DEBUG c.c.c.c.d.i.n.u.i.NativeLibraryLoader - -Dio.netty.native.tryPatchShadedId: true
10:04:42.298 [main] DEBUG c.c.c.c.d.i.n.u.i.NativeLibraryLoader - Successfully loaded the library /tmp/libcom_couchbase_client_core_deps_netty_transport_native_epoll_x86_648814985619126817114.so
10:04:42.314 [main] DEBUG c.c.c.c.deps.io.netty.util.NetUtil - -Djava.net.preferIPv4Stack: false
10:04:42.315 [main] DEBUG c.c.c.c.deps.io.netty.util.NetUtil - -Djava.net.preferIPv6Addresses: false
10:04:42.321 [main] DEBUG c.c.c.c.d.i.n.u.NetUtilInitializations - Loopback interface: lo (lo, 127.0.0.1)
10:04:42.323 [main] DEBUG c.c.c.c.deps.io.netty.util.NetUtil - /proc/sys/net/core/somaxconn: 128
10:04:42.331 [main] DEBUG c.c.c.c.d.i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 16
10:04:42.348 [main] DEBUG c.c.c.c.d.i.n.u.i.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
10:04:42.348 [main] DEBUG c.c.c.c.d.i.n.u.i.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
10:04:42.909 [main] DEBUG c.c.c.c.d.i.n.c.DefaultChannelId - -Dio.netty.processId: 8523 (auto-detected)
10:04:42.911 [main] DEBUG c.c.c.c.d.i.n.c.DefaultChannelId - -Dio.netty.machineId: 00:50:56:ff:fe:a5:71:2e (auto-detected)
10:04:42.953 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas: 4
10:04:42.953 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.numDirectArenas: 4
10:04:42.953 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192
10:04:42.953 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 11
10:04:42.954 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize: 16777216
10:04:42.954 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.smallCacheSize: 256
10:04:42.954 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.normalCacheSize: 64
10:04:42.954 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
10:04:42.954 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimInterval: 8192
10:04:42.954 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimIntervalMillis: 0
10:04:42.954 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.useCacheForAllThreads: true
10:04:42.954 [main] DEBUG c.c.c.c.d.i.n.b.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
10:04:42.966 [main] DEBUG c.c.c.c.d.i.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: pooled
10:04:42.966 [main] DEBUG c.c.c.c.d.i.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 0
10:04:42.966 [main] DEBUG c.c.c.c.d.i.netty.buffer.ByteBufUtil - -Dio.netty.maxThreadLocalCharBufferSize: 16384
10:04:43.074 [cb-io-kv-5-2] DEBUG c.c.c.c.deps.io.netty.util.Recycler - -Dio.netty.recycler.maxCapacityPerThread: 4096
10:04:43.074 [cb-io-kv-5-2] DEBUG c.c.c.c.deps.io.netty.util.Recycler - -Dio.netty.recycler.maxSharedCapacityFactor: 2
10:04:43.074 [cb-io-kv-5-2] DEBUG c.c.c.c.deps.io.netty.util.Recycler - -Dio.netty.recycler.linkCapacity: 16
10:04:43.074 [cb-io-kv-5-2] DEBUG c.c.c.c.deps.io.netty.util.Recycler - -Dio.netty.recycler.ratio: 8
10:04:43.074 [cb-io-kv-5-2] DEBUG c.c.c.c.deps.io.netty.util.Recycler - -Dio.netty.recycler.delayedQueue.ratio: 8
10:04:43.096 [cb-io-kv-5-2] DEBUG c.c.c.c.d.i.n.buffer.AbstractByteBuf - -Dcom.couchbase.client.core.deps.io.netty.buffer.checkAccessible: true
10:04:43.096 [cb-io-kv-5-2] DEBUG c.c.c.c.d.i.n.buffer.AbstractByteBuf - -Dcom.couchbase.client.core.deps.io.netty.buffer.checkBounds: true
10:04:43.096 [cb-io-kv-5-2] DEBUG c.c.c.c.d.i.n.u.ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: com.couchbase.client.core.deps.io.netty.util.ResourceLeakDetector@76a76380
10:04:43.199 [cb-events] INFO  com.couchbase.core - [com.couchbase.core][CoreCreatedEvent] {"clientVersion":"3.2.4","clientGitHash":"${buildNumber}","coreVersion":"2.2.4","coreGitHash":"${buildNumber}","u
serAgent":"couchbase-java/3.2.4 (Linux 3.10.0-693.17.1.el7.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_144-b01)","maxNumRequestsInRetry":32768,"ioEnvironment":{"nativeIoEnabled":true,"eventLoopT
hreadCount":4,"eventLoopGroups":["EpollEventLoopGroup"]},"ioConfig":{"captureTraffic":[],"mutationTokensEnabled":true,"networkResolution":"auto","dnsSrvEnabled":true,"tcpKeepAlivesEnabled":true,"tcpKeepAl
iveTimeMs":60000,"configPollIntervalMs":2500,"kvCircuitBreakerConfig":"disabled","queryCircuitBreakerConfig":"disabled","viewCircuitBreakerConfig":"disabled","searchCircuitBreakerConfig":"disabled","analy
ticsCircuitBreakerConfig":"disabled","managerCircuitBreakerConfig":"disabled","eventingCircuitBreakerConfig":"disabled","numKvConnections":1,"maxHttpConnections":12,"idleHttpConnectionTimeoutMs":4500,"con
figIdleRedialTimeoutMs":300000,"memcachedHashingStrategy":"StandardMemcachedHashingStrategy"},"compressionConfig":{"enabled":true,"minRatio":0.83,"minSize":32},"securityConfig":{"tlsEnabled":false,"native
TlsEnabled":true,"hostnameVerificationEnabled":true,"hasTrustCertificates":false,"trustManagerFactory":null,"ciphers":[]},"timeoutConfig":{"kvMs":2500,"kvDurableMs":10000,"managementMs":75000,"queryMs":36
00000,"viewMs":360000,"searchMs":75000,"analyticsMs":75000,"connectMs":10000,"disconnectMs":10000,"eventingMs":75000},"loggerConfig":{"customLogger":null,"fallbackToConsole":false,"consoleLogLevel":{"name
":"INFO","resourceBundleName":"sun.util.logging.resources.logging","localizedName":"INFO"},"disableSlf4j":false,"loggerName":"CouchbaseLogger","diagnosticContextEnabled":false},"orphanReporterConfig":{"em
itIntervalMs":10000,"sampleSize":10,"queueLength":1024,"enabled":true},"thresholdLoggingTracerConfig":{"enabled":true,"emitIntervalMs":10000,"sampleSize":10,"queueLength":1024,"kvThresholdMs":500,"queryTh
resholdMs":1000,"searchThresholdMs":1000,"analyticsThresholdMs":1000,"viewThresholdMs":1000},"loggingMeterConfig":{"enabled":true,"emitIntervalMs":600000},"retryStrategy":"BestEffortRetryStrategy","reques
tTracer":"ThresholdLoggingTracer","meter":"LoggingMeter","numRequestCallbacks":0} {"coreId":"0xc1c963b100000001","seedNodes":[{"address":"xxx-xxxxxxxxxxxx.xx-xxx-xxx-xxx-xxxxx.xxxx.xxxxxxx"},{"address":"x
xx-xxxxxxxxxxxx.xx-xxx-xxx-xxx.xxxxx.xxxx.xxxxxxx"}]}
10:04:43.204 [cb-events] INFO  com.couchbase.node - [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xc1c963b100000001","managerPort":"8091","remote":"SalesSyncStep"}
10:04:43.204 [cb-events] INFO  com.couchbase.node - [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xc1c963b100000001","managerPort":"8091","remote":"xxx-xxxxxxxxxxx.xx-xx-xxx-xxx.xx
e2b.dcnw.xxxxxxx"}
10:04:43.505 [cb-events] INFO  com.couchbase.node - [com.couchbase.node][NodeConnectedEvent] Node connected {"coreId":"0xc1c963b100000001","managerPort":"8091","remote":"xxx-xxxxxxxxxxx.xx-xxx-xxx-xxx.xx
xxx.xxxx.xxxxxxx"}
10:04:43.505 [cb-events] INFO  com.couchbase.core - [com.couchbase.core][BucketOpenedEvent][396ms] Opened bucket "xxx-sales" {"coreId":"0xc1c963b100000001"}
10:04:53.178 [main] WARN  j.c.r.a.r.b.t.config.CouchbaseConfig - Unable to get connection to couchbase salesBucket buckets
com.couchbase.client.core.error.UnambiguousTimeoutException: WaitUntilReady timed out {"bucket":"xxx-sales","checkedServices":["MANAGER","KV","QUERY","VIEWS"],"currentState":"ONLINE","desiredState":"ON
LINE","services":{"kv":[{"last_activity_us":409412,"namespace":"xxx-sales","state":"connected","id":"0x70ed3a87","remote":"xxx-xxxxxxxxxxxx.xx-xxx-xxx-xxx.xxxxx.xxxx.xxxxxxx:xxxxx","local":"100.67.68.1
48:53624"},{"last_activity_us":14245,"state":"connected","id":"0x51a5b2fa","remote":"xxx-xxxxxxxxxxxx.xx-xxx-xxx-xxx.xxxxx.xxxx.xxxxxxx:xxxxx","local":"100.67.68.148:53618"},{"last_activity_us":415352,"na
mespace":"xxx-sales","state":"connected","id":"0xb3bc952d","remote":"xxx-xxxxxxxxxxxx.xx-xxx-xxx-xxx-xxxxx.xxxx.xxxxxxx:xxxxx","local":"100.67.68.148:44546"},{"last_activity_us":440,"state":"connected"
,"id":"0x8fb65d8d","remote":"xxx-xxxxxxxxxxxx.xx-xxx-xxx-xxx-xxxxx.xxxx.xxxxxxx:xxxxx","local":"100.67.68.148:44542"}]},"state":{"current_stage":"CONFIG_LOAD","current_stage_since_ms":10064,"timings_ms":{
},"total_ms":10064},"timeoutMs":10000}
    at com.couchbase.client.java.AsyncUtils.block(AsyncUtils.java:51)
    at com.couchbase.client.java.Bucket.waitUntilReady(Bucket.java:229)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.template.config.CouchbaseConfig.getBucket(CouchbaseConfig.java:161)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.template.config.CouchbaseConfig.salesBucket(CouchbaseConfig.java:149)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.template.config.CouchbaseConfig$$EnhancerBySpringCGLIB$$541b2b0a.CGLIB$salesBucket$1(<generated>)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.template.config.CouchbaseConfig$$EnhancerBySpringCGLIB$$541b2b0a$$FastClassBySpringCGLIB$$f33749e2.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.template.config.CouchbaseConfig$$EnhancerBySpringCGLIB$$541b2b0a.salesBucket(<generated>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1181)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1075)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1138)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1066)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:585)
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1272)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.template.xxxBatchApplicationBase.run(xxxBatchApplicationBase.java:37)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.functions.xxxBatchApplication.main(xxxBatchApplication.java:31)
    Suppressed: java.lang.Exception: The above exception was originally thrown by another thread at the following location.
        at com.couchbase.client.core.diagnostics.WaitUntilReadyHelper.lambda$waitUntilReady$12(WaitUntilReadyHelper.java:179)
        at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
        at reactor.core.publisher.Mono.subscribe(Mono.java:4400)
        at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.handleTimeout(FluxTimeout.java:301)
        at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.doTimeout(FluxTimeout.java:280)
        at reactor.core.publisher.FluxTimeout$TimeoutTimeoutSubscriber.onNext(FluxTimeout.java:419)
        at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)
        at reactor.core.publisher.MonoDelay$MonoDelayRunnable.propagateDelay(MonoDelay.java:271)
        at reactor.core.publisher.MonoDelay$MonoDelayRunnable.run(MonoDelay.java:286)
        at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)
        at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
10:05:14.157 [main] INFO  com.zaxxer.hikari.HikariDataSource - xxx_batch_connection_pool - Started.
10:05:14.527 [main] INFO  com.zaxxer.hikari.HikariDataSource - pr_crdr_connection_pool - Started.
10:05:17.154 [main] INFO  com.amazonaws.http.AmazonHttpClient - Configuring Proxy. Proxy Host: dev-proxy.db.xxxxxxx.co.xx Proxy Port: 9501
10:05:17.352 [main] INFO  com.amazonaws.http.AmazonHttpClient - Configuring Proxy. Proxy Host: dev-proxy.db.xxxxxxx.co.xx Proxy Port: 9501
10:05:19.635 [main] WARN  o.s.b.c.l.AbstractListenerFactoryBean - org.springframework.batch.sales.SalesWriter is an interface.  The implementing class will not be queried for annotation based listener confi
gurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
10:05:19.636 [main] WARN  o.s.b.c.l.AbstractListenerFactoryBean - org.springframework.batch.sales.SalesProcessor is an interface.  The implementing class will not be queried for annotation based listener co
nfigurations.  If using @StepScope on a @Bean method, be sure to return the implementing class so listener annotations can be used.
10:05:22.672 [main] INFO  o.s.b.c.l.support.SimpleJobLauncher - No TaskExecutor has been set, defaulting to synchronous executor.
10:05:22.784 [main] INFO  o.s.b.c.l.support.SimpleJobLauncher - Job: [SimpleJob: [name=SalesSyncJob]] launched with the following parameters: [{-logging.path=/usr/local/proad/xxx/xxx-batch-data-d
istribution/current/logs, -spring.profiles.active=stg, run.id=1, -spring.batch.job.names=SalesSyncJob}]
10:05:22.808 [main] INFO  o.s.batch.core.job.SimpleStepHandler - Executing step: [SalesSyncAdFixedDateStep]
10:05:22.912 [main] INFO  j.c.r.a.r.b.c.t.AbstractOperationDateTasklet - operation date = 2022-07-10 00:00:00.0
10:05:22.928 [main] INFO  o.s.batch.core.job.SimpleStepHandler - Executing step: [SalesSyncStep]
10:05:23.859 [main] INFO  j.c.r.a.r.b.f.r.SalesSyncReader - Get sales all  [ size = 23667 ]
10:05:23.859 [main] INFO  j.c.r.a.r.b.f.r.SalesSyncReader - Summarized client [ size = 355 ]
10:05:27.916 [main] INFO  j.c.r.a.r.b.f.r.SalesSyncReader - Get target shops [ size = 355 ]
10:05:27.960 [main] WARN  j.c.r.a.r.b.c.aop.DaoInterceptor - Error has occured in Dao. [ error message = Failed to connect Couchbase. / params = ArrayList.SubList[parent=java.util.ArrayList@25cafb88,paren
tOffset=0,offset=0,size=355] ]
10:05:27.962 [main] ERROR o.s.batch.core.step.AbstractStep - Encountered an error executing step SalesSyncStep in job SalesSyncJob
java.lang.RuntimeException: Failed to connect Couchbase.
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.common.aop.DaoInterceptor.beforeCouchbaseDao(DaoInterceptor.java:76)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:609)
    at org.springframework.aop.aspectj.AspectJMethodBeforeAdvice.before(AspectJMethodBeforeAdvice.java:43)
    at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:51)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:62)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.dao.couchbase.ShopCouchbaseDao$$EnhancerBySpringCGLIB$$9898386a.findByIdList(<generated>)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.logic.component.SalesLogic.lambda$getShopEntityMap$1(SalesLogic.java:74)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.logic.component.SalesLogic.getShopEntityMap(SalesLogic.java:73)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.functions.processor.SalesSyncProcessor.getShopEntity(SalesSyncProcessor.java:134)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.functions.processor.SalesSyncProcessor.processValidatedItem(SalesSyncProcessor.java:74)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.functions.processor.SalesSyncProcessor.processValidatedItem(SalesSyncProcessor.java:35)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.template.processor.xxxInputBeanValidatingSalesProcessor.process(xxxInputBeanValidatingSalesProcessor.java:58)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.template.processor.xxxInputBeanValidatingSalesProcessor$$FastClassBySpringCGLIB$$63fc2e15.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
    at xx.co.xxxxxxx.xxxxxxx.xxx.batch.functions.processor.SalesSyncProcessor$$EnhancerBySpringCGLIB$$91c59d8e.process(<generated>)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.doProcess(SimpleChunkProcessor.java:126)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.transform(SimpleChunkProcessor.java:293)
    at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:192)
    at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:75)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:272)
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
    at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
    at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)
    at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144)...
10:05:28.090 [main] INFO  j.c.r.a.r.b.s.util.AlertMailUtil - send mail-------------------------------
..hostname=xxx-xxxxxxxxxxx.xxx.xx.local..toaddress=prj-rap-xxx@mail.xxxxxxx.com..fromaddress=monitor-pr-xxx-batch@mail.xxxxxxx.com..subject=[xxx-xxxxxxxxxx][xxxxxxx_xxx:ERR]Error has occured in SalesSyncJob...body=
ORIGINAL To Address=monitor-pr-xxx-batch@mail.xxxxxxx.com,monitor-pr-xxx-batch@alert.xxxxxxx.co.xx


10:05:28.091 [main] INFO  j.c.r.a.r.b.t.l.m.ExecutionTimeLog - [SalesSyncJob] [select_salesAll] time = 874msec. (Count = 1) 
10:05:28.091 [main] INFO  j.c.r.a.r.b.t.l.m.ExecutionTimeLog - [SalesSyncJob] [select_client_by_id_list] time = 42msec. (Count = 4) 
10:05:28.091 [main] INFO  j.c.r.a.r.b.t.l.m.ExecutionTimeLog - [SalesSyncJob] [select_has_active_sales_shops_by_thread] time = 4001msec. (Count = 1) 
10:05:28.094 [main] INFO  j.c.r.a.r.b.t.l.xxxJobListener - "SalesSyncJob","2022-07-11 10:05:22","2022-07-11 10:05:28","5","542113792","257371904","47"
10:05:28.096 [main] INFO  o.s.b.c.l.support.SimpleJobLauncher - Job: [SimpleJob: [name=SalesSyncJob]] completed with the following parameters: [{-logging.path=/usr/local/proad/xxx/xxx-batch-data-
distribution/current/logs, -spring.profiles.active=stg, run.id=1, -spring.batch.job.names=SalesSyncJob}] and the following status: [FAILED]
10:05:28.099 [main] INFO  j.c.r.a.r.b.f.xxxBatchApplication - Started xxxBatchApplication in 50.887 seconds (JVM running for 51.429)
10:05:28.178 [cb-io-kv-5-3] DEBUG c.c.c.c.d.i.n.buffer.PoolThreadCache - Freed 14 thread-local buffer(s) from thread: cb-io-kv-5-3
10:05:28.180 [cb-io-kv-5-4] WARN  c.c.c.c.d.i.n.c.AbstractChannel - Force-closing a channel whose registration task was not accepted by an event loop: [id: 0x550dacb1]
java.util.concurrent.RejectedExecutionException: event executor terminated
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:923)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:350)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:343)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:825)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:815)
    at com.couchbase.client.core.deps.io.netty.channel.AbstractChannel$AbstractUnsafe.register(AbstractChannel.java:483)
    at com.couchbase.client.core.deps.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:87)
    at com.couchbase.client.core.deps.io.netty.channel.SingleThreadEventLoop.register(SingleThreadEventLoop.java:81)
    at com.couchbase.client.core.deps.io.netty.channel.MultithreadEventLoopGroup.register(MultithreadEventLoopGroup.java:86)
    at com.couchbase.client.core.deps.io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:323)
    at com.couchbase.client.core.deps.io.netty.bootstrap.Bootstrap.doResolveAndConnect(Bootstrap.java:155)
    at com.couchbase.client.core.deps.io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:116)
    at com.couchbase.client.core.endpoint.BaseEndpoint.lambda$reconnect$1(BaseEndpoint.java:336)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
    at reactor.core......
    at java.lang.Thread.run(Thread.java:748)
10:05:28.180 [cb-io-kv-5-2] DEBUG c.c.c.c.d.i.n.buffer.PoolThreadCache - Freed 4 thread-local buffer(s) from thread: cb-io-kv-5-2
10:05:28.181 [cb-io-kv-5-4] ERROR c.c.c.c.d.i.n.u.c.D.rejectedExecution - Failed to submit a listener notification task. Event loop shut down?
java.util.concurrent.RejectedExecutionException: event executor terminated
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:923)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.offerTask(SingleThreadEventExecutor.java:350)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:343)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:825)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:815)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:841)
    at com.couchbase.client.core.deps.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:499)...
    at java.lang.Thread.run(Thread.java:748)
10:05:28.181 [cb-io-kv-5-1] DEBUG c.c.c.c.d.i.n.buffer.PoolThreadCache - Freed 5 thread-local buffer(s) from thread: cb-io-kv-5-1
10:05:28.182 [cb-io-kv-5-4] DEBUG c.c.c.c.d.i.n.buffer.PoolThreadCache - Freed 14 thread-local buffer(s) from thread: cb-io-kv-5-4
2022/07/11 10:05:28 [INFO] [SalesSyncJob] END JOB
2022/07/11 10:05:28 [INFO] [END] SalesSyncJob

rishabh_katyal · July 11, 2022, 1:57am

Also, i confirmed that the port is accessible.

rishabh_katyal · July 11, 2022, 2:32pm

@daschl I was able to connect when i commented out the waitUntilReady(10). But this makes me curious why the code is not working with waitUntilReady(10)

mreiche · July 13, 2022, 5:54pm

getting a bucket doesn’t actually connect to the server. Only when a get() or other operation (such as waitUntilReady()) is executed is the connection actually made.

there are several client-to-node ports that are used. The configuration is read from the kv port ( plain=11210, encrypted=11207). The 8901 shown in the log message is not necessarily the one being used.