Could not bootstrap (NullReference when add Nodes) - Swarm

Hello,
I have a cluster under a swarm deployment with 2 nodes.

When I try to access the cluster with the SDK, I get an exception when bootstraping the cluster.
As the project is in ASPCore, I use the Dependency Injection extention Nuget & the Linq2Couchbase nuget.

My configuration file is :
“Couchbase”: {
“Servers”: [
http://localhost:8091/
],
“Buckets”: [
{
“Name”: “Referential”
}
]
}

I load the services by :
public void ConfigureServices(IServiceCollection services) { // Add Couchbase services services .AddCouchbase(Configuration.GetSection("Couchbase")) .AddCouchbaseBucket<IReferentialBucketProvider>("Referential"); ...}

And when I call a the bucket, It used the dependency extension :
var bucket = _bucketProvider.GetBucket(); var doc = new Document<dynamic> { Id = key, Content = item }; var result = bucket.Upsert(doc);

Full stack trace :

[14:48:40 DBG] Waiting to check configs...        
[14:48:40 INF] Version: 2.4.8.0      
[14:48:40 INF] Configuration: {"UseConnectionPooling":false,"EnableDeadServiceUriPing":true,"VBucketRetrySleepTime":100,"QueryFailedThreshold":2,"QueryRequestTimeout":75000,"EnableQueryTiming":false,"SearchRequestTimeout":75000,"NodeAvailableCheckInterval":1000,"EnableTcpKeepAlives":true,"TcpKeepAliveTime":7200000,"TcpKeepAliveInterval":1000,"IOErrorThreshold":10,"IOErrorCheckInterval":500,"LoggerFactory":{},"UseSsl":false,"SslPort":11207,"ApiPort":8092,"MgmtPort":8091,"DirectPort":11210,"HttpsMgmtPort":18091,"HttpsApiPort":18092,"ObserveTimeout":500,"ObserveInterval":10,"MaxViewRetries":2,"ViewHardTimeout":30000,"Servers":["http://localhost:8091/pools"],"SerializationSettings":{"ReferenceLoopHandling":0,"MissingMemberHandling":0,"ObjectCreationHandling":0,"NullValueHandling":0,"DefaultValueHandling":0,"Converters":[],"PreserveReferencesHandling":0,"TypeNameHandling":0,"MetadataPropertyHandling":0,"TypeNameAssemblyFormat":0,"TypeNameAssemblyFormatHandling":0,"ConstructorHandling":0,"ContractResolver":{"DynamicCodeGeneration":false,"SerializeCompilerGeneratedMembers":false,"IgnoreSerializableInterface":false,"IgnoreSerializableAttribute":true,"NamingStrategy":{"ProcessDictionaryKeys":true,"ProcessExtensionDataNames":false,"OverrideSpecifiedNames":true}},"EqualityComparer":null,"ReferenceResolver":null,"ReferenceResolverProvider":null,"TraceWriter":null,"Binder":null,"SerializationBinder":null,"Error":null,"Context":{},"DateFormatString":"yyyy'-'MM'-'dd'T'HH':'mm':'ss.FFFFFFFK","MaxDepth":null,"Formatting":0,"DateFormatHandling":0,"DateTimeZoneHandling":3,"DateParseHandling":1,"FloatFormatHandling":0,"FloatParseHandling":0,"StringEscapeHandling":0,"Culture":{"LCID":127,"Name":"","DisplayName":"Langue invariante (Pays invariant)","NativeName":"Invariant Language (Invariant Country)","EnglishName":"Invariant Language (Invariant Country)","TwoLetterISOLanguageName":"iv","ThreeLetterISOLanguageName":"ivl","ThreeLetterWindowsLanguageName":"IVL","CompareInfo":{"Name":"","LCID":127},"TextInfo":{"ANSICodePage":1252,"OEMCodePage":437,"MacCodePage":10000,"EBCDICCodePage":37,"LCID":127,"CultureName":"","IsReadOnly":true,"ListSeparator":",","IsRightToLeft":false},"IsNeutralCulture":false,"NumberFormat":{"CurrencyDecimalDigits":2,"CurrencyDecimalSeparator":".","IsReadOnly":true,"CurrencyGroupSizes":[3],"NumberGroupSizes":[3],"PercentGroupSizes":[3],"CurrencyGroupSeparator":",","CurrencySymbol":"┬ñ","NaNSymbol":"NaN","CurrencyNegativePattern":0,"NumberNegativePattern":1,"PercentPositivePattern":0,"PercentNegativePattern":0,"NegativeInfinitySymbol":"-Infinity","NegativeSign":"-","NumberDecimalDigits":2,"NumberDecimalSeparator":".","NumberGroupSeparator":",","CurrencyPositivePattern":0,"PositiveInfinitySymbol":"Infinity","PositiveSign":"+","PercentDecimalDigits":2,"PercentDecimalSeparator":".","PercentGroupSeparator":",","PercentSymbol":"%","PerMilleSymbol":"ÔÇ░","NativeDigits":["0","1","2","3","4","5","6","7","8","9"],"DigitSubstitution":1},"DateTimeFormat":{"AMDesignator":"AM","Calendar":{"MinSupportedDateTime":"0001-01-01T00:00:00","MaxSupportedDateTime":"9999-12-31T23:59:59.9999999","AlgorithmType":1,"CalendarType":1,"Eras":[1],"TwoDigitYearMax":2029,"IsReadOnly":true},"DateSeparator":"/","FirstDayOfWeek":0,"CalendarWeekRule":0,"FullDateTimePattern":"dddd, dd MMMM yyyy HH:mm:ss","LongDatePattern":"dddd, dd MMMM yyyy","LongTimePattern":"HH:mm:ss","MonthDayPattern":"MMMM dd","PMDesignator":"PM","RFC1123Pattern":"ddd, dd MMM yyyy HH':'mm':'ss 'GMT'","ShortDatePattern":"MM/dd/yyyy","ShortTimePattern":"HH:mm","SortableDateTimePattern":"yyyy'-'MM'-'dd'T'HH':'mm':'ss","TimeSeparator":":","UniversalSortableDateTimePattern":"yyyy'-'MM'-'dd HH':'mm':'ss'Z'","YearMonthPattern":"yyyy MMMM","AbbreviatedDayNames":["Sun","Mon","Tue","Wed","Thu","Fri","Sat"],"ShortestDayNames":["Su","Mo","Tu","We","Th","Fr","Sa"],"DayNames":["Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday"],"AbbreviatedMonthNames":["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec",""],"MonthNames":["January","February","March","April","May","June","July","August","September","October","November","December",""],"IsReadOnly":true,"NativeCalendarName":"Gregorian Calendar","AbbreviatedMonthGenitiveNames":["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec",""],"MonthGenitiveNames":["January","February","March","April","May","June","July","August","September","October","November","December",""]},"Calendar":{"MinSupportedDateTime":"0001-01-01T00:00:00","MaxSupportedDateTime":"9999-12-31T23:59:59.9999999","AlgorithmType":1,"CalendarType":1,"Eras":[1],"TwoDigitYearMax":2029,"IsReadOnly":true},"OptionalCalendars":[{"MinSupportedDateTime":"0001-01-01T00:00:00","MaxSupportedDateTime":"9999-12-31T23:59:59.9999999","AlgorithmType":1,"CalendarType":1,"Eras":[1],"TwoDigitYearMax":2029,"IsReadOnly":false}],"UseUserOverride":false,"IsReadOnly":true},"CheckAdditionalContent":false},"DeserializationSettings":{"ReferenceLoopHandling":0,"MissingMemberHandling":0,"ObjectCreationHandling":0,"NullValueHandling":0,"DefaultValueHandling":0,"Converters":[],"PreserveReferencesHandling":0,"TypeNameHandling":0,"MetadataPropertyHandling":0,"TypeNameAssemblyFormat":0,"TypeNameAssemblyFormatHandling":0,"ConstructorHandling":0,"ContractResolver":{"DynamicCodeGeneration":false,"SerializeCompilerGeneratedMembers":false,"IgnoreSerializableInterface":false,"IgnoreSerializableAttribute":true,"NamingStrategy":{"ProcessDictionaryKeys":true,"ProcessExtensionDataNames":false,"OverrideSpecifiedNames":true}},"EqualityComparer":null,"ReferenceResolver":null,"ReferenceResolverProvider":null,"TraceWriter":null,"Binder":null,"SerializationBinder":null,"Error":null,"Context":{},"DateFormatString":"yyyy'-'MM'-'dd'T'HH':'mm':'ss.FFFFFFFK","MaxDepth":null,"Formatting":0,"DateFormatHandling":0,"DateTimeZoneHandling":3,"DateParseHandling":1,"FloatFormatHandling":0,"FloatParseHandling":0,"StringEscapeHandling":0,"Culture":{"LCID":127,"Name":"","DisplayName":"Langue invariante (Pays invariant)","NativeName":"Invariant Language (Invariant Country)","EnglishName":"Invariant Language (Invariant Country)","TwoLetterISOLanguageName":"iv","ThreeLetterISOLanguageName":"ivl","ThreeLetterWindowsLanguageName":"IVL","CompareInfo":{"Name":"","LCID":127},"TextInfo":{"ANSICodePage":1252,"OEMCodePage":437,"MacCodePage":10000,"EBCDICCodePage":37,"LCID":127,"CultureName":"","IsReadOnly":true,"ListSeparator":",","IsRightToLeft":false},"IsNeutralCulture":false,"NumberFormat":{"CurrencyDecimalDigits":2,"CurrencyDecimalSeparator":".","IsReadOnly":true,"CurrencyGroupSizes":[3],"NumberGroupSizes":[3],"PercentGroupSizes":[3],"CurrencyGroupSeparator":",","CurrencySymbol":"┬ñ","NaNSymbol":"NaN","CurrencyNegativePattern":0,"NumberNegativePattern":1,"PercentPositivePattern":0,"PercentNegativePattern":0,"NegativeInfinitySymbol":"-Infinity","NegativeSign":"-","NumberDecimalDigits":2,"NumberDecimalSeparator":".","NumberGroupSeparator":",","CurrencyPositivePattern":0,"PositiveInfinitySymbol":"Infinity","PositiveSign":"+","PercentDecimalDigits":2,"PercentDecimalSeparator":".","PercentGroupSeparator":",","PercentSymbol":"%","PerMilleSymbol":"ÔÇ░","NativeDigits":["0","1","2","3","4","5","6","7","8","9"],"DigitSubstitution":1},"DateTimeFormat":{"AMDesignator":"AM","Calendar":{"MinSupportedDateTime":"0001-01-01T00:00:00","MaxSupportedDateTime":"9999-12-31T23:59:59.9999999","AlgorithmType":1,"CalendarType":1,"Eras":[1],"TwoDigitYearMax":2029,"IsReadOnly":true},"DateSeparator":"/","FirstDayOfWeek":0,"CalendarWeekRule":0,"FullDateTimePattern":"dddd, dd MMMM yyyy HH:mm:ss","LongDatePattern":"dddd, dd MMMM yyyy","LongTimePattern":"HH:mm:ss","MonthDayPattern":"MMMM dd","PMDesignator":"PM","RFC1123Pattern":"ddd, dd MMM yyyy HH':'mm':'ss 'GMT'","ShortDatePattern":"MM/dd/yyyy","ShortTimePattern":"HH:mm","SortableDateTimePattern":"yyyy'-'MM'-'dd'T'HH':'mm':'ss","TimeSeparator":":","UniversalSortableDateTimePattern":"yyyy'-'MM'-'dd HH':'mm':'ss'Z'","YearMonthPattern":"yyyy MMMM","AbbreviatedDayNames":["Sun","Mon","Tue","Wed","Thu","Fri","Sat"],"ShortestDayNames":["Su","Mo","Tu","We","Th","Fr","Sa"],"DayNames":["Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday"],"AbbreviatedMonthNames":["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec",""],"MonthNames":["January","February","March","April","May","June","July","August","September","October","November","December",""],"IsReadOnly":true,"NativeCalendarName":"Gregorian Calendar","AbbreviatedMonthGenitiveNames":["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec",""],"MonthGenitiveNames":["January","February","March","April","May","June","July","August","September","October","November","December",""]},"Calendar":{"MinSupportedDateTime":"0001-01-01T00:00:00","MaxSupportedDateTime":"9999-12-31T23:59:59.9999999","AlgorithmType":1,"CalendarType":1,"Eras":[1],"TwoDigitYearMax":2029,"IsReadOnly":true},"OptionalCalendars":[{"MinSupportedDateTime":"0001-01-01T00:00:00","MaxSupportedDateTime":"9999-12-31T23:59:59.9999999","AlgorithmType":1,"CalendarType":1,"Eras":[1],"TwoDigitYearMax":2029,"IsReadOnly":false}],"UseUserOverride":false,"IsReadOnly":true},"CheckAdditionalContent":false},"BucketConfigs":{"Referential":{"UseEnhancedDurability":false,"UseKvErrorMap":false,"UseSsl":false,"Servers":["http://localhost:8091/pools"],"Port":11210,"BucketName":"Referential","Password":"","Username":"","PoolConfiguration":{"EnableTcpKeepAlives":true,"TcpKeepAliveTime":7200000,"TcpKeepAliveInterval":1000,"MaxSize":2,"MinSize":1,"WaitTimeout":2500,"MaxAcquireIterationCount":5,"RecieveTimeout":2500,"ShutdownTimeout":10000,"OperationTimeout":2500,"UseSsl":false,"SendTimeout":15000,"ConnectTimeout":10000,"MaxCloseAttempts":5,"CloseAttemptInterval":100,"EnableOperationTiming":false,"BufferSize":16384,"Uri":null},"ObserveTimeout":500,"ObserveInterval":10,"DefaultOperationLifespan":2500},"Menu":{"UseEnhancedDurability":false,"UseKvErrorMap":false,"UseSsl":false,"Servers":["http://localhost:8091/pools"],"Port":11210,"BucketName":"Menu","Password":"","Username":"","PoolConfiguration":{"EnableTcpKeepAlives":true,"TcpKeepAliveTime":7200000,"TcpKeepAliveInterval":1000,"MaxSize":2,"MinSize":1,"WaitTimeout":2500,"MaxAcquireIterationCount":5,"RecieveTimeout":2500,"ShutdownTimeout":10000,"OperationTimeout":2500,"UseSsl":false,"SendTimeout":15000,"ConnectTimeout":10000,"MaxCloseAttempts":5,"CloseAttemptInterval":100,"EnableOperationTiming":false,"BufferSize":16384,"Uri":null},"ObserveTimeout":500,"ObserveInterval":10,"DefaultOperationLifespan":2500},"Identity":{"UseEnhancedDurability":false,"UseKvErrorMap":false,"UseSsl":false,"Servers":["http://localhost:8091/pools"],"Port":11210,"BucketName":"Identity","Password":"","Username":"","PoolConfiguration":{"EnableTcpKeepAlives":true,"TcpKeepAliveTime":7200000,"TcpKeepAliveInterval":1000,"MaxSize":2,"MinSize":1,"WaitTimeout":2500,"MaxAcquireIterationCount":5,"RecieveTimeout":2500,"ShutdownTimeout":10000,"OperationTimeout":2500,"UseSsl":false,"SendTimeout":15000,"ConnectTimeout":10000,"MaxCloseAttempts":5,"CloseAttemptInterval":100,"EnableOperationTiming":false,"BufferSize":16384,"Uri":null},"ObserveTimeout":500,"ObserveInterval":10,"DefaultOperationLifespan":2500},"Product":{"UseEnhancedDurability":false,"UseKvErrorMap":false,"UseSsl":false,"Servers":["http://localhost:8091/pools"],"Port":11210,"BucketName":"Product","Password":"","Username":"","PoolConfiguration":{"EnableTcpKeepAlives":true,"TcpKeepAliveTime":7200000,"TcpKeepAliveInterval":1000,"MaxSize":2,"MinSize":1,"WaitTimeout":2500,"MaxAcquireIterationCount":5,"RecieveTimeout":2500,"ShutdownTimeout":10000,"OperationTimeout":2500,"UseSsl":false,"SendTimeout":15000,"ConnectTimeout":10000,"MaxCloseAttempts":5,"CloseAttemptInterval":100,"EnableOperationTiming":false,"BufferSize":16384,"Uri":null},"ObserveTimeout":500,"ObserveInterval":10,"DefaultOperationLifespan":2500},"Rawmaterials":{"UseEnhancedDurability":false,"UseKvErrorMap":false,"UseSsl":false,"Servers":["http://localhost:8091/pools"],"Port":11210,"BucketName":"Rawmaterials","Password":"","Username":"","PoolConfiguration":{"EnableTcpKeepAlives":true,"TcpKeepAliveTime":7200000,"TcpKeepAliveInterval":1000,"MaxSize":2,"MinSize":1,"WaitTimeout":2500,"MaxAcquireIterationCount":5,"RecieveTimeout":2500,"ShutdownTimeout":10000,"OperationTimeout":2500,"UseSsl":false,"SendTimeout":15000,"ConnectTimeout":10000,"MaxCloseAttempts":5,"CloseAttemptInterval":100,"EnableOperationTiming":false,"BufferSize":16384,"Uri":null},"ObserveTimeout":500,"ObserveInterval":10,"DefaultOperationLifespan":2500},"Nonconformitysheets":{"UseEnhancedDurability":false,"UseKvErrorMap":false,"UseSsl":false,"Servers":["http://localhost:8091/pools"],"Port":11210,"BucketName":"Nonconformitysheets","Password":"","Username":"","PoolConfiguration":{"EnableTcpKeepAlives":true,"TcpKeepAliveTime":7200000,"TcpKeepAliveInterval":1000,"MaxSize":2,"MinSize":1,"WaitTimeout":2500,"MaxAcquireIterationCount":5,"RecieveTimeout":2500,"ShutdownTimeout":10000,"OperationTimeout":2500,"UseSsl":false,"SendTimeout":15000,"ConnectTimeout":10000,"MaxCloseAttempts":5,"CloseAttemptInterval":100,"EnableOperationTiming":false,"BufferSize":16384,"Uri":null},"ObserveTimeout":500,"ObserveInterval":10,"DefaultOperationLifespan":2500}},"PoolConfiguration":{"EnableTcpKeepAlives":true,"TcpKeepAliveTime":7200000,"TcpKeepAliveInterval":1000,"MaxSize":2,"MinSize":1,"WaitTimeout":2500,"MaxAcquireIterationCount":5,"RecieveTimeout":2500,"ShutdownTimeout":10000,"OperationTimeout":2500,"UseSsl":false,"SendTimeout":15000,"ConnectTimeout":10000,"MaxCloseAttempts":5,"CloseAttemptInterval":100,"EnableOperationTiming":false,"BufferSize":16384,"Uri":null},"HeartbeatConfigInterval":10000.0,"HeartbeatConfigCheckFloor":0,"ViewRequestTimeout":75000,"DefaultConnectionLimit":5,"MaxServicePointIdleTime":100,"Expect100Continue":false,"EnableConfigHeartBeat":true,"EnableOperationTiming":false,"BufferSize":0,"DefaultOperationLifespan":2500,"AnalyticsRequestTimeout":75000}
[14:48:40 DBG] Trying to bootstrap with Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.      
[14:48:40 DBG] Getting config for bucket Referential      
[14:48:40 DBG] Bootstrapping with 127.0.0.1:11210      
[14:48:40 DBG] Trying to acquire new connection! Refs=0     
[14:48:40 VRB] 1. Checking authentication [False|False]: 127.0.0.1 :11210 - 9eebd986-858e-44ef-9886-5a848d465ac8      
[14:48:40 VRB] 2. Checking authentication [False|False]: 127.0.0.1:11210 - 9eebd986-858e-44ef-9886-5a848d465ac8       
[14:48:40 DBG] Acquire new: 9eebd986-858e-44ef-9886-5a848d465ac8 | 127.0.0.1:11210 | [0, 2] - 1e1283fb-09f2-45e5-890c-98a06ca6c2e3 - Disposed: False      
[14:48:40 DBG] Trying to acquire new connection! Refs=1      
[14:48:40 VRB] 1. Checking authentication [False|False]: 127.0.0.1:11210 - b7669171-33d0-41b6-866c-bcbe4facf5c8      
[14:48:40 VRB] 2. Checking authentication [False|False]: 127.0.0.1:11210 - b7669171-33d0-41b6-866c-bcbe4facf5c8      
[14:48:40 VRB] 3. Checking authentication [False]: 127.0.0.1:11210 - b7669171-33d0-41b6-866c-bcbe4facf5c8      
[14:48:40 DBG] Client First Message 127.0.0.1:11210 - b7669171-33d0-41b6-866c-bcbe4facf5c8: n,,n=Referential,r=zcJv4ZrUiBeXDDp5e8qbAdLI5bP/ [U:Referential|P:      
[14:48:40 DBG] Server First Message 127.0.0.1:11210 - b7669171-33d0-41b6-866c-bcbe4facf5c8: r=zcJv4ZrUiBeXDDp5e8qbAdLI5bP/f2e2681eec545306,s=pA41Nj1nOIIrd6N+b3H2qTK8g5Y=,i=4096     
[14:48:41 DBG] Client Final Message 127.0.0.1:11210 - b7669171-33d0-41b6-866c-bcbe4facf5c8: c=biws,r=zcJv4ZrUiBeXDDp5e8qbAdLI5bP/f2e2681eec545306,p=rPEfETmwizn0ZqDxlnf0md4ZXxM=     
[14:48:41 DBG] Server Final Message 127.0.0.1:11210 - b7669171-33d0-41b6-866c-bcbe4facf5c8:     
[14:48:41 INF] 4. Authenticated Referential using Couchbase.Authentication.SASL.ScramShaMechanism - a6a9a250-542b-4291-811f-937aedb9e118 - b7669171-33d0-41b6-866c-bcbe4facf5c8 [127.0.0.1:11210].     
[14:48:41 DBG] Acquire new: b7669171-33d0-41b6-866c-bcbe4facf5c8 | 127.0.0.1:11210 | [1, 2] - 1e1283fb-09f2-45e5-890c-98a06ca6c2e3 - Disposed: False     
[14:48:41 VRB] 1. Checking authentication [False|False]: 127.0.0.1:11210 - 9eebd986-858e-44ef-9886-5a848d465ac8     
[14:48:41 VRB] 2. Checking authentication [False|False]: 127.0.0.1:11210 - 9eebd986-858e-44ef-9886-5a848d465ac8     
[14:48:41 VRB] 3. Checking authentication [False]: 127.0.0.1:11210 - 9eebd986-858e-44ef-9886-5a848d465ac8    
[14:48:41 DBG] Client First Message 127.0.0.1:11210 - 9eebd986-858e-44ef-9886-5a848d465ac8:      
 n,,n=Referential,r=zcJv4ZrUiBeXDDp5e8qbAdLI5bP/ [U:Referential|P:     
[14:48:41 DBG] Server First Message 127.0.0.1:11210 - 9eebd986-858e-44ef-9886-5a848d465ac8: r=zcJv4ZrUiBeXDDp5e8qbAdLI5bP/f386c3c6cb5eedcc,s=pA41Nj1nOIIrd6N+b3H2qTK8g5Y=,i=4096     
[14:48:41 DBG] Client Final Message 127.0.0.1:11210 - 9eebd986-858e-44ef-9886-5a848d465ac8: c=biws,r=zcJv4ZrUiBeXDDp5e8qbAdLI5bP/f386c3c6cb5eedcc,p=e3yrdBNsl2bWsQYj8rp+tt9uemA=    
[14:48:41 DBG] Server Final Message 127.0.0.1:11210 - 9eebd986-858e-44ef-9886-5a848d465ac8:    
[14:48:41 INF] 4. Authenticated Referential using Couchbase.Authentication.SASL.ScramShaMechanism - a6a9a250-542b-4291-811f-937aedb9e118 - 9eebd986-858e-44ef-9886-5a848d465ac8 [127.0.0.1:11210].    
[14:48:41 DBG] Creating PooledIOService 8fe087e7-38cf-46da-abc7-5505731698a5    
[14:48:41 VRB] Using conn b7669171-33d0-41b6-866c-bcbe4facf5c8 on 127.0.0.1:11210     
[14:48:41 INF] Bootstrap config: {"name":"Referential","bucketType":"couchbase","authType":null,"saslPassword":null,"proxyPort":0,"replicaIndex":false,"uri":"/pools/default/buckets/Referential?bucket_uuid=5c3f1a1e4c65ced86c89400dd97dff33","streamingUri":"/pools/default/bucketsStreaming/Referential?bucket_uuid=5c3f1a1e4c65ced86c89400dd97dff33","terseBucketsBase":null,"terseStreamingBucketsBase":null,"localRandomKeyUri":null,"controllers":null,"nodes":[{"couchApiBase":"http://10.0.0.3:8092/Referential%2B5c3f1a1e4c65ced86c89400dd97dff33","couchApiBaseHTTPS":"https://$HOST:18092/default","replication":0.0,"clusterMembership":null,"status":null,"thisNode":false,"hostname":"10.0.0.3:8091","clusterCompatibility":0,"version":null,"os":null,"otpNode":null,"ports":{"proxy":11211,"direct":11210,"sslDirect":11207,"httpsCAPI":18092,"httpsMgmt":18091},"services":null},{"couchApiBase":"http://10.0.0.5:8092/Referential%2B5c3f1a1e4c65ced86c89400dd97dff33","couchApiBaseHTTPS":"https://$HOST:18092/default","replication":0.0,"clusterMembership":null,"status":null,"thisNode":false,"hostname":"10.0.0.5:8091","clusterCompatibility":0,"version":null,"os":null,"otpNode":null,"ports":{"proxy":11211,"direct":11210,"sslDirect":11207,"httpsCAPI":18092,"httpsMgmt":18091},"services":null}],"nodesExt":[{"services":{"fts":0,"mgmt":8091,"moxi":11211,"kv":11210,"capi":8092,"kvSSL":11207,"capiSSL":18092,"mgmtSSL":18091,"projector":9999,"indexAdmin":9100,"indexScan":9101,"indexHttp":9102,"indexStreamInit":9103,"indexStreamCatchup":9104,"indexStreamMaint":9105,"n1ql":8093,"n1qlSSL":18093},"hostname":"10.0.0.3"},{"services":{"fts":0,"mgmt":8091,"moxi":11211,"kv":11210,"capi":8092,"kvSSL":11207,"capiSSL":18092,"mgmtSSL":18091,"projector":9999,"indexAdmin":0,"indexScan":0,"indexHttp":0,"indexStreamInit":0,"indexStreamCatchup":0,"indexStreamMaint":0,"n1ql":0,"n1qlSSL":0},"hostname":"10.0.0.5"}],"stats":null,"ddocs":{"uri":"/pools/default/buckets/Referential/ddocs"},"nodeLocator":"vbucket","uuid":"5c3f1a1e4c65ced86c89400dd97dff33","vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":1,"serverList":["10.0.0.3:11210","10.0.0.5:11210"],"vBucketMap":[[0,1]...[1,0],[1,0],[1,0],[1,0]],"vBucketMapForward":[]},"replicaNumber":0,"threadsNumber":0,"quota":null,"basicStats":null,"bucketCapabilitiesVer":"","bucketCapabilities":["cbhello","touch","couchapi","cccp","xdcrCheckpointing","nodesExt","dcp"],"rev":1063,"UseSsl":false,"SurrogateHost":"127.0.0.1","Password":""}     
[14:48:41 INF] o2-Creating the Servers list using rev#1063     
[14:48:41 DBG] Trying to acquire new connection! Refs=0     
[14:48:50 DBG] Checking configs...      
[14:48:50 DBG] Waiting to check configs...      
[14:48:51 ERR]
System.Net.Sockets.SocketException: Une tentative de connexion a ├®chou├® car le parti connect├® nÔÇÖa pas r├®pondu convenablement au-del├á dÔÇÖune certaine dur├®e ou une connexion ├®tablie a ├®chou├® car lÔÇÖh├┤te de connexion nÔÇÖa pas r├®pondu
   at Couchbase.IO.DefaultConnectionFactory.<>c__0`1.<GetGeneric>b__0_0(IConnectionPool`1 p, IByteConverter c, BufferAllocator b)
   at Couchbase.IO.SharedConnectionPool`1.CreateAndAuthConnection()
   at Couchbase.IO.SharedConnectionPool`1.Acquire()
   at Couchbase.Authentication.SASL.SaslFactory.<>c.<GetFactory>b__2_0(String username, String password, IConnectionPool pool, ITypeTranscoder transcoder)       
[14:48:51 DBG] Trying to acquire new connection! Refs=0
[14:49:00 DBG] Checking configs...
[14:49:00 DBG] Waiting to check configs...
[14:49:01 ERR] Initialize failed for cd18236f-dcae-4204-8a42-2b7359ded1e9
[14:49:01 DBG] Creating PooledIOService 8a159b08-f655-4b2b-98f2-5426b26c6976
[14:49:01 ERR] Could not add server 10.0.0.3:11210. Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at Couchbase.IO.Services.PooledIOService.CheckEnabledServerFeatures(IConnection connection)
   at Couchbase.IO.Services.PooledIOService..ctor(IConnectionPool connectionPool)
   at Couchbase.IO.IOServiceFactory.<>c__DisplayClass0_0.<GetFactory>b__0(IConnectionPool pool)
   at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IIOService ioService)      
[14:49:01 DBG] Trying to acquire new connection! Refs=0     
[14:49:10 DBG] Checking configs...       
[14:49:10 DBG] Waiting to check configs...      
[14:49:11 ERR]
System.Net.Sockets.SocketException: Une tentative de connexion a ├®chou├® car le parti connect├® nÔÇÖa pas r├®pondu convenablement au-del├á dÔÇÖune certaine dur├®e ou une connexion ├®tablie a ├®chou├® car lÔÇÖh├┤te de connexion nÔÇÖa pas r├®pondu
   at Couchbase.IO.DefaultConnectionFactory.<>c__0`1.<GetGeneric>b__0_0(IConnectionPool`1 p, IByteConverter c, BufferAllocator b)
   at Couchbase.IO.SharedConnectionPool`1.CreateAndAuthConnection()
   at Couchbase.IO.SharedConnectionPool`1.Acquire()
   at Couchbase.Authentication.SASL.SaslFactory.<>c.<GetFactory>b__2_0(String username, String password, IConnectionPool pool, ITypeTranscoder transcoder)     
[14:49:11 DBG] Trying to acquire new connection! Refs=0      
[14:49:20 DBG] Checking configs...     
[14:49:20 DBG] Waiting to check configs...      
[14:49:21 ERR] Initialize failed for 52d1b494-e22e-4c8b-8e5b-a070795063f4     
[14:49:21 DBG] Creating PooledIOService 7223ab62-768a-466a-b98c-da51f1ddf391      
[14:49:21 ERR] Could not add server 10.0.0.5:11210. Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at Couchbase.IO.Services.PooledIOService.CheckEnabledServerFeatures(IConnection connection)
   at Couchbase.IO.Services.PooledIOService..ctor(IConnectionPool connectionPool)
   at Couchbase.IO.IOServiceFactory.<>c__DisplayClass0_0.<GetFactory>b__0(IConnectionPool pool)
   at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IIOService ioService)      
[14:49:21 DBG] Bootstrapping with 127.0.0.1:11210 failed.      
[14:49:21 WRN]
Couchbase.Configuration.CouchbaseBootstrapException: After bootstrapping the nodes list has zero elements indicating that client failed during bootstrapping. Please check the client logs for more information as for why it failed.
   at Couchbase.Configuration.CouchbaseConfigContext.UpdateServices(Dictionary`2 servers)
   at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IIOService ioService)
   at Couchbase.Configuration.Server.Providers.CarrierPublication.CarrierPublicationProvider.GetConfig(String bucketName, String username, String password)     
[14:49:21 DBG] Creating bucket refCount# 0    
[14:49:21 INF] Config updated old/new: 0, 1063    

docker–compose.yml :
version: “3”

services:

  couchbase-master:
    image: arungupta/couchbase:swarm
    ports:
      - 8091:8091
      - 8092:8092
      - 8093:8093
      - 11210:11210
    networks:
      - networkcluster
    environment:
      - TYPE=MASTER

  couchbase-worker:
    image: arungupta/couchbase:swarm
    networks:
      - networkcluster
    environment:
      - TYPE=WORKER
      - COUCHBASE_MASTER=couchbase-master 
      - AUTO_REBALANCE=true
    deploy:
      mode: replicated
      replicas: 1
      
networks:
    networkcluster:

docker cmd to build the swarm cluster

`docker swarm init
docker stack deploy --compose-file docker-compose.yml cluster`

If needed to delete the swarm mode : docker swarm leave -f

There is no authentication on bucket.

I have no clue why there is a null reference exception during bootsrap when it try to add nodes, because it work fine with one node in swarm mode ( to try delete worker node in dockercompose)

Hi @PierloCt,

I noticed a couple of things that might be the issue. First, you are using an arungupta/couchbase:swarm as your base image. That image probably contains an older version of Couchbase. It’s also not going to be supported in the long run. I would recommend that you use the official Couchbase image here: https://hub.docker.com/_/couchbase/

Second, I noticed that ports 8091-8093 are exposed, but not 8094. You may be doing an operation that needs that port.

@matthew.groves
Hello Matthew, we are based on arungupta image but we modified his dockerfile to get the lastest image from couchbase docker hub. (it gets the 4.6.0 entreprise )


We tried to open the 8094 port, but its the same error.

Exceptions I don’t understand are these 2 :
[17:51:53 DBG] Trying to acquire new connection! Refs=0
[17:52:02 DBG] Checking configs…
[17:52:02 DBG] Waiting to check configs…
[17:52:03 ERR] Initialize failed for 1cd1201a-05e0-4cc7-8dca-5da658b196b6
[17:52:03 DBG] Creating PooledIOService ac951d1a-448b-430a-a64d-de127b66c70b
[17:52:03 ERR] Could not add server 10.0.0.3:11210. Exception: System.NullReferenceException: Object reference not set to an instance of an object.
at Couchbase.IO.Services.PooledIOService.CheckEnabledServerFeatures(IConnection connection)
at Couchbase.IO.Services.PooledIOService…ctor(IConnectionPool connectionPool)
at Couchbase.IO.IOServiceFactory.<>c__DisplayClass0_0.b__0(IConnectionPool pool)
at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IIOService ioService)
[17:52:03 DBG] Trying to acquire new connection! Refs=0
[17:52:12 DBG] Checking configs…
[17:52:12 DBG] Waiting to check configs…
[17:52:13 ERR] [17:52:22 DBG] Waiting to check configs…
[17:52:23 ERR] Initialize failed for 41a4ecca-40e8-45bc-a4c1-7edfc9d9dcde
[17:52:23 DBG] Creating PooledIOService 4db4b4e7-494f-4a2a-852b-c83b70b63dd6
[17:52:23 ERR] Could not add server 10.0.0.5:11210. Exception: System.NullReferenceException: Object reference not set to an instance of an object.
at Couchbase.IO.Services.PooledIOService.CheckEnabledServerFeatures(IConnection connection)
at Couchbase.IO.Services.PooledIOService…ctor(IConnectionPool connectionPool)
at Couchbase.IO.IOServiceFactory.<>c__DisplayClass0_0.b__0(IConnectionPool pool)
at Couchbase.Configuration.CouchbaseConfigContext.LoadConfig(IIOService ioService)
[17:52:23 DBG] Bootstrapping with 127.0.0.1:11210 failed.

It seems detecte the 2 nodes, but when the code is executed to add servers to the configuration, node are null.
What is disturbing, it’s if we deleted the worker node, and only have the master in the swarm, everything works fine.

@PierloCt,

Are you accessing the cluster from outside the docker host? Or another way of asking, is each node running within its own host? That’s a problem we’ve seen before: the client needs access to each node in the cluster. From outside the host, only one node is typically available (because the port can only be assigned to one node).

Something you might want to try is https://github.com/couchbaselabs/sdk-doctor which is an experimental diagnosis tool that might give you some more information.

@matthew.groves

Hello,
We access the cluster from outside the host. we used this blog article to create our dockerfile : https://blog.couchbase.com/docker-service-swarm-mode-couchbase-cluster/

And we get the same results to each of the docker CLI

   PS  > docker service ls
    ID                  NAME                       MODE                REPLICAS            IMAGE                        PORTS
    g4zb3ctn8dy0        cluster_couchbase-master   replicated          1/1                 arungupta/couchbase:latest   *:8091->8091/tcp,*:8092->8092/tcp,*:8093->8093/tcp,*:8094->8094/tcp,*:11210->11210/tcp
    z0xwp6xg6kyk        cluster_couchbase-worker   replicated          1/1                 arungupta/couchbase:latest

    PS  > docker ps
    CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                  NAMES
    e665e96b0992        arungupta/couchbase:latest   "/entrypoint.sh /o..."   16 hours ago        Up 16 hours         8091-8094/tcp, 11207/tcp, 11210-11211/tcp,18091-18093/tcp   cluster_couchbase-worker.1.c8enjpgnr7bphugx9qozlwzlu
    281951f1f772        arungupta/couchbase:latest   "/entrypoint.sh /o..."   16 hours ago        Up 16 hours         8091-8094/tcp, 11207/tcp, 11210-11211/tcp,18091-18093/tcp   cluster_couchbase-master.1.roh3i42b9qkec5eyesmwpt9ab

And the result of my SDK Doctor :

|====================================================================|
|          ___ ___  _  __   ___   ___   ___ _____ ___  ___           |
|         / __|   \| |/ /__|   \ / _ \ / __|_   _/ _ \| _ \          |
|         \__ \ |) | ' <___| |) | (_) | (__  | || (_) |   /          |
|         |___/___/|_|\_\  |___/ \___/ \___| |_| \___/|_|_\          |
|                                                                    |
|====================================================================|

Note: Diagnostics can only provide accurate results when your cluster
 is in a stable state.  Active rebalancing and other cluster configuration
 changes can cause the output of the doctor to be inconsistent or in the
 worst cases, completely incorrect.

18:19:27.321 INFO ▶ Parsing connection string `couchbase://127.0.0.1/Referential`
18:19:27.383 INFO ▶ Connection string was parsed as a potential DNS SRV record
18:19:27.400 INFO ▶ Connection string identifies the following CCCP endpoints:
18:19:27.400 INFO ▶   1. 127.0.0.1:11210
18:19:27.400 INFO ▶ Connection string identifies the following HTTP endpoints:
18:19:27.401 INFO ▶   1. 127.0.0.1:8091
18:19:27.402 INFO ▶ Connection string specifies bucket `Referential`
18:19:27.403 WARN ▶ Your connection string specifies only a single host.  You should consider adding additional static n
odes from your cluster to this list to improve your applications fault-tolerance
18:19:27.404 INFO ▶ Performing DNS lookup for host `127.0.0.1`
18:19:27.405 INFO ▶ Attempting to connect to cluster via CCCP
18:19:27.406 INFO ▶ Attempting to fetch config via cccp from `127.0.0.1:11210`
18:19:27.420 WARN ▶ Bootstrap host `127.0.0.1` is not using the canonical node hostname of `10.0.0.5`.  This is not necc
essarily an error, but has been known to result in strange and difficult-to-diagnose errors in the future when routing g
ets changed.
18:19:27.421 INFO ▶ Identified the following nodes:
18:19:27.421 INFO ▶   [0] 10.0.0.3
18:19:27.422 INFO ▶                  capiSSL: 18092,                 capi:  8092,                kvSSL: 11207
18:19:27.423 INFO ▶                projector:  9999,                   kv: 11210,                 moxi: 11211
18:19:27.424 INFO ▶                     mgmt:  8091,              mgmtSSL: 18091
18:19:27.424 INFO ▶   [1] 10.0.0.5
18:19:27.425 INFO ▶                    kvSSL: 11207,            projector:  9999,            indexHttp:  9102
18:19:27.427 INFO ▶          indexStreamInit:  9103,   indexStreamCatchup:  9104,                   kv: 11210
18:19:27.428 INFO ▶                  mgmtSSL: 18091,           indexAdmin:  9100,              capiSSL: 18092
18:19:27.430 INFO ▶                  n1qlSSL: 18093,     indexStreamMaint:  9105,                 capi:  8092
18:19:27.430 INFO ▶                     moxi: 11211,                 n1ql:  8093,                 mgmt:  8091
18:19:27.431 INFO ▶                indexScan:  9101
18:19:27.431 INFO ▶ Fetching config from `10.0.0.3:8091`
18:19:29.433 INFO ▶ Failed to retreive cluster information (error: Get http://10.0.0.3:8091/pools/default: net/http: req
uest canceled while waiting for connection (Client.Timeout exceeded while awaiting headers))
18:19:31.433 ERRO ▶ Failed to connect to KV service at `10.0.0.3:11210` (error: dial tcp 10.0.0.3:11210: i/o timeout)
18:19:33.434 ERRO ▶ Failed to connect to MGMT service at `10.0.0.3:8091` (error: Get http://10.0.0.3:8091/: net/http: re
quest canceled while waiting for connection (Client.Timeout exceeded while awaiting headers))
18:19:35.434 ERRO ▶ Failed to connect to CAPI service at `10.0.0.3:8092` (error: Get http://10.0.0.3:8092/: net/http: re
quest canceled while waiting for connection (Client.Timeout exceeded while awaiting headers))
18:19:37.434 ERRO ▶ Failed to connect to KV service at `10.0.0.5:11210` (error: dial tcp 10.0.0.5:11210: i/o timeout)
18:19:39.435 ERRO ▶ Failed to connect to MGMT service at `10.0.0.5:8091` (error: Get http://10.0.0.5:8091/: net/http: re
quest canceled while waiting for connection (Client.Timeout exceeded while awaiting headers))
18:19:41.435 ERRO ▶ Failed to connect to CAPI service at `10.0.0.5:8092` (error: Get http://10.0.0.5:8092/: net/http: re
quest canceled while waiting for connection (Client.Timeout exceeded while awaiting headers))
18:19:43.435 ERRO ▶ Failed to connect to N1QL service at `10.0.0.5:8093` (error: Get http://10.0.0.5:8093/: net/http: re
quest canceled while waiting for connection (Client.Timeout exceeded while awaiting headers))
18:19:45.436 WARN ▶ Failed to perform KV connection performance analysis on `10.0.0.3:11210` (error: %!d(string=dial tcp
 10.0.0.3:11210: i/o timeout))
18:19:47.436 WARN ▶ Failed to perform KV connection performance analysis on `10.0.0.5:11210` (error: %!d(string=dial tcp
 10.0.0.5:11210: i/o timeout))
18:19:47.436 INFO ▶ Diagnostics completed

Summary:
[WARN] Your connection string specifies only a single host.  You should consider adding additional static nodes from you
r cluster to this list to improve your applications fault-tolerance
[WARN] Bootstrap host `127.0.0.1` is not using the canonical node hostname of `10.0.0.5`.  This is not neccessarily an e
rror, but has been known to result in strange and difficult-to-diagnose errors in the future when routing gets changed.
[WARN] Failed to perform KV connection performance analysis on `10.0.0.3:11210` (error: %!d(string=dial tcp 10.0.0.3:112
10: i/o timeout))
[WARN] Failed to perform KV connection performance analysis on `10.0.0.5:11210` (error: %!d(string=dial tcp 10.0.0.5:112
10: i/o timeout))
[ERRO] Failed to connect to KV service at `10.0.0.3:11210` (error: dial tcp 10.0.0.3:11210: i/o timeout)
[ERRO] Failed to connect to MGMT service at `10.0.0.3:8091` (error: Get http://10.0.0.3:8091/: net/http: request cancele
d while waiting for connection (Client.Timeout exceeded while awaiting headers))
[ERRO] Failed to connect to CAPI service at `10.0.0.3:8092` (error: Get http://10.0.0.3:8092/: net/http: request cancele
d while waiting for connection (Client.Timeout exceeded while awaiting headers))
[ERRO] Failed to connect to KV service at `10.0.0.5:11210` (error: dial tcp 10.0.0.5:11210: i/o timeout)
[ERRO] Failed to connect to MGMT service at `10.0.0.5:8091` (error: Get http://10.0.0.5:8091/: net/http: request cancele
d while waiting for connection (Client.Timeout exceeded while awaiting headers))
[ERRO] Failed to connect to CAPI service at `10.0.0.5:8092` (error: Get http://10.0.0.5:8092/: net/http: request cancele
d while waiting for connection (Client.Timeout exceeded while awaiting headers))
[ERRO] Failed to connect to N1QL service at `10.0.0.5:8093` (error: Get http://10.0.0.5:8093/: net/http: request cancele
d while waiting for connection (Client.Timeout exceeded while awaiting headers))

We deploy it on a azure swarm to try outside localhost environment, same result for the sdl doctor utility

|====================================================================|
|          ___ ___  _  __   ___   ___   ___ _____ ___  ___           |
|         / __|   \| |/ /__|   \ / _ \ / __|_   _/ _ \| _ \          |
|         \__ \ |) | ' <___| |) | (_) | (__  | || (_) |   /          |
|         |___/___/|_|\_\  |___/ \___/ \___| |_| \___/|_|_\          |
|                                                                    |
|====================================================================|

Note: Diagnostics can only provide accurate results when your cluster
 is in a stable state.  Active rebalancing and other cluster configuration
 changes can cause the output of the doctor to be inconsistent or in the
 worst cases, completely incorrect.

10:14:47.167 INFO ▶ Parsing connection string `couchbase://52.178.35.181/Referential`
10:14:47.581 INFO ▶ Connection string was parsed as a potential DNS SRV record
10:14:47.818 INFO ▶ Connection string identifies the following CCCP endpoints:
10:14:47.831 INFO ▶   1. 52.178.35.181:11210
10:14:47.847 INFO ▶ Connection string identifies the following HTTP endpoints:
10:14:47.860 INFO ▶   1. 52.178.35.181:8091
10:14:47.866 INFO ▶ Connection string specifies bucket `Referential`
10:14:47.873 WARN ▶ Your connection string specifies only a single host.  You should consider adding additional static nodes from your cluster to this list t
o improve your applications fault-tolerance
10:14:47.880 INFO ▶ Performing DNS lookup for host `52.178.35.181`
10:14:47.881 INFO ▶ Attempting to connect to cluster via CCCP
10:14:47.886 INFO ▶ Attempting to fetch config via cccp from `52.178.35.181:11210`
10:14:48.069 WARN ▶ Bootstrap host `52.178.35.181` is not using the canonical node hostname of `10.0.0.3`.  This is not neccessarily an error, but has been k
nown to result in strange and difficult-to-diagnose errors in the future when routing gets changed.
10:14:48.084 INFO ▶ Identified the following nodes:
10:14:48.088 INFO ▶   [0] 10.0.0.3
10:14:48.101 INFO ▶                  mgmtSSL: 18091,                 moxi: 11211,              n1qlSSL: 18093
10:14:48.122 INFO ▶         indexStreamMaint:  9105,                   kv: 11210,              capiSSL: 18092
10:14:48.130 INFO ▶                projector:  9999,            indexHttp:  9102,      indexStreamInit:  9103
10:14:48.136 INFO ▶       indexStreamCatchup:  9104,                 capi:  8092,                kvSSL: 11207
10:14:48.149 INFO ▶                     mgmt:  8091,           indexAdmin:  9100,            indexScan:  9101
10:14:48.168 INFO ▶                     n1ql:  8093
10:14:48.178 INFO ▶   [1] 10.0.0.5
10:14:48.182 INFO ▶                  capiSSL: 18092,                 capi:  8092,                kvSSL: 11207
10:14:48.195 INFO ▶                projector:  9999,                   kv: 11210,                 moxi: 11211
10:14:48.205 INFO ▶                     mgmt:  8091,              mgmtSSL: 18091
10:14:48.227 INFO ▶ Fetching config from `10.0.0.3:8091`
10:14:50.237 INFO ▶ Failed to retreive cluster information (error: Get http://10.0.0.3:8091/pools/default: net/http: request canceled while waiting for conne
ction (Client.Timeout exceeded while awaiting headers))
10:14:52.238 ERRO ▶ Failed to connect to KV service at `10.0.0.3:11210` (error: dial tcp 10.0.0.3:11210: i/o timeout)
10:14:54.240 ERRO ▶ Failed to connect to MGMT service at `10.0.0.3:8091` (error: Get http://10.0.0.3:8091/: net/http: request canceled while waiting for conn
ection (Client.Timeout exceeded while awaiting headers))
10:14:56.242 ERRO ▶ Failed to connect to CAPI service at `10.0.0.3:8092` (error: Get http://10.0.0.3:8092/: net/http: request canceled while waiting for conn
ection (Client.Timeout exceeded while awaiting headers))
10:14:58.242 ERRO ▶ Failed to connect to N1QL service at `10.0.0.3:8093` (error: Get http://10.0.0.3:8093/: net/http: request canceled while waiting for conn
ection (Client.Timeout exceeded while awaiting headers))
10:15:00.242 ERRO ▶ Failed to connect to KV service at `10.0.0.5:11210` (error: dial tcp 10.0.0.5:11210: i/o timeout)
10:15:02.242 ERRO ▶ Failed to connect to MGMT service at `10.0.0.5:8091` (error: Get http://10.0.0.5:8091/: net/http: request canceled while waiting for conn
ection (Client.Timeout exceeded while awaiting headers))
10:15:04.242 ERRO ▶ Failed to connect to CAPI service at `10.0.0.5:8092` (error: Get http://10.0.0.5:8092/: net/http: request canceled while waiting for conn
ection (Client.Timeout exceeded while awaiting headers))
10:15:06.243 WARN ▶ Failed to perform KV connection performance analysis on `10.0.0.3:11210` (error: %!d(string=dial tcp 10.0.0.3:11210: i/o timeout))
10:15:08.244 WARN ▶ Failed to perform KV connection performance analysis on `10.0.0.5:11210` (error: %!d(string=dial tcp 10.0.0.5:11210: i/o timeout))
10:15:08.244 INFO ▶ Diagnostics completed

Hi @PierloCt

I’ve tried to follow the blog and see the same issue with connectivity as you have. If I connect with just the couchbase-master service running, it looks to work as expected. I’m not certain but I think it may be something to do with how the ingress ports are mapped. Internally the containers get private IPs in the sub-network but externally we still have to map to known IP:Port combinations.

I’ll keep looking into this.

Not sure if this is useful for you, but there is a more recent blog post that talks about using Couchbase on both OpenShift and Kubernates.

Regards
Mike

Hello @MikeGoldsmith ,

Thanks for looking deep into :slight_smile:

For the moment, as we have a demonstration tomorrow, we only have the master container. As you said, it works. But our target for production is to be scalable. I hope we can find a solution until then.

For the combinaison External/Internal IP, I was thinking on it too due to the answer of @matthew.groves.

unfortunalty we must be on swarm for a first time, and migrate containers to kubernetes during next year.
Schema on the blog article for kubernetes shows a container data by pods, so all containers may have their external IP with the pods. So it could easier mapped the External/Internal IP Combinaisons for the cluster.

Hello @MikeGoldsmith & @matthew.groves,

Have you find something about the issue for swarm ?

Thanks a lot for your answers and your work

Hello @matthew.groves & @MikeGoldsmith ,

We’ve seen that the SDK has changed, but is Docker Swarm problem for Nodes connection is solved or not in the 2.5.0 ?

We need that feature to speed up bulk insert of data, and it start to be urgent for our project.
We need to konw quickly to decide for a switch to an alternative way (as Kubenetes) if Docker Swarm is not supported by the netClient ?

Thanks a lot

Hi @PierloCt

Unfortunately this is not something we can solve easily. This is because docker swarm nodes maps external > internal ports so you can have multiple instances of the service running on 1 or more hosts.

This doesn’t work with Couchbase because the client needs to know the ports it can communicate with the cluster with and it doesn’t know about the port mapping. I believe there are ways around it but I don’t have experience doing that.

I believe Kubenetes understands some services are more stateful and will handle port mapping and exposing them to consumer applications better.