Unable to connect neo4j and elasticsearch in docker
up vote
0
down vote
favorite
I'm trying to connect neo4j to elasticsearch in docker container with the graphaware plugin and keep getting the same error :
ERROR Failed to create Elasticsearch index. org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
Everything works well when I run ES and neo4j without any container.
Here is the total log of what happen when i run docker compose up
:
Creating network "neo4jesdocker_neo-elk" with driver "bridge"
Creating elastic_container ...
Creating neo4j ...
Creating elastic_container
Creating neo4j ... done
Attaching to elastic_container, neo4j
elastic_container | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
elastic_container | [2018-11-10T17:28:22,827][INFO ][o.e.n.Node ] initializing ...
elastic_container | [2018-11-10T17:28:23,023][INFO ][o.e.e.NodeEnvironment ] [8a-rsYj] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda6)]], net usable_space [486.2gb], net total_space [533.2gb], types [ext4]
elastic_container | [2018-11-10T17:28:23,024][INFO ][o.e.e.NodeEnvironment ] [8a-rsYj] heap size [247.5mb], compressed ordinary object pointers [true]
elastic_container | [2018-11-10T17:28:23,028][INFO ][o.e.n.Node ] [8a-rsYj] node name derived from node ID [8a-rsYjdR6Gh9liZ3H5oEQ]; set [node.name] to override
elastic_container | [2018-11-10T17:28:23,029][INFO ][o.e.n.Node ] [8a-rsYj] version[6.4.2], pid[1], build[default/tar/04711c2/2018-09-26T13:34:09.098244Z], OS[Linux/4.15.0-38-generic/amd64], JVM["Oracle Corporation"/OpenJDK 64-Bit Server VM/10.0.2/10.0.2+13]
elastic_container | [2018-11-10T17:28:23,029][INFO ][o.e.n.Node ] [8a-rsYj] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.nbXRlMe7, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.cgroups.hierarchy.override=/, -Xmx256m, -Xms256m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [aggs-matrix-stats]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [analysis-common]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [ingest-common]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-expression]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-mustache]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-painless]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [mapper-extras]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [parent-join]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [percolator]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [rank-eval]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [reindex]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [repository-url]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [transport-netty4]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [tribe]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-core]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-deprecation]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-graph]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-logstash]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-ml]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-monitoring]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-rollup]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-security]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-sql]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-upgrade]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-watcher]
elastic_container | [2018-11-10T17:28:25,010][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded plugin [ingest-geoip]
elastic_container | [2018-11-10T17:28:25,010][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded plugin [ingest-user-agent]
elastic_container | [2018-11-10T17:28:27,224][WARN ][o.e.d.s.ScriptModule ] Script: returning default values for missing document values is deprecated. Set system property '-Des.scripting.exception_for_missing_value=true' to make behaviour compatible with future major versions.
neo4j | Active database: graph.db
neo4j | Directories in use:
neo4j | home: /var/lib/neo4j
neo4j | config: /var/lib/neo4j/conf
neo4j | logs: /var/lib/neo4j/logs
neo4j | plugins: /plugins
neo4j | import: NOT SET
neo4j | data: /var/lib/neo4j/data
neo4j | certificates: /var/lib/neo4j/certificates
neo4j | run: /var/lib/neo4j/run
neo4j | Starting Neo4j.
neo4j | 2018-11-10 17:28:28.932+0000 WARN Unknown config option: causal_clustering.discovery_listen_address
neo4j | 2018-11-10 17:28:28.936+0000 WARN Unknown config option: causal_clustering.raft_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.raft_listen_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: ha.host.coordination
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.transaction_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.discovery_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: ha.host.data
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.transaction_listen_address
neo4j | 2018-11-10 17:28:28.958+0000 INFO ======== Neo4j 3.4.7 ========
neo4j | 2018-11-10 17:28:29.020+0000 INFO Starting...
elastic_container | [2018-11-10T17:28:30,366][INFO ][o.e.x.s.a.s.FileRolesStore] [8a-rsYj] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
elastic_container | [2018-11-10T17:28:31,261][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/92] [Main.cc@109] controller (64 bit): Version 6.4.2 (Build 660eefe6f2ea55) Copyright (c) 2018 Elasticsearch BV
elastic_container | [2018-11-10T17:28:32,118][INFO ][o.e.d.DiscoveryModule ] [8a-rsYj] using discovery type [zen]
elastic_container | [2018-11-10T17:28:33,388][INFO ][o.e.n.Node ] [8a-rsYj] initialized
elastic_container | [2018-11-10T17:28:33,395][INFO ][o.e.n.Node ] [8a-rsYj] starting ...
elastic_container | [2018-11-10T17:28:33,653][INFO ][o.e.t.TransportService ] [8a-rsYj] publish_address {172.27.0.2:9300}, bound_addresses {0.0.0.0:9300}
elastic_container | [2018-11-10T17:28:33,687][INFO ][o.e.b.BootstrapChecks ] [8a-rsYj] bound or publishing to a non-loopback address, enforcing bootstrap checks
neo4j | 2018-11-10 17:28:34.159+0000 INFO Bolt enabled on 0.0.0.0:7687.
neo4j | 2018-11-10 17:28:34.162+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime enabled, bootstrapping...
neo4j | 2018-11-10 17:28:34.208+0000 INFO [c.g.r.b.RuntimeKernelExtension] Bootstrapping module with order 2, ID ES, using com.graphaware.module.es.ElasticSearchModuleBootstrapper
neo4j | 2018-11-10 17:28:34.354+0000 INFO ElasticSearch index-prefix set to neo4j-index
neo4j | 2018-11-10 17:28:34.355+0000 INFO ElasticSearch key-property set to uuid
neo4j | 2018-11-10 17:28:34.355+0000 INFO ElasticSearch force-strings set to false
neo4j | 2018-11-10 17:28:34.358+0000 INFO Elasticsearch URI set to localhost
neo4j | 2018-11-10 17:28:34.358+0000 ERROR Elasticsearch protocol set to default protocol http
neo4j | 2018-11-10 17:28:34.358+0000 INFO Elasticsearch port set to 9200
neo4j | 2018-11-10 17:28:34.359+0000 INFO ElasticSearch index-prefix set to neo4j-index
neo4j | 2018-11-10 17:28:34.359+0000 INFO ElasticSearch key-property set to uuid
neo4j | 2018-11-10 17:28:34.360+0000 INFO ElasticSearch force-strings set to false
neo4j | 2018-11-10 17:28:34.361+0000 INFO Elasticsearch mapping configured with class com.graphaware.module.es.mapping.DefaultMapping
neo4j | 2018-11-10 17:28:34.388+0000 INFO Registering module ES with GraphAware Runtime.
neo4j | 2018-11-10 17:28:34.392+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime bootstrapped, starting the Runtime...
elastic_container | [2018-11-10T17:28:36,843][INFO ][o.e.c.s.MasterService ] [8a-rsYj] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
elastic_container | [2018-11-10T17:28:36,849][INFO ][o.e.c.s.ClusterApplierService] [8a-rsYj] new_master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
elastic_container | [2018-11-10T17:28:36,879][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [8a-rsYj] publish_address {172.27.0.2:9200}, bound_addresses {0.0.0.0:9200}
elastic_container | [2018-11-10T17:28:36,883][INFO ][o.e.n.Node ] [8a-rsYj] started
elastic_container | [2018-11-10T17:28:37,504][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [8a-rsYj] Failed to clear cache for realms []
elastic_container | [2018-11-10T17:28:37,629][INFO ][o.e.l.LicenseService ] [8a-rsYj] license [39ad9926-bc4e-420d-bf69-bc14ad5751a5] mode [basic] - valid
elastic_container | [2018-11-10T17:28:37,652][INFO ][o.e.g.GatewayService ] [8a-rsYj] recovered [0] indices into cluster_state
neo4j | 2018-11-10 17:28:42.056+0000 INFO Starting GraphAware...
neo4j | 2018-11-10 17:28:42.058+0000 INFO Loading module metadata...
neo4j | 2018-11-10 17:28:42.059+0000 INFO Loading metadata for module ES
neo4j | 2018-11-10 17:28:42.132+0000 INFO Module ES seems to have been registered before, metadata loaded successfully.
neo4j | 2018-11-10 17:28:42.133+0000 INFO Module ES has not changed configuration since last run, already initialized.
neo4j | 2018-11-10 17:28:42.183+0000 INFO Module metadata loaded.
neo4j | 2018-11-10 17:28:42.184+0000 INFO Starting transaction-driven modules...
neo4j | 2018-11-10 17:28:42.184+0000 INFO Starting Elasticsearch Writer...
neo4j | 2018-11-10 17:28:42.207+0000 INFO Creating Jest Client...
neo4j | SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
neo4j | SLF4J: Defaulting to no-operation (NOP) logger implementation
neo4j | SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
neo4j | 2018-11-10 17:28:42.257+0000 INFO Created Jest Client.
neo4j | 2018-11-10 17:28:42.285+0000 ERROR Failed to create Elasticsearch index. org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | java.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.lambda$createIndexAndMapping$0(BaseMapping.java:185)
neo4j | at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
neo4j | at java.util.stream.DistinctOps$1$2.accept(DistinctOps.java:175)
neo4j | at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
neo4j | at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
neo4j | at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
neo4j | at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
neo4j | at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
neo4j | at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
neo4j | at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.createIndexAndMapping(BaseMapping.java:181)
neo4j | at com.graphaware.module.es.ElasticSearchWriter.createIndexIfNotExist(ElasticSearchWriter.java:220)
neo4j | at com.graphaware.module.es.ElasticSearchWriter.start(ElasticSearchWriter.java:92)
neo4j | at com.graphaware.runtime.module.thirdparty.WriterBasedThirdPartyIntegrationModule.start(WriterBasedThirdPartyIntegrationModule.java:61)
neo4j | at com.graphaware.module.es.ElasticSearchModule.start(ElasticSearchModule.java:87)
neo4j | at com.graphaware.runtime.manager.ProductionTxDrivenModuleManager.start(ProductionTxDrivenModuleManager.java:49)
neo4j | at com.graphaware.runtime.manager.BaseTxDrivenModuleManager.startModules(BaseTxDrivenModuleManager.java:113)
neo4j | at com.graphaware.runtime.TxDrivenRuntime.startModules(TxDrivenRuntime.java:147)
neo4j | at com.graphaware.runtime.ProductionRuntime.startModules(ProductionRuntime.java:70)
neo4j | at com.graphaware.runtime.BaseGraphAwareRuntime.start(BaseGraphAwareRuntime.java:134)
neo4j | at com.graphaware.runtime.bootstrap.RuntimeKernelExtension.lambda$start$0(RuntimeKernelExtension.java:117)
neo4j | at java.lang.Thread.run(Thread.java:748)
neo4j | Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
neo4j | at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
neo4j | at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
neo4j | at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
neo4j | at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
neo4j | at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
neo4j | at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
neo4j | at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
neo4j | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
neo4j | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
neo4j | at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:47)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.createIndexAndMapping(BaseMapping.java:199)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.lambda$createIndexAndMapping$0(BaseMapping.java:183)
neo4j | ... 21 more
neo4j | Caused by: java.net.ConnectException: Connection refused (Connection refused)
neo4j | at java.net.PlainSocketImpl.socketConnect(Native Method)
neo4j | at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
neo4j | at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
neo4j | at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
neo4j | at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
neo4j | at java.net.Socket.connect(Socket.java:589)
neo4j | at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
neo4j | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
neo4j | ... 33 more
neo4j | 2018-11-10 17:28:42.288+0000 INFO Started Elasticsearch Writer.
neo4j | 2018-11-10 17:28:42.289+0000 INFO Transaction-driven modules started.
neo4j | 2018-11-10 17:28:42.289+0000 INFO There are no timer-driven runtime modules. Not scheduling any tasks.
neo4j | 2018-11-10 17:28:42.290+0000 INFO GraphAware started.
neo4j | 2018-11-10 17:28:42.291+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime automatically started.
neo4j | 2018-11-10 17:28:45.022+0000 INFO Started.
neo4j | 2018-11-10 17:28:45.114+0000 INFO [c.g.s.f.b.GraphAwareServerBootstrapper] started
neo4j | 2018-11-10 17:28:45.115+0000 INFO Mounted unmanaged extension [com.graphaware.server] at [/graphaware]
neo4j | 2018-11-10 17:28:45.296+0000 INFO Google Analytics enabled
neo4j | 2018-11-10 17:28:45.305+0000 INFO Mounting GraphAware Framework at /graphaware
neo4j | 2018-11-10 17:28:45.312+0000 INFO Will try to scan the following packages: {com.**.graphaware.**,org.**.graphaware.**,net.**.graphaware.**}
neo4j | 2018-11-10 17:28:47.382+0000 INFO Remote interface available at http://localhost:7474/
I have created the folders for the docker volumes with the following bash file :
#!/bin/bash
mkdir -p elasticsearch/config elasticsearch/data
mkdir -p neo4j/conf neo4j/data neo4j/plugins neo4j/logs
cp ../docker-utilities/apoc-3.4.0.3-all.jar neo4j/plugins/
cp ../docker-utilities/graphaware-neo4j-to-elasticsearch-3.4.9.52.11.jar neo4j/plugins/
cp ../docker-utilities/graphaware-server-community-all-3.4.9.52.jar neo4j/plugins/
cp ../docker-utilities/neo4j.confmini neo4j/conf/neo4j.conf
cp ../docker-utilities/elasticsearch.yml elasticsearch/config/
This bash file put in the right places the required plugins and two conf files :
elasticsearch.yml :
cluster.name: test_cluster
node.name: "node"
############################## Network And HTTP ###############################
# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens
# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node
# communication. (the range means that if the port is busy, it will automatically
# try the next port).
# Set the bind address specifically (IPv4 or IPv6):
#
#network.bind_host: 192.168.0.1
# Set the address other nodes will use to communicate with this node. If not
# set, it is automatically derived. It must point to an actual IP address.
#
#network.publish_host: 192.168.0.1
# Set both 'bind_host' and 'publish_host':
#
#network.host: 192.168.0.1
# Set a custom port for the node to node communication (9300 by default):
#
#transport.tcp.port: 9300
# Enable compression for all communication between nodes (disabled by default):
#
#transport.tcp.compress: true
# Set a custom port to listen for HTTP traffic:
#
#http.port: 9200
# Set a custom allowed content length:
#
#http.max_content_length: 100mb
# Disable HTTP completely:
#
#http.enabled: false
http.host: 0.0.0.0
network.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: "X-Requested-With, Content-Type, Content-Length, X-User"
xpack.security.enabled: false
and neo4j.conf:
ACCEPT.LICENSE.AGREEMENT=yes
dbms.unmanaged_extension_classes=com.graphaware.server=/graphaware
com.graphaware.runtime.enabled=true
com.graphaware.module.UIDM.initializeUntil=0
#ES becomes the module ID:
com.graphaware.module.ES.2=com.graphaware.module.es.ElasticSearchModuleBootstrapper
#URI of Elasticsearch
com.graphaware.module.ES.uri=localhost
#Port of Elasticsearch
com.graphaware.module.ES.port=9200
Finally the docker compose to link all that :
version: '3.3'
services:
neo4jESdock:
restart: always
image: neo4j:3.4.7
container_name: neo4j
environment:
- NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
- NEO4J_AUTH=none
- NEO4J_dbms_connector_http_listen__address=:7474
- NEO4J_dbms_connector_https_listen__address=:6477
- NEO4J_dbms_connector_bolt_listen__address=:7687
- elastic="http://elastic_container"
volumes:
- ./neo4j/data:/data
- ./neo4j/plugins:/plugins
- ./neo4j/conf:/conf
- ./neo4j/logs:/neologs
ports:
- 7474:7474
- 6477:6477
- 7687:7687
networks:
- neo-elk
elastic:
image: elasticsearch:6.4.2
restart: always
container_name: elastic_container
ports:
- 9200:9200
volumes:
- ./elasticsearch/data:/usr/share/elasticsearch/data
- ./elasticsearch/config/elasticsearch.yml:/etc/elasticsearch/config/elasticsearch.yml
environment:
- "ES_JAVA_OPTS=-Xmx256m -Xms256m"
- neo4j="http://neo4j"
networks:
- neo-elk
networks:
neo-elk:
driver: bridge
And some documentation on similar projects :
https://github.com/graphaware/graph-aided-search-demo
Connect docker-compose files by shared network
https://github.com/mitdbg/aurum-datadiscovery
I'm not very experienced with docker...
docker elasticsearch neo4j graphaware
add a comment |
up vote
0
down vote
favorite
I'm trying to connect neo4j to elasticsearch in docker container with the graphaware plugin and keep getting the same error :
ERROR Failed to create Elasticsearch index. org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
Everything works well when I run ES and neo4j without any container.
Here is the total log of what happen when i run docker compose up
:
Creating network "neo4jesdocker_neo-elk" with driver "bridge"
Creating elastic_container ...
Creating neo4j ...
Creating elastic_container
Creating neo4j ... done
Attaching to elastic_container, neo4j
elastic_container | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
elastic_container | [2018-11-10T17:28:22,827][INFO ][o.e.n.Node ] initializing ...
elastic_container | [2018-11-10T17:28:23,023][INFO ][o.e.e.NodeEnvironment ] [8a-rsYj] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda6)]], net usable_space [486.2gb], net total_space [533.2gb], types [ext4]
elastic_container | [2018-11-10T17:28:23,024][INFO ][o.e.e.NodeEnvironment ] [8a-rsYj] heap size [247.5mb], compressed ordinary object pointers [true]
elastic_container | [2018-11-10T17:28:23,028][INFO ][o.e.n.Node ] [8a-rsYj] node name derived from node ID [8a-rsYjdR6Gh9liZ3H5oEQ]; set [node.name] to override
elastic_container | [2018-11-10T17:28:23,029][INFO ][o.e.n.Node ] [8a-rsYj] version[6.4.2], pid[1], build[default/tar/04711c2/2018-09-26T13:34:09.098244Z], OS[Linux/4.15.0-38-generic/amd64], JVM["Oracle Corporation"/OpenJDK 64-Bit Server VM/10.0.2/10.0.2+13]
elastic_container | [2018-11-10T17:28:23,029][INFO ][o.e.n.Node ] [8a-rsYj] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.nbXRlMe7, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.cgroups.hierarchy.override=/, -Xmx256m, -Xms256m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [aggs-matrix-stats]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [analysis-common]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [ingest-common]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-expression]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-mustache]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-painless]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [mapper-extras]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [parent-join]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [percolator]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [rank-eval]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [reindex]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [repository-url]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [transport-netty4]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [tribe]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-core]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-deprecation]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-graph]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-logstash]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-ml]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-monitoring]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-rollup]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-security]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-sql]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-upgrade]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-watcher]
elastic_container | [2018-11-10T17:28:25,010][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded plugin [ingest-geoip]
elastic_container | [2018-11-10T17:28:25,010][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded plugin [ingest-user-agent]
elastic_container | [2018-11-10T17:28:27,224][WARN ][o.e.d.s.ScriptModule ] Script: returning default values for missing document values is deprecated. Set system property '-Des.scripting.exception_for_missing_value=true' to make behaviour compatible with future major versions.
neo4j | Active database: graph.db
neo4j | Directories in use:
neo4j | home: /var/lib/neo4j
neo4j | config: /var/lib/neo4j/conf
neo4j | logs: /var/lib/neo4j/logs
neo4j | plugins: /plugins
neo4j | import: NOT SET
neo4j | data: /var/lib/neo4j/data
neo4j | certificates: /var/lib/neo4j/certificates
neo4j | run: /var/lib/neo4j/run
neo4j | Starting Neo4j.
neo4j | 2018-11-10 17:28:28.932+0000 WARN Unknown config option: causal_clustering.discovery_listen_address
neo4j | 2018-11-10 17:28:28.936+0000 WARN Unknown config option: causal_clustering.raft_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.raft_listen_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: ha.host.coordination
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.transaction_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.discovery_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: ha.host.data
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.transaction_listen_address
neo4j | 2018-11-10 17:28:28.958+0000 INFO ======== Neo4j 3.4.7 ========
neo4j | 2018-11-10 17:28:29.020+0000 INFO Starting...
elastic_container | [2018-11-10T17:28:30,366][INFO ][o.e.x.s.a.s.FileRolesStore] [8a-rsYj] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
elastic_container | [2018-11-10T17:28:31,261][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/92] [Main.cc@109] controller (64 bit): Version 6.4.2 (Build 660eefe6f2ea55) Copyright (c) 2018 Elasticsearch BV
elastic_container | [2018-11-10T17:28:32,118][INFO ][o.e.d.DiscoveryModule ] [8a-rsYj] using discovery type [zen]
elastic_container | [2018-11-10T17:28:33,388][INFO ][o.e.n.Node ] [8a-rsYj] initialized
elastic_container | [2018-11-10T17:28:33,395][INFO ][o.e.n.Node ] [8a-rsYj] starting ...
elastic_container | [2018-11-10T17:28:33,653][INFO ][o.e.t.TransportService ] [8a-rsYj] publish_address {172.27.0.2:9300}, bound_addresses {0.0.0.0:9300}
elastic_container | [2018-11-10T17:28:33,687][INFO ][o.e.b.BootstrapChecks ] [8a-rsYj] bound or publishing to a non-loopback address, enforcing bootstrap checks
neo4j | 2018-11-10 17:28:34.159+0000 INFO Bolt enabled on 0.0.0.0:7687.
neo4j | 2018-11-10 17:28:34.162+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime enabled, bootstrapping...
neo4j | 2018-11-10 17:28:34.208+0000 INFO [c.g.r.b.RuntimeKernelExtension] Bootstrapping module with order 2, ID ES, using com.graphaware.module.es.ElasticSearchModuleBootstrapper
neo4j | 2018-11-10 17:28:34.354+0000 INFO ElasticSearch index-prefix set to neo4j-index
neo4j | 2018-11-10 17:28:34.355+0000 INFO ElasticSearch key-property set to uuid
neo4j | 2018-11-10 17:28:34.355+0000 INFO ElasticSearch force-strings set to false
neo4j | 2018-11-10 17:28:34.358+0000 INFO Elasticsearch URI set to localhost
neo4j | 2018-11-10 17:28:34.358+0000 ERROR Elasticsearch protocol set to default protocol http
neo4j | 2018-11-10 17:28:34.358+0000 INFO Elasticsearch port set to 9200
neo4j | 2018-11-10 17:28:34.359+0000 INFO ElasticSearch index-prefix set to neo4j-index
neo4j | 2018-11-10 17:28:34.359+0000 INFO ElasticSearch key-property set to uuid
neo4j | 2018-11-10 17:28:34.360+0000 INFO ElasticSearch force-strings set to false
neo4j | 2018-11-10 17:28:34.361+0000 INFO Elasticsearch mapping configured with class com.graphaware.module.es.mapping.DefaultMapping
neo4j | 2018-11-10 17:28:34.388+0000 INFO Registering module ES with GraphAware Runtime.
neo4j | 2018-11-10 17:28:34.392+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime bootstrapped, starting the Runtime...
elastic_container | [2018-11-10T17:28:36,843][INFO ][o.e.c.s.MasterService ] [8a-rsYj] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
elastic_container | [2018-11-10T17:28:36,849][INFO ][o.e.c.s.ClusterApplierService] [8a-rsYj] new_master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
elastic_container | [2018-11-10T17:28:36,879][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [8a-rsYj] publish_address {172.27.0.2:9200}, bound_addresses {0.0.0.0:9200}
elastic_container | [2018-11-10T17:28:36,883][INFO ][o.e.n.Node ] [8a-rsYj] started
elastic_container | [2018-11-10T17:28:37,504][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [8a-rsYj] Failed to clear cache for realms []
elastic_container | [2018-11-10T17:28:37,629][INFO ][o.e.l.LicenseService ] [8a-rsYj] license [39ad9926-bc4e-420d-bf69-bc14ad5751a5] mode [basic] - valid
elastic_container | [2018-11-10T17:28:37,652][INFO ][o.e.g.GatewayService ] [8a-rsYj] recovered [0] indices into cluster_state
neo4j | 2018-11-10 17:28:42.056+0000 INFO Starting GraphAware...
neo4j | 2018-11-10 17:28:42.058+0000 INFO Loading module metadata...
neo4j | 2018-11-10 17:28:42.059+0000 INFO Loading metadata for module ES
neo4j | 2018-11-10 17:28:42.132+0000 INFO Module ES seems to have been registered before, metadata loaded successfully.
neo4j | 2018-11-10 17:28:42.133+0000 INFO Module ES has not changed configuration since last run, already initialized.
neo4j | 2018-11-10 17:28:42.183+0000 INFO Module metadata loaded.
neo4j | 2018-11-10 17:28:42.184+0000 INFO Starting transaction-driven modules...
neo4j | 2018-11-10 17:28:42.184+0000 INFO Starting Elasticsearch Writer...
neo4j | 2018-11-10 17:28:42.207+0000 INFO Creating Jest Client...
neo4j | SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
neo4j | SLF4J: Defaulting to no-operation (NOP) logger implementation
neo4j | SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
neo4j | 2018-11-10 17:28:42.257+0000 INFO Created Jest Client.
neo4j | 2018-11-10 17:28:42.285+0000 ERROR Failed to create Elasticsearch index. org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | java.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.lambda$createIndexAndMapping$0(BaseMapping.java:185)
neo4j | at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
neo4j | at java.util.stream.DistinctOps$1$2.accept(DistinctOps.java:175)
neo4j | at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
neo4j | at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
neo4j | at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
neo4j | at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
neo4j | at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
neo4j | at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
neo4j | at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.createIndexAndMapping(BaseMapping.java:181)
neo4j | at com.graphaware.module.es.ElasticSearchWriter.createIndexIfNotExist(ElasticSearchWriter.java:220)
neo4j | at com.graphaware.module.es.ElasticSearchWriter.start(ElasticSearchWriter.java:92)
neo4j | at com.graphaware.runtime.module.thirdparty.WriterBasedThirdPartyIntegrationModule.start(WriterBasedThirdPartyIntegrationModule.java:61)
neo4j | at com.graphaware.module.es.ElasticSearchModule.start(ElasticSearchModule.java:87)
neo4j | at com.graphaware.runtime.manager.ProductionTxDrivenModuleManager.start(ProductionTxDrivenModuleManager.java:49)
neo4j | at com.graphaware.runtime.manager.BaseTxDrivenModuleManager.startModules(BaseTxDrivenModuleManager.java:113)
neo4j | at com.graphaware.runtime.TxDrivenRuntime.startModules(TxDrivenRuntime.java:147)
neo4j | at com.graphaware.runtime.ProductionRuntime.startModules(ProductionRuntime.java:70)
neo4j | at com.graphaware.runtime.BaseGraphAwareRuntime.start(BaseGraphAwareRuntime.java:134)
neo4j | at com.graphaware.runtime.bootstrap.RuntimeKernelExtension.lambda$start$0(RuntimeKernelExtension.java:117)
neo4j | at java.lang.Thread.run(Thread.java:748)
neo4j | Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
neo4j | at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
neo4j | at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
neo4j | at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
neo4j | at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
neo4j | at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
neo4j | at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
neo4j | at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
neo4j | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
neo4j | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
neo4j | at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:47)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.createIndexAndMapping(BaseMapping.java:199)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.lambda$createIndexAndMapping$0(BaseMapping.java:183)
neo4j | ... 21 more
neo4j | Caused by: java.net.ConnectException: Connection refused (Connection refused)
neo4j | at java.net.PlainSocketImpl.socketConnect(Native Method)
neo4j | at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
neo4j | at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
neo4j | at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
neo4j | at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
neo4j | at java.net.Socket.connect(Socket.java:589)
neo4j | at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
neo4j | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
neo4j | ... 33 more
neo4j | 2018-11-10 17:28:42.288+0000 INFO Started Elasticsearch Writer.
neo4j | 2018-11-10 17:28:42.289+0000 INFO Transaction-driven modules started.
neo4j | 2018-11-10 17:28:42.289+0000 INFO There are no timer-driven runtime modules. Not scheduling any tasks.
neo4j | 2018-11-10 17:28:42.290+0000 INFO GraphAware started.
neo4j | 2018-11-10 17:28:42.291+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime automatically started.
neo4j | 2018-11-10 17:28:45.022+0000 INFO Started.
neo4j | 2018-11-10 17:28:45.114+0000 INFO [c.g.s.f.b.GraphAwareServerBootstrapper] started
neo4j | 2018-11-10 17:28:45.115+0000 INFO Mounted unmanaged extension [com.graphaware.server] at [/graphaware]
neo4j | 2018-11-10 17:28:45.296+0000 INFO Google Analytics enabled
neo4j | 2018-11-10 17:28:45.305+0000 INFO Mounting GraphAware Framework at /graphaware
neo4j | 2018-11-10 17:28:45.312+0000 INFO Will try to scan the following packages: {com.**.graphaware.**,org.**.graphaware.**,net.**.graphaware.**}
neo4j | 2018-11-10 17:28:47.382+0000 INFO Remote interface available at http://localhost:7474/
I have created the folders for the docker volumes with the following bash file :
#!/bin/bash
mkdir -p elasticsearch/config elasticsearch/data
mkdir -p neo4j/conf neo4j/data neo4j/plugins neo4j/logs
cp ../docker-utilities/apoc-3.4.0.3-all.jar neo4j/plugins/
cp ../docker-utilities/graphaware-neo4j-to-elasticsearch-3.4.9.52.11.jar neo4j/plugins/
cp ../docker-utilities/graphaware-server-community-all-3.4.9.52.jar neo4j/plugins/
cp ../docker-utilities/neo4j.confmini neo4j/conf/neo4j.conf
cp ../docker-utilities/elasticsearch.yml elasticsearch/config/
This bash file put in the right places the required plugins and two conf files :
elasticsearch.yml :
cluster.name: test_cluster
node.name: "node"
############################## Network And HTTP ###############################
# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens
# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node
# communication. (the range means that if the port is busy, it will automatically
# try the next port).
# Set the bind address specifically (IPv4 or IPv6):
#
#network.bind_host: 192.168.0.1
# Set the address other nodes will use to communicate with this node. If not
# set, it is automatically derived. It must point to an actual IP address.
#
#network.publish_host: 192.168.0.1
# Set both 'bind_host' and 'publish_host':
#
#network.host: 192.168.0.1
# Set a custom port for the node to node communication (9300 by default):
#
#transport.tcp.port: 9300
# Enable compression for all communication between nodes (disabled by default):
#
#transport.tcp.compress: true
# Set a custom port to listen for HTTP traffic:
#
#http.port: 9200
# Set a custom allowed content length:
#
#http.max_content_length: 100mb
# Disable HTTP completely:
#
#http.enabled: false
http.host: 0.0.0.0
network.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: "X-Requested-With, Content-Type, Content-Length, X-User"
xpack.security.enabled: false
and neo4j.conf:
ACCEPT.LICENSE.AGREEMENT=yes
dbms.unmanaged_extension_classes=com.graphaware.server=/graphaware
com.graphaware.runtime.enabled=true
com.graphaware.module.UIDM.initializeUntil=0
#ES becomes the module ID:
com.graphaware.module.ES.2=com.graphaware.module.es.ElasticSearchModuleBootstrapper
#URI of Elasticsearch
com.graphaware.module.ES.uri=localhost
#Port of Elasticsearch
com.graphaware.module.ES.port=9200
Finally the docker compose to link all that :
version: '3.3'
services:
neo4jESdock:
restart: always
image: neo4j:3.4.7
container_name: neo4j
environment:
- NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
- NEO4J_AUTH=none
- NEO4J_dbms_connector_http_listen__address=:7474
- NEO4J_dbms_connector_https_listen__address=:6477
- NEO4J_dbms_connector_bolt_listen__address=:7687
- elastic="http://elastic_container"
volumes:
- ./neo4j/data:/data
- ./neo4j/plugins:/plugins
- ./neo4j/conf:/conf
- ./neo4j/logs:/neologs
ports:
- 7474:7474
- 6477:6477
- 7687:7687
networks:
- neo-elk
elastic:
image: elasticsearch:6.4.2
restart: always
container_name: elastic_container
ports:
- 9200:9200
volumes:
- ./elasticsearch/data:/usr/share/elasticsearch/data
- ./elasticsearch/config/elasticsearch.yml:/etc/elasticsearch/config/elasticsearch.yml
environment:
- "ES_JAVA_OPTS=-Xmx256m -Xms256m"
- neo4j="http://neo4j"
networks:
- neo-elk
networks:
neo-elk:
driver: bridge
And some documentation on similar projects :
https://github.com/graphaware/graph-aided-search-demo
Connect docker-compose files by shared network
https://github.com/mitdbg/aurum-datadiscovery
I'm not very experienced with docker...
docker elasticsearch neo4j graphaware
There is a mistake in ES volumes, proper path to config is : - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
– machinus
Nov 11 at 20:08
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I'm trying to connect neo4j to elasticsearch in docker container with the graphaware plugin and keep getting the same error :
ERROR Failed to create Elasticsearch index. org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
Everything works well when I run ES and neo4j without any container.
Here is the total log of what happen when i run docker compose up
:
Creating network "neo4jesdocker_neo-elk" with driver "bridge"
Creating elastic_container ...
Creating neo4j ...
Creating elastic_container
Creating neo4j ... done
Attaching to elastic_container, neo4j
elastic_container | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
elastic_container | [2018-11-10T17:28:22,827][INFO ][o.e.n.Node ] initializing ...
elastic_container | [2018-11-10T17:28:23,023][INFO ][o.e.e.NodeEnvironment ] [8a-rsYj] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda6)]], net usable_space [486.2gb], net total_space [533.2gb], types [ext4]
elastic_container | [2018-11-10T17:28:23,024][INFO ][o.e.e.NodeEnvironment ] [8a-rsYj] heap size [247.5mb], compressed ordinary object pointers [true]
elastic_container | [2018-11-10T17:28:23,028][INFO ][o.e.n.Node ] [8a-rsYj] node name derived from node ID [8a-rsYjdR6Gh9liZ3H5oEQ]; set [node.name] to override
elastic_container | [2018-11-10T17:28:23,029][INFO ][o.e.n.Node ] [8a-rsYj] version[6.4.2], pid[1], build[default/tar/04711c2/2018-09-26T13:34:09.098244Z], OS[Linux/4.15.0-38-generic/amd64], JVM["Oracle Corporation"/OpenJDK 64-Bit Server VM/10.0.2/10.0.2+13]
elastic_container | [2018-11-10T17:28:23,029][INFO ][o.e.n.Node ] [8a-rsYj] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.nbXRlMe7, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.cgroups.hierarchy.override=/, -Xmx256m, -Xms256m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [aggs-matrix-stats]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [analysis-common]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [ingest-common]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-expression]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-mustache]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-painless]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [mapper-extras]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [parent-join]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [percolator]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [rank-eval]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [reindex]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [repository-url]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [transport-netty4]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [tribe]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-core]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-deprecation]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-graph]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-logstash]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-ml]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-monitoring]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-rollup]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-security]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-sql]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-upgrade]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-watcher]
elastic_container | [2018-11-10T17:28:25,010][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded plugin [ingest-geoip]
elastic_container | [2018-11-10T17:28:25,010][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded plugin [ingest-user-agent]
elastic_container | [2018-11-10T17:28:27,224][WARN ][o.e.d.s.ScriptModule ] Script: returning default values for missing document values is deprecated. Set system property '-Des.scripting.exception_for_missing_value=true' to make behaviour compatible with future major versions.
neo4j | Active database: graph.db
neo4j | Directories in use:
neo4j | home: /var/lib/neo4j
neo4j | config: /var/lib/neo4j/conf
neo4j | logs: /var/lib/neo4j/logs
neo4j | plugins: /plugins
neo4j | import: NOT SET
neo4j | data: /var/lib/neo4j/data
neo4j | certificates: /var/lib/neo4j/certificates
neo4j | run: /var/lib/neo4j/run
neo4j | Starting Neo4j.
neo4j | 2018-11-10 17:28:28.932+0000 WARN Unknown config option: causal_clustering.discovery_listen_address
neo4j | 2018-11-10 17:28:28.936+0000 WARN Unknown config option: causal_clustering.raft_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.raft_listen_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: ha.host.coordination
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.transaction_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.discovery_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: ha.host.data
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.transaction_listen_address
neo4j | 2018-11-10 17:28:28.958+0000 INFO ======== Neo4j 3.4.7 ========
neo4j | 2018-11-10 17:28:29.020+0000 INFO Starting...
elastic_container | [2018-11-10T17:28:30,366][INFO ][o.e.x.s.a.s.FileRolesStore] [8a-rsYj] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
elastic_container | [2018-11-10T17:28:31,261][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/92] [Main.cc@109] controller (64 bit): Version 6.4.2 (Build 660eefe6f2ea55) Copyright (c) 2018 Elasticsearch BV
elastic_container | [2018-11-10T17:28:32,118][INFO ][o.e.d.DiscoveryModule ] [8a-rsYj] using discovery type [zen]
elastic_container | [2018-11-10T17:28:33,388][INFO ][o.e.n.Node ] [8a-rsYj] initialized
elastic_container | [2018-11-10T17:28:33,395][INFO ][o.e.n.Node ] [8a-rsYj] starting ...
elastic_container | [2018-11-10T17:28:33,653][INFO ][o.e.t.TransportService ] [8a-rsYj] publish_address {172.27.0.2:9300}, bound_addresses {0.0.0.0:9300}
elastic_container | [2018-11-10T17:28:33,687][INFO ][o.e.b.BootstrapChecks ] [8a-rsYj] bound or publishing to a non-loopback address, enforcing bootstrap checks
neo4j | 2018-11-10 17:28:34.159+0000 INFO Bolt enabled on 0.0.0.0:7687.
neo4j | 2018-11-10 17:28:34.162+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime enabled, bootstrapping...
neo4j | 2018-11-10 17:28:34.208+0000 INFO [c.g.r.b.RuntimeKernelExtension] Bootstrapping module with order 2, ID ES, using com.graphaware.module.es.ElasticSearchModuleBootstrapper
neo4j | 2018-11-10 17:28:34.354+0000 INFO ElasticSearch index-prefix set to neo4j-index
neo4j | 2018-11-10 17:28:34.355+0000 INFO ElasticSearch key-property set to uuid
neo4j | 2018-11-10 17:28:34.355+0000 INFO ElasticSearch force-strings set to false
neo4j | 2018-11-10 17:28:34.358+0000 INFO Elasticsearch URI set to localhost
neo4j | 2018-11-10 17:28:34.358+0000 ERROR Elasticsearch protocol set to default protocol http
neo4j | 2018-11-10 17:28:34.358+0000 INFO Elasticsearch port set to 9200
neo4j | 2018-11-10 17:28:34.359+0000 INFO ElasticSearch index-prefix set to neo4j-index
neo4j | 2018-11-10 17:28:34.359+0000 INFO ElasticSearch key-property set to uuid
neo4j | 2018-11-10 17:28:34.360+0000 INFO ElasticSearch force-strings set to false
neo4j | 2018-11-10 17:28:34.361+0000 INFO Elasticsearch mapping configured with class com.graphaware.module.es.mapping.DefaultMapping
neo4j | 2018-11-10 17:28:34.388+0000 INFO Registering module ES with GraphAware Runtime.
neo4j | 2018-11-10 17:28:34.392+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime bootstrapped, starting the Runtime...
elastic_container | [2018-11-10T17:28:36,843][INFO ][o.e.c.s.MasterService ] [8a-rsYj] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
elastic_container | [2018-11-10T17:28:36,849][INFO ][o.e.c.s.ClusterApplierService] [8a-rsYj] new_master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
elastic_container | [2018-11-10T17:28:36,879][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [8a-rsYj] publish_address {172.27.0.2:9200}, bound_addresses {0.0.0.0:9200}
elastic_container | [2018-11-10T17:28:36,883][INFO ][o.e.n.Node ] [8a-rsYj] started
elastic_container | [2018-11-10T17:28:37,504][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [8a-rsYj] Failed to clear cache for realms []
elastic_container | [2018-11-10T17:28:37,629][INFO ][o.e.l.LicenseService ] [8a-rsYj] license [39ad9926-bc4e-420d-bf69-bc14ad5751a5] mode [basic] - valid
elastic_container | [2018-11-10T17:28:37,652][INFO ][o.e.g.GatewayService ] [8a-rsYj] recovered [0] indices into cluster_state
neo4j | 2018-11-10 17:28:42.056+0000 INFO Starting GraphAware...
neo4j | 2018-11-10 17:28:42.058+0000 INFO Loading module metadata...
neo4j | 2018-11-10 17:28:42.059+0000 INFO Loading metadata for module ES
neo4j | 2018-11-10 17:28:42.132+0000 INFO Module ES seems to have been registered before, metadata loaded successfully.
neo4j | 2018-11-10 17:28:42.133+0000 INFO Module ES has not changed configuration since last run, already initialized.
neo4j | 2018-11-10 17:28:42.183+0000 INFO Module metadata loaded.
neo4j | 2018-11-10 17:28:42.184+0000 INFO Starting transaction-driven modules...
neo4j | 2018-11-10 17:28:42.184+0000 INFO Starting Elasticsearch Writer...
neo4j | 2018-11-10 17:28:42.207+0000 INFO Creating Jest Client...
neo4j | SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
neo4j | SLF4J: Defaulting to no-operation (NOP) logger implementation
neo4j | SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
neo4j | 2018-11-10 17:28:42.257+0000 INFO Created Jest Client.
neo4j | 2018-11-10 17:28:42.285+0000 ERROR Failed to create Elasticsearch index. org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | java.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.lambda$createIndexAndMapping$0(BaseMapping.java:185)
neo4j | at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
neo4j | at java.util.stream.DistinctOps$1$2.accept(DistinctOps.java:175)
neo4j | at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
neo4j | at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
neo4j | at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
neo4j | at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
neo4j | at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
neo4j | at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
neo4j | at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.createIndexAndMapping(BaseMapping.java:181)
neo4j | at com.graphaware.module.es.ElasticSearchWriter.createIndexIfNotExist(ElasticSearchWriter.java:220)
neo4j | at com.graphaware.module.es.ElasticSearchWriter.start(ElasticSearchWriter.java:92)
neo4j | at com.graphaware.runtime.module.thirdparty.WriterBasedThirdPartyIntegrationModule.start(WriterBasedThirdPartyIntegrationModule.java:61)
neo4j | at com.graphaware.module.es.ElasticSearchModule.start(ElasticSearchModule.java:87)
neo4j | at com.graphaware.runtime.manager.ProductionTxDrivenModuleManager.start(ProductionTxDrivenModuleManager.java:49)
neo4j | at com.graphaware.runtime.manager.BaseTxDrivenModuleManager.startModules(BaseTxDrivenModuleManager.java:113)
neo4j | at com.graphaware.runtime.TxDrivenRuntime.startModules(TxDrivenRuntime.java:147)
neo4j | at com.graphaware.runtime.ProductionRuntime.startModules(ProductionRuntime.java:70)
neo4j | at com.graphaware.runtime.BaseGraphAwareRuntime.start(BaseGraphAwareRuntime.java:134)
neo4j | at com.graphaware.runtime.bootstrap.RuntimeKernelExtension.lambda$start$0(RuntimeKernelExtension.java:117)
neo4j | at java.lang.Thread.run(Thread.java:748)
neo4j | Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
neo4j | at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
neo4j | at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
neo4j | at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
neo4j | at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
neo4j | at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
neo4j | at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
neo4j | at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
neo4j | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
neo4j | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
neo4j | at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:47)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.createIndexAndMapping(BaseMapping.java:199)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.lambda$createIndexAndMapping$0(BaseMapping.java:183)
neo4j | ... 21 more
neo4j | Caused by: java.net.ConnectException: Connection refused (Connection refused)
neo4j | at java.net.PlainSocketImpl.socketConnect(Native Method)
neo4j | at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
neo4j | at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
neo4j | at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
neo4j | at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
neo4j | at java.net.Socket.connect(Socket.java:589)
neo4j | at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
neo4j | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
neo4j | ... 33 more
neo4j | 2018-11-10 17:28:42.288+0000 INFO Started Elasticsearch Writer.
neo4j | 2018-11-10 17:28:42.289+0000 INFO Transaction-driven modules started.
neo4j | 2018-11-10 17:28:42.289+0000 INFO There are no timer-driven runtime modules. Not scheduling any tasks.
neo4j | 2018-11-10 17:28:42.290+0000 INFO GraphAware started.
neo4j | 2018-11-10 17:28:42.291+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime automatically started.
neo4j | 2018-11-10 17:28:45.022+0000 INFO Started.
neo4j | 2018-11-10 17:28:45.114+0000 INFO [c.g.s.f.b.GraphAwareServerBootstrapper] started
neo4j | 2018-11-10 17:28:45.115+0000 INFO Mounted unmanaged extension [com.graphaware.server] at [/graphaware]
neo4j | 2018-11-10 17:28:45.296+0000 INFO Google Analytics enabled
neo4j | 2018-11-10 17:28:45.305+0000 INFO Mounting GraphAware Framework at /graphaware
neo4j | 2018-11-10 17:28:45.312+0000 INFO Will try to scan the following packages: {com.**.graphaware.**,org.**.graphaware.**,net.**.graphaware.**}
neo4j | 2018-11-10 17:28:47.382+0000 INFO Remote interface available at http://localhost:7474/
I have created the folders for the docker volumes with the following bash file :
#!/bin/bash
mkdir -p elasticsearch/config elasticsearch/data
mkdir -p neo4j/conf neo4j/data neo4j/plugins neo4j/logs
cp ../docker-utilities/apoc-3.4.0.3-all.jar neo4j/plugins/
cp ../docker-utilities/graphaware-neo4j-to-elasticsearch-3.4.9.52.11.jar neo4j/plugins/
cp ../docker-utilities/graphaware-server-community-all-3.4.9.52.jar neo4j/plugins/
cp ../docker-utilities/neo4j.confmini neo4j/conf/neo4j.conf
cp ../docker-utilities/elasticsearch.yml elasticsearch/config/
This bash file put in the right places the required plugins and two conf files :
elasticsearch.yml :
cluster.name: test_cluster
node.name: "node"
############################## Network And HTTP ###############################
# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens
# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node
# communication. (the range means that if the port is busy, it will automatically
# try the next port).
# Set the bind address specifically (IPv4 or IPv6):
#
#network.bind_host: 192.168.0.1
# Set the address other nodes will use to communicate with this node. If not
# set, it is automatically derived. It must point to an actual IP address.
#
#network.publish_host: 192.168.0.1
# Set both 'bind_host' and 'publish_host':
#
#network.host: 192.168.0.1
# Set a custom port for the node to node communication (9300 by default):
#
#transport.tcp.port: 9300
# Enable compression for all communication between nodes (disabled by default):
#
#transport.tcp.compress: true
# Set a custom port to listen for HTTP traffic:
#
#http.port: 9200
# Set a custom allowed content length:
#
#http.max_content_length: 100mb
# Disable HTTP completely:
#
#http.enabled: false
http.host: 0.0.0.0
network.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: "X-Requested-With, Content-Type, Content-Length, X-User"
xpack.security.enabled: false
and neo4j.conf:
ACCEPT.LICENSE.AGREEMENT=yes
dbms.unmanaged_extension_classes=com.graphaware.server=/graphaware
com.graphaware.runtime.enabled=true
com.graphaware.module.UIDM.initializeUntil=0
#ES becomes the module ID:
com.graphaware.module.ES.2=com.graphaware.module.es.ElasticSearchModuleBootstrapper
#URI of Elasticsearch
com.graphaware.module.ES.uri=localhost
#Port of Elasticsearch
com.graphaware.module.ES.port=9200
Finally the docker compose to link all that :
version: '3.3'
services:
neo4jESdock:
restart: always
image: neo4j:3.4.7
container_name: neo4j
environment:
- NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
- NEO4J_AUTH=none
- NEO4J_dbms_connector_http_listen__address=:7474
- NEO4J_dbms_connector_https_listen__address=:6477
- NEO4J_dbms_connector_bolt_listen__address=:7687
- elastic="http://elastic_container"
volumes:
- ./neo4j/data:/data
- ./neo4j/plugins:/plugins
- ./neo4j/conf:/conf
- ./neo4j/logs:/neologs
ports:
- 7474:7474
- 6477:6477
- 7687:7687
networks:
- neo-elk
elastic:
image: elasticsearch:6.4.2
restart: always
container_name: elastic_container
ports:
- 9200:9200
volumes:
- ./elasticsearch/data:/usr/share/elasticsearch/data
- ./elasticsearch/config/elasticsearch.yml:/etc/elasticsearch/config/elasticsearch.yml
environment:
- "ES_JAVA_OPTS=-Xmx256m -Xms256m"
- neo4j="http://neo4j"
networks:
- neo-elk
networks:
neo-elk:
driver: bridge
And some documentation on similar projects :
https://github.com/graphaware/graph-aided-search-demo
Connect docker-compose files by shared network
https://github.com/mitdbg/aurum-datadiscovery
I'm not very experienced with docker...
docker elasticsearch neo4j graphaware
I'm trying to connect neo4j to elasticsearch in docker container with the graphaware plugin and keep getting the same error :
ERROR Failed to create Elasticsearch index. org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
Everything works well when I run ES and neo4j without any container.
Here is the total log of what happen when i run docker compose up
:
Creating network "neo4jesdocker_neo-elk" with driver "bridge"
Creating elastic_container ...
Creating neo4j ...
Creating elastic_container
Creating neo4j ... done
Attaching to elastic_container, neo4j
elastic_container | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
elastic_container | [2018-11-10T17:28:22,827][INFO ][o.e.n.Node ] initializing ...
elastic_container | [2018-11-10T17:28:23,023][INFO ][o.e.e.NodeEnvironment ] [8a-rsYj] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda6)]], net usable_space [486.2gb], net total_space [533.2gb], types [ext4]
elastic_container | [2018-11-10T17:28:23,024][INFO ][o.e.e.NodeEnvironment ] [8a-rsYj] heap size [247.5mb], compressed ordinary object pointers [true]
elastic_container | [2018-11-10T17:28:23,028][INFO ][o.e.n.Node ] [8a-rsYj] node name derived from node ID [8a-rsYjdR6Gh9liZ3H5oEQ]; set [node.name] to override
elastic_container | [2018-11-10T17:28:23,029][INFO ][o.e.n.Node ] [8a-rsYj] version[6.4.2], pid[1], build[default/tar/04711c2/2018-09-26T13:34:09.098244Z], OS[Linux/4.15.0-38-generic/amd64], JVM["Oracle Corporation"/OpenJDK 64-Bit Server VM/10.0.2/10.0.2+13]
elastic_container | [2018-11-10T17:28:23,029][INFO ][o.e.n.Node ] [8a-rsYj] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.nbXRlMe7, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.cgroups.hierarchy.override=/, -Xmx256m, -Xms256m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [aggs-matrix-stats]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [analysis-common]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [ingest-common]
elastic_container | [2018-11-10T17:28:25,006][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-expression]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-mustache]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [lang-painless]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [mapper-extras]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [parent-join]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [percolator]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [rank-eval]
elastic_container | [2018-11-10T17:28:25,007][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [reindex]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [repository-url]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [transport-netty4]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [tribe]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-core]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-deprecation]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-graph]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-logstash]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-ml]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-monitoring]
elastic_container | [2018-11-10T17:28:25,008][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-rollup]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-security]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-sql]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-upgrade]
elastic_container | [2018-11-10T17:28:25,009][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded module [x-pack-watcher]
elastic_container | [2018-11-10T17:28:25,010][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded plugin [ingest-geoip]
elastic_container | [2018-11-10T17:28:25,010][INFO ][o.e.p.PluginsService ] [8a-rsYj] loaded plugin [ingest-user-agent]
elastic_container | [2018-11-10T17:28:27,224][WARN ][o.e.d.s.ScriptModule ] Script: returning default values for missing document values is deprecated. Set system property '-Des.scripting.exception_for_missing_value=true' to make behaviour compatible with future major versions.
neo4j | Active database: graph.db
neo4j | Directories in use:
neo4j | home: /var/lib/neo4j
neo4j | config: /var/lib/neo4j/conf
neo4j | logs: /var/lib/neo4j/logs
neo4j | plugins: /plugins
neo4j | import: NOT SET
neo4j | data: /var/lib/neo4j/data
neo4j | certificates: /var/lib/neo4j/certificates
neo4j | run: /var/lib/neo4j/run
neo4j | Starting Neo4j.
neo4j | 2018-11-10 17:28:28.932+0000 WARN Unknown config option: causal_clustering.discovery_listen_address
neo4j | 2018-11-10 17:28:28.936+0000 WARN Unknown config option: causal_clustering.raft_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.raft_listen_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: ha.host.coordination
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.transaction_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.discovery_advertised_address
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: ha.host.data
neo4j | 2018-11-10 17:28:28.937+0000 WARN Unknown config option: causal_clustering.transaction_listen_address
neo4j | 2018-11-10 17:28:28.958+0000 INFO ======== Neo4j 3.4.7 ========
neo4j | 2018-11-10 17:28:29.020+0000 INFO Starting...
elastic_container | [2018-11-10T17:28:30,366][INFO ][o.e.x.s.a.s.FileRolesStore] [8a-rsYj] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
elastic_container | [2018-11-10T17:28:31,261][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/92] [Main.cc@109] controller (64 bit): Version 6.4.2 (Build 660eefe6f2ea55) Copyright (c) 2018 Elasticsearch BV
elastic_container | [2018-11-10T17:28:32,118][INFO ][o.e.d.DiscoveryModule ] [8a-rsYj] using discovery type [zen]
elastic_container | [2018-11-10T17:28:33,388][INFO ][o.e.n.Node ] [8a-rsYj] initialized
elastic_container | [2018-11-10T17:28:33,395][INFO ][o.e.n.Node ] [8a-rsYj] starting ...
elastic_container | [2018-11-10T17:28:33,653][INFO ][o.e.t.TransportService ] [8a-rsYj] publish_address {172.27.0.2:9300}, bound_addresses {0.0.0.0:9300}
elastic_container | [2018-11-10T17:28:33,687][INFO ][o.e.b.BootstrapChecks ] [8a-rsYj] bound or publishing to a non-loopback address, enforcing bootstrap checks
neo4j | 2018-11-10 17:28:34.159+0000 INFO Bolt enabled on 0.0.0.0:7687.
neo4j | 2018-11-10 17:28:34.162+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime enabled, bootstrapping...
neo4j | 2018-11-10 17:28:34.208+0000 INFO [c.g.r.b.RuntimeKernelExtension] Bootstrapping module with order 2, ID ES, using com.graphaware.module.es.ElasticSearchModuleBootstrapper
neo4j | 2018-11-10 17:28:34.354+0000 INFO ElasticSearch index-prefix set to neo4j-index
neo4j | 2018-11-10 17:28:34.355+0000 INFO ElasticSearch key-property set to uuid
neo4j | 2018-11-10 17:28:34.355+0000 INFO ElasticSearch force-strings set to false
neo4j | 2018-11-10 17:28:34.358+0000 INFO Elasticsearch URI set to localhost
neo4j | 2018-11-10 17:28:34.358+0000 ERROR Elasticsearch protocol set to default protocol http
neo4j | 2018-11-10 17:28:34.358+0000 INFO Elasticsearch port set to 9200
neo4j | 2018-11-10 17:28:34.359+0000 INFO ElasticSearch index-prefix set to neo4j-index
neo4j | 2018-11-10 17:28:34.359+0000 INFO ElasticSearch key-property set to uuid
neo4j | 2018-11-10 17:28:34.360+0000 INFO ElasticSearch force-strings set to false
neo4j | 2018-11-10 17:28:34.361+0000 INFO Elasticsearch mapping configured with class com.graphaware.module.es.mapping.DefaultMapping
neo4j | 2018-11-10 17:28:34.388+0000 INFO Registering module ES with GraphAware Runtime.
neo4j | 2018-11-10 17:28:34.392+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime bootstrapped, starting the Runtime...
elastic_container | [2018-11-10T17:28:36,843][INFO ][o.e.c.s.MasterService ] [8a-rsYj] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
elastic_container | [2018-11-10T17:28:36,849][INFO ][o.e.c.s.ClusterApplierService] [8a-rsYj] new_master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {8a-rsYj}{8a-rsYjdR6Gh9liZ3H5oEQ}{hCd93BhoRCWIPTrONQxw0g}{172.27.0.2}{172.27.0.2:9300}{ml.machine_memory=12476059648, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
elastic_container | [2018-11-10T17:28:36,879][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [8a-rsYj] publish_address {172.27.0.2:9200}, bound_addresses {0.0.0.0:9200}
elastic_container | [2018-11-10T17:28:36,883][INFO ][o.e.n.Node ] [8a-rsYj] started
elastic_container | [2018-11-10T17:28:37,504][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [8a-rsYj] Failed to clear cache for realms []
elastic_container | [2018-11-10T17:28:37,629][INFO ][o.e.l.LicenseService ] [8a-rsYj] license [39ad9926-bc4e-420d-bf69-bc14ad5751a5] mode [basic] - valid
elastic_container | [2018-11-10T17:28:37,652][INFO ][o.e.g.GatewayService ] [8a-rsYj] recovered [0] indices into cluster_state
neo4j | 2018-11-10 17:28:42.056+0000 INFO Starting GraphAware...
neo4j | 2018-11-10 17:28:42.058+0000 INFO Loading module metadata...
neo4j | 2018-11-10 17:28:42.059+0000 INFO Loading metadata for module ES
neo4j | 2018-11-10 17:28:42.132+0000 INFO Module ES seems to have been registered before, metadata loaded successfully.
neo4j | 2018-11-10 17:28:42.133+0000 INFO Module ES has not changed configuration since last run, already initialized.
neo4j | 2018-11-10 17:28:42.183+0000 INFO Module metadata loaded.
neo4j | 2018-11-10 17:28:42.184+0000 INFO Starting transaction-driven modules...
neo4j | 2018-11-10 17:28:42.184+0000 INFO Starting Elasticsearch Writer...
neo4j | 2018-11-10 17:28:42.207+0000 INFO Creating Jest Client...
neo4j | SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
neo4j | SLF4J: Defaulting to no-operation (NOP) logger implementation
neo4j | SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
neo4j | 2018-11-10 17:28:42.257+0000 INFO Created Jest Client.
neo4j | 2018-11-10 17:28:42.285+0000 ERROR Failed to create Elasticsearch index. org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | java.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.lambda$createIndexAndMapping$0(BaseMapping.java:185)
neo4j | at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
neo4j | at java.util.stream.DistinctOps$1$2.accept(DistinctOps.java:175)
neo4j | at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
neo4j | at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
neo4j | at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
neo4j | at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
neo4j | at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
neo4j | at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
neo4j | at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.createIndexAndMapping(BaseMapping.java:181)
neo4j | at com.graphaware.module.es.ElasticSearchWriter.createIndexIfNotExist(ElasticSearchWriter.java:220)
neo4j | at com.graphaware.module.es.ElasticSearchWriter.start(ElasticSearchWriter.java:92)
neo4j | at com.graphaware.runtime.module.thirdparty.WriterBasedThirdPartyIntegrationModule.start(WriterBasedThirdPartyIntegrationModule.java:61)
neo4j | at com.graphaware.module.es.ElasticSearchModule.start(ElasticSearchModule.java:87)
neo4j | at com.graphaware.runtime.manager.ProductionTxDrivenModuleManager.start(ProductionTxDrivenModuleManager.java:49)
neo4j | at com.graphaware.runtime.manager.BaseTxDrivenModuleManager.startModules(BaseTxDrivenModuleManager.java:113)
neo4j | at com.graphaware.runtime.TxDrivenRuntime.startModules(TxDrivenRuntime.java:147)
neo4j | at com.graphaware.runtime.ProductionRuntime.startModules(ProductionRuntime.java:70)
neo4j | at com.graphaware.runtime.BaseGraphAwareRuntime.start(BaseGraphAwareRuntime.java:134)
neo4j | at com.graphaware.runtime.bootstrap.RuntimeKernelExtension.lambda$start$0(RuntimeKernelExtension.java:117)
neo4j | at java.lang.Thread.run(Thread.java:748)
neo4j | Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:9200 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
neo4j | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
neo4j | at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
neo4j | at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
neo4j | at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
neo4j | at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
neo4j | at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
neo4j | at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
neo4j | at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
neo4j | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
neo4j | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
neo4j | at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:47)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.createIndexAndMapping(BaseMapping.java:199)
neo4j | at com.graphaware.module.es.mapping.BaseMapping.lambda$createIndexAndMapping$0(BaseMapping.java:183)
neo4j | ... 21 more
neo4j | Caused by: java.net.ConnectException: Connection refused (Connection refused)
neo4j | at java.net.PlainSocketImpl.socketConnect(Native Method)
neo4j | at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
neo4j | at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
neo4j | at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
neo4j | at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
neo4j | at java.net.Socket.connect(Socket.java:589)
neo4j | at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
neo4j | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
neo4j | ... 33 more
neo4j | 2018-11-10 17:28:42.288+0000 INFO Started Elasticsearch Writer.
neo4j | 2018-11-10 17:28:42.289+0000 INFO Transaction-driven modules started.
neo4j | 2018-11-10 17:28:42.289+0000 INFO There are no timer-driven runtime modules. Not scheduling any tasks.
neo4j | 2018-11-10 17:28:42.290+0000 INFO GraphAware started.
neo4j | 2018-11-10 17:28:42.291+0000 INFO [c.g.r.b.RuntimeKernelExtension] GraphAware Runtime automatically started.
neo4j | 2018-11-10 17:28:45.022+0000 INFO Started.
neo4j | 2018-11-10 17:28:45.114+0000 INFO [c.g.s.f.b.GraphAwareServerBootstrapper] started
neo4j | 2018-11-10 17:28:45.115+0000 INFO Mounted unmanaged extension [com.graphaware.server] at [/graphaware]
neo4j | 2018-11-10 17:28:45.296+0000 INFO Google Analytics enabled
neo4j | 2018-11-10 17:28:45.305+0000 INFO Mounting GraphAware Framework at /graphaware
neo4j | 2018-11-10 17:28:45.312+0000 INFO Will try to scan the following packages: {com.**.graphaware.**,org.**.graphaware.**,net.**.graphaware.**}
neo4j | 2018-11-10 17:28:47.382+0000 INFO Remote interface available at http://localhost:7474/
I have created the folders for the docker volumes with the following bash file :
#!/bin/bash
mkdir -p elasticsearch/config elasticsearch/data
mkdir -p neo4j/conf neo4j/data neo4j/plugins neo4j/logs
cp ../docker-utilities/apoc-3.4.0.3-all.jar neo4j/plugins/
cp ../docker-utilities/graphaware-neo4j-to-elasticsearch-3.4.9.52.11.jar neo4j/plugins/
cp ../docker-utilities/graphaware-server-community-all-3.4.9.52.jar neo4j/plugins/
cp ../docker-utilities/neo4j.confmini neo4j/conf/neo4j.conf
cp ../docker-utilities/elasticsearch.yml elasticsearch/config/
This bash file put in the right places the required plugins and two conf files :
elasticsearch.yml :
cluster.name: test_cluster
node.name: "node"
############################## Network And HTTP ###############################
# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens
# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node
# communication. (the range means that if the port is busy, it will automatically
# try the next port).
# Set the bind address specifically (IPv4 or IPv6):
#
#network.bind_host: 192.168.0.1
# Set the address other nodes will use to communicate with this node. If not
# set, it is automatically derived. It must point to an actual IP address.
#
#network.publish_host: 192.168.0.1
# Set both 'bind_host' and 'publish_host':
#
#network.host: 192.168.0.1
# Set a custom port for the node to node communication (9300 by default):
#
#transport.tcp.port: 9300
# Enable compression for all communication between nodes (disabled by default):
#
#transport.tcp.compress: true
# Set a custom port to listen for HTTP traffic:
#
#http.port: 9200
# Set a custom allowed content length:
#
#http.max_content_length: 100mb
# Disable HTTP completely:
#
#http.enabled: false
http.host: 0.0.0.0
network.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: "X-Requested-With, Content-Type, Content-Length, X-User"
xpack.security.enabled: false
and neo4j.conf:
ACCEPT.LICENSE.AGREEMENT=yes
dbms.unmanaged_extension_classes=com.graphaware.server=/graphaware
com.graphaware.runtime.enabled=true
com.graphaware.module.UIDM.initializeUntil=0
#ES becomes the module ID:
com.graphaware.module.ES.2=com.graphaware.module.es.ElasticSearchModuleBootstrapper
#URI of Elasticsearch
com.graphaware.module.ES.uri=localhost
#Port of Elasticsearch
com.graphaware.module.ES.port=9200
Finally the docker compose to link all that :
version: '3.3'
services:
neo4jESdock:
restart: always
image: neo4j:3.4.7
container_name: neo4j
environment:
- NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
- NEO4J_AUTH=none
- NEO4J_dbms_connector_http_listen__address=:7474
- NEO4J_dbms_connector_https_listen__address=:6477
- NEO4J_dbms_connector_bolt_listen__address=:7687
- elastic="http://elastic_container"
volumes:
- ./neo4j/data:/data
- ./neo4j/plugins:/plugins
- ./neo4j/conf:/conf
- ./neo4j/logs:/neologs
ports:
- 7474:7474
- 6477:6477
- 7687:7687
networks:
- neo-elk
elastic:
image: elasticsearch:6.4.2
restart: always
container_name: elastic_container
ports:
- 9200:9200
volumes:
- ./elasticsearch/data:/usr/share/elasticsearch/data
- ./elasticsearch/config/elasticsearch.yml:/etc/elasticsearch/config/elasticsearch.yml
environment:
- "ES_JAVA_OPTS=-Xmx256m -Xms256m"
- neo4j="http://neo4j"
networks:
- neo-elk
networks:
neo-elk:
driver: bridge
And some documentation on similar projects :
https://github.com/graphaware/graph-aided-search-demo
Connect docker-compose files by shared network
https://github.com/mitdbg/aurum-datadiscovery
I'm not very experienced with docker...
docker elasticsearch neo4j graphaware
docker elasticsearch neo4j graphaware
asked Nov 10 at 17:45
machinus
1214
1214
There is a mistake in ES volumes, proper path to config is : - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
– machinus
Nov 11 at 20:08
add a comment |
There is a mistake in ES volumes, proper path to config is : - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
– machinus
Nov 11 at 20:08
There is a mistake in ES volumes, proper path to config is : - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
– machinus
Nov 11 at 20:08
There is a mistake in ES volumes, proper path to config is : - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
– machinus
Nov 11 at 20:08
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
accepted
It seems that neo4j tries to connect to elasticsearch on localhost, whereas the elasticsearch container is on a different ip.
According to https://docs.docker.com/compose/networking/ you can use the name of the service, so instead of localhost you can put elastic.
New contributor
Hello, I've been banging my head on this all day thank you... Indeed, I don't know what docker does with ips but I changed one line in the neo4j.conf file : com.graphaware.module.ES.uri=elastic And it works. TY !
– machinus
Nov 10 at 18:14
I know the feeling :). Glad to be of help
– Andrei Dumitrescu-Tudor
Nov 10 at 18:14
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
accepted
It seems that neo4j tries to connect to elasticsearch on localhost, whereas the elasticsearch container is on a different ip.
According to https://docs.docker.com/compose/networking/ you can use the name of the service, so instead of localhost you can put elastic.
New contributor
Hello, I've been banging my head on this all day thank you... Indeed, I don't know what docker does with ips but I changed one line in the neo4j.conf file : com.graphaware.module.ES.uri=elastic And it works. TY !
– machinus
Nov 10 at 18:14
I know the feeling :). Glad to be of help
– Andrei Dumitrescu-Tudor
Nov 10 at 18:14
add a comment |
up vote
0
down vote
accepted
It seems that neo4j tries to connect to elasticsearch on localhost, whereas the elasticsearch container is on a different ip.
According to https://docs.docker.com/compose/networking/ you can use the name of the service, so instead of localhost you can put elastic.
New contributor
Hello, I've been banging my head on this all day thank you... Indeed, I don't know what docker does with ips but I changed one line in the neo4j.conf file : com.graphaware.module.ES.uri=elastic And it works. TY !
– machinus
Nov 10 at 18:14
I know the feeling :). Glad to be of help
– Andrei Dumitrescu-Tudor
Nov 10 at 18:14
add a comment |
up vote
0
down vote
accepted
up vote
0
down vote
accepted
It seems that neo4j tries to connect to elasticsearch on localhost, whereas the elasticsearch container is on a different ip.
According to https://docs.docker.com/compose/networking/ you can use the name of the service, so instead of localhost you can put elastic.
New contributor
It seems that neo4j tries to connect to elasticsearch on localhost, whereas the elasticsearch container is on a different ip.
According to https://docs.docker.com/compose/networking/ you can use the name of the service, so instead of localhost you can put elastic.
New contributor
New contributor
answered Nov 10 at 17:51
Andrei Dumitrescu-Tudor
874
874
New contributor
New contributor
Hello, I've been banging my head on this all day thank you... Indeed, I don't know what docker does with ips but I changed one line in the neo4j.conf file : com.graphaware.module.ES.uri=elastic And it works. TY !
– machinus
Nov 10 at 18:14
I know the feeling :). Glad to be of help
– Andrei Dumitrescu-Tudor
Nov 10 at 18:14
add a comment |
Hello, I've been banging my head on this all day thank you... Indeed, I don't know what docker does with ips but I changed one line in the neo4j.conf file : com.graphaware.module.ES.uri=elastic And it works. TY !
– machinus
Nov 10 at 18:14
I know the feeling :). Glad to be of help
– Andrei Dumitrescu-Tudor
Nov 10 at 18:14
Hello, I've been banging my head on this all day thank you... Indeed, I don't know what docker does with ips but I changed one line in the neo4j.conf file : com.graphaware.module.ES.uri=elastic And it works. TY !
– machinus
Nov 10 at 18:14
Hello, I've been banging my head on this all day thank you... Indeed, I don't know what docker does with ips but I changed one line in the neo4j.conf file : com.graphaware.module.ES.uri=elastic And it works. TY !
– machinus
Nov 10 at 18:14
I know the feeling :). Glad to be of help
– Andrei Dumitrescu-Tudor
Nov 10 at 18:14
I know the feeling :). Glad to be of help
– Andrei Dumitrescu-Tudor
Nov 10 at 18:14
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53241741%2funable-to-connect-neo4j-and-elasticsearch-in-docker%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
There is a mistake in ES volumes, proper path to config is : - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
– machinus
Nov 11 at 20:08