Hi all,
I have a multi-jvm integration test that involves 3 JVMs and runs several tests.
Now, these 3 JVMs need to communicate with an “external” server.
Currently, i start this server beforehand, then run the integration test, and after the tests i close the server.
To avoid starting and closing the server manually i want to let the multi-jvm test start by running the server, perform the multi-jvm tests, and then close the server. However, the JVMs cannot connect to the server when it is started from one of the JVMs spawned by the multi-jvm test.
The server is written using akka-http and accepts incoming websocket requests:
object DistributedLockServer {
implicit val system = ActorSystem("DistributedLocks")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
val route = path("ws-lock") {
parameter(Symbol("user")) { user =>
handleWebSocketMessages(WebSocket.listen(user))
}
}
def main(args: Array[String]): Unit = {
val bindingFuture = Http().bindAndHandle(route, "0.0.0.0", 8080)
println(s"Running server on port $port\nPress RETURN to stop...")
StdIn.readLine()
bindingFuture
.flatMap(_.unbind())
.onComplete(_ => system.terminate())
}
}
object WebSocket {
def listen(user: String): Flow[Message, Message, _] = {
println(s"User connected: $user")
val inbound: Sink[Message, Any] = Sink.foreach(msg => handleMessage(msg, user))
val outbound: Source[Message, SourceQueueWithComplete[Message]] = Source.queue[Message](16, OverflowStrategy.fail)
Flow.fromSinkAndSourceMat(inbound, outbound)((_, outboundMat) => {
clientConnections += user -> outboundMat.offer
NotUsed
})
}
private def handleMessage(msg: Message, user: String): Unit = {
// ...
}
}
The multi-jvm test runs 3 JVMs and one of them first starts the server.
However, the other JVMs cannot connect to the server:
class DistributedStoreMultiJvmNode1 extends DistributedStoreTest
class DistributedStoreMultiJvmNode2 extends DistributedStoreTest
class DistributedStoreMultiJvmNode3 extends DistributedStoreTest
class DistributedStoreTest extends MultiNodeSpec(ThreeNodeConfig) with ImplicitSender with ScalaTestMultiNodeSpec {
import ThreeNodeConfig._
// tell the testkit how many nodes we expect to participate at first
override def initialParticipants = 3
"The distributed store" must {
"start the lock server" in within(5.minutes) {
runOn(node1) {
val serverThread = new Thread {
override def run: Unit = {
DistributedLockServer.main(Array())
}
}
serverThread.start()
}
Thread.sleep(5*1000*60) // 5 minutes
}
}
}
object ThreeNodeConfig extends MultiNodeConfig {
val node1 = role("Alice")
val node2 = role("Bob")
val node3 = role("Charlie")
testTransport(on = true)
// common configuration for all nodes
commonConfig(ConfigFactory.parseString(
"""
|akka.loglevel=INFO
|akka.actor.provider = cluster
|akka.actor.allow-java-serialization = on
|akka.remote.artery.enabled = on
|akka.coordinated-shutdown.run-by-jvm-shutdown-hook = off
|akka.coordinated-shutdown.terminate-actor-system = off
|akka.cluster.run-coordinated-shutdown-when-down = off
""".stripMargin))
}
In this example i start the server and wait 5 minutes to give me time to manually debug the problem.
I then try to connect to the server using the wsc terminal client for websockets:
$ wsc ws://127.0.0.1:8080/ws-lock
Error: connect ECONNREFUSED 127.0.0.1:8080
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1113:14)
Emitted 'error' event at:
at ClientRequest.onerror (/usr/local/lib/node_modules/wsc/node_modules/ws/lib/WebSocket.js:711:10)
at ClientRequest.emit (events.js:182:13)
at Socket.socketErrorListener (_http_client.js:391:9)
at Socket.emit (events.js:182:13)
at emitErrorNT (internal/streams/destroy.js:82:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
at process._tickCallback (internal/process/next_tick.js:63:19)
So, the server refuses the connection, but i don’t know why.
When i run a regular unit test (with ScalaTest) i can connect to the server,
but for some reason the connection is refused in the multi-jvm test.
Is there some configuration i need to do to tell the multi-jvm test to allow connections on port 8080?