Hi,
I have exported akka-persistance-cassandra database and loaded on another machine. When I send a new command with a new shard id or existing shard id, an entity doesn’t receiving the command. I did export/load using cassnadra COPY/FROM command. I can see it is hitting shard id extractor and entity id extractor.
Please, could anyone help me what could be the issue?
Thanks.
I got below error:
2709/09 09:27:59 ERROR[ngage-actor-system-akka.actor.default-dispatcher-22] a.c.s.PersistentShardCoordinator - Persistence failure when replaying events for persistenceId [/sharding/GroupEntityCoordinator]. Last known sequence number [0]
java.lang.NullPointerException: null
at akka.persistence.journal.WriteJournalBase.adaptFromJournal(WriteJournalBase.scala:27)
at akka.persistence.journal.WriteJournalBase.adaptFromJournal$(WriteJournalBase.scala:26)
at akka.persistence.cassandra.journal.CassandraJournal.adaptFromJournal(CassandraJournal.scala:54)
at akka.persistence.journal.AsyncWriteJournal$$anonfun$receiveWriteJournal$1.$anonfun$applyOrElse$15(AsyncWriteJournal.scala:147)
at akka.persistence.journal.AsyncWriteJournal$$anonfun$receiveWriteJournal$1.$anonfun$applyOrElse$15$adapted(AsyncWriteJournal.scala:145)
at akka.stream.impl.fusing.Map$$anon$1.onPush(Ops.scala:54)
at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:523)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:409)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:606)
at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:485)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:581)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:749)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$shortCircuitBatch(ActorGraphInterpreter.scala:739)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:765)
at akka.actor.Actor.aroundReceive(Actor.scala:539)
at akka.actor.Actor.aroundReceive$(Actor.scala:537)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:671)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:612)
at akka.actor.ActorCell.invoke(ActorCell.scala:581)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:268)
at akka.dispatch.Mailbox.run(Mailbox.scala:229)
at akka.dispatch.Mailbox.exec(Mailbox.scala:241)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Maybe you can verify that all data was included by running some cqlsh queries on the messages table.
select * from akka.messages where persistence_id='<a specific persistence id>' and partition_nr=1;
does it look the same as in the original database?
Yes, I can see the entry with same persistence id and partition_nr but for me, partition_nr is 0.
Ok, it probably starts at 0. All fields the same in old and new db?
Yeah. I checked entries are correct. Below are the debug logs.
12:30:00.163 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-31] a.p.c.journal.CassandraJournal - [/sharding/UserFieldUniquenessCheckEntityCoordinator] asyncReplayMessages from [1] to [2]
12:30:00.165 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-3] a.p.c.journal.CassandraJournal - [/sharding/UserFieldUniquenessCheckEntityCoordinator] Recovery sending tag progress: [Map()]
12:30:00.166 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-3] a.p.cassandra.journal.TagWriters - Pid [/sharding/UserFieldUniquenessCheckEntityCoordinator] set tag progress [Map()]. Tags to reset as not in progress: [Set()]
12:30:00.166 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-3] a.p.c.journal.CassandraJournal - [/sharding/UserFieldUniquenessCheckEntityCoordinator] Recovery is starting before the latest tag writes tag progress. Min progress [1]. From sequence nr of recovery: [1]
12:30:00.166 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-3] a.p.c.journal.CassandraJournal - [/sharding/UserFieldUniquenessCheckEntityCoordinator] starting recovery with tag progress: [Map()]. From [1] to [2]
12:30:00.166 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-31] a.p.c.q.s.CassandraReadJournal - Creating EventByPersistentIdState graph
12:30:00.169 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-31] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/UserFieldUniquenessCheckEntityCoordinator] Query from seqNr [1] in partition [0]
12:30:00.171 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-31] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/UserFieldUniquenessCheckEntityCoordinator] Query took [1] ms
12:30:00.173 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-28] a.p.cassandra.journal.TagWriters - Persistent actor terminated [Actor[akka://ngage-actor-system/system/sharding/UserFieldUniquenessCheckEntityCoordinator/singleton/coordinator#1767673975]]. Informing TagWriter actors to drop state for pid: [/sharding/UserFieldUniquenessCheckEntityCoordinator]
12:30:00.769 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-28] a.p.c.journal.CassandraJournal - [/sharding/UserEntityCoordinator] asyncReadHighestSequenceNr [0] [Actor[akka://ngage-actor-system/system/sharding/UserEntityCoordinator/singleton/coordinator#-611062645]]
12:30:00.774 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-28] a.p.c.journal.CassandraJournal - [/sharding/UserEntityCoordinator] Persistent actor starting [Actor[akka://ngage-actor-system/system/sharding/UserEntityCoordinator/singleton/coordinator#-611062645]]
12:30:00.774 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-28] a.p.cassandra.journal.TagWriters - Watching pid [/sharding/UserEntityCoordinator] actor [Actor[akka://ngage-actor-system/system/sharding/UserEntityCoordinator/singleton/coordinator#-611062645]]
12:30:00.774 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-28] a.p.c.journal.CassandraJournal - [/sharding/UserEntityCoordinator] asyncReplayMessages from [1] to [2]
12:30:00.777 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-37] a.p.c.journal.CassandraJournal - [/sharding/UserEntityCoordinator] Recovery sending tag progress: [Map()]
12:30:00.777 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-37] a.p.cassandra.journal.TagWriters - Pid [/sharding/UserEntityCoordinator] set tag progress [Map()]. Tags to reset as not in progress: [Set()]
12:30:00.778 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-37] a.p.c.journal.CassandraJournal - [/sharding/UserEntityCoordinator] Recovery is starting before the latest tag writes tag progress. Min progress [1]. From sequence nr of recovery: [1]
12:30:00.778 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-37] a.p.c.journal.CassandraJournal - [/sharding/UserEntityCoordinator] starting recovery with tag progress: [Map()]. From [1] to [2]
12:30:00.778 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-37] a.p.c.q.s.CassandraReadJournal - Creating EventByPersistentIdState graph
12:30:00.781 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-28] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/UserEntityCoordinator] Query from seqNr [1] in partition [0]
12:30:00.782 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-28] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/UserEntityCoordinator] Query took [1] ms
12:30:00.784 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-3] a.p.cassandra.journal.TagWriters - Persistent actor terminated [Actor[akka://ngage-actor-system/system/sharding/UserEntityCoordinator/singleton/coordinator#-611062645]]. Informing TagWriter actors to drop state for pid: [/sharding/UserEntityCoordinator]
12:30:07.329 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-31] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] asyncReadHighestSequenceNr [0] [Actor[akka://ngage-actor-system/system/sharding/GroupEntityCoordinator/singleton/coordinator#-628561555]]
12:30:07.336 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-31] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] Persistent actor starting [Actor[akka://ngage-actor-system/system/sharding/GroupEntityCoordinator/singleton/coordinator#-628561555]]
12:30:07.336 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-31] a.p.cassandra.journal.TagWriters - Watching pid [/sharding/GroupEntityCoordinator] actor [Actor[akka://ngage-actor-system/system/sharding/GroupEntityCoordinator/singleton/coordinator#-628561555]]
12:30:07.336 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-31] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] asyncReplayMessages from [1] to [2]
12:30:07.339 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] Recovery sending tag progress: [Map()]
12:30:07.339 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.cassandra.journal.TagWriters - Pid [/sharding/GroupEntityCoordinator] set tag progress [Map()]. Tags to reset as not in progress: [Set()]
12:30:07.339 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] Recovery is starting before the latest tag writes tag progress. Min progress [1]. From sequence nr of recovery: [1]
12:30:07.339 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] starting recovery with tag progress: [Map()]. From [1] to [2]
12:30:07.339 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.q.s.CassandraReadJournal - Creating EventByPersistentIdState graph
12:30:07.344 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/GroupEntityCoordinator] Query from seqNr [1] in partition [0]
12:30:07.346 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/GroupEntityCoordinator] Query took [2] ms
12:30:07.348 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.cassandra.journal.TagWriters - Persistent actor terminated [Actor[akka://ngage-actor-system/system/sharding/GroupEntityCoordinator/singleton/coordinator#-628561555]]. Informing TagWriter actors to drop state for pid: [/sharding/GroupEntityCoordinator]
Hi @patriknw do you think setting - cassandra-query-journal.gap-free-sequence-numbers = off
would help here? I saw you suggesting this in another thread where the issue was with RemoveInternalClusterShardingData
The reason I was asking for verifying that the transfer was correct was that the null pointer indicates that something is missing in the messages table.
gap-free-sequence-numbers is something else.
BTW, which version of akka-persistence-cassandra is this? Which version was used to write the data and which version is used now?
I am using the same akka-persistence-cassandra version 0.98 for while importing and exporting tables. Yes, transfer data is correct.
Let me explain bit more,
1- I have seed node that doesn’t host any entity and I have another node that host three entities. I also have another node, from there I am sending a commend to create an entity and I checked that entry has created successfully. I can see entries related to created entity.
2-I did stop everything including seed nodes.
3- I have script that exports all tables from akka and akka_snapshot keyspace.
4- I have removed previously created akka and akka_snapshot.
5- Loaded akka and akka_snapshots schema and data as well.
6- I started seed node, entity node and another node where there I was sending a commend to create entity
7- Now, I am sending command with same persistence id and got below error(I already posted same error)
12:56:32.875 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-22] a.p.cassandra.journal.TagWriters - Pid [/sharding/GroupEntityCoordinator] set tag progress [Map()]. Tags to reset as not in progress: [Set()]
12:57:15.043 TRACE [ngage-actor-system-akka.actor.default-dispatcher-25] ngage.user.UserEntity - assigning state handler Init
12:57:15.044 TRACE [ngage-actor-system-akka.actor.default-dispatcher-25] ngage.user.UserEntity - assigning state handler Active
12:57:15.044 TRACE [ngage-actor-system-akka.actor.default-dispatcher-25] ngage.user.UserEntity - assigning state handler Inactive
12:57:15.045 TRACE [ngage-actor-system-akka.actor.default-dispatcher-25] ngage.user.UserEntity - assigning state handler Deleted
12:57:15.102 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-25] a.p.c.journal.CassandraJournal - [User_0013] asyncReadHighestSequenceNr [0] [Actor[akka://ngage-actor-system/system/sharding/UserEntity/1/User_0013#151794441]]
12:57:15.108 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-25] a.p.c.journal.CassandraJournal - [User_0013] Persistent actor starting [Actor[akka://ngage-actor-system/system/sharding/UserEntity/1/User_0013#151794441]]
12:57:15.109 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-25] a.p.cassandra.journal.TagWriters - Watching pid [User_0013] actor [Actor[akka://ngage-actor-system/system/sharding/UserEntity/1/User_0013#151794441]]
12:57:15.110 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-25] a.p.c.journal.CassandraJournal - [User_0013] asyncReplayMessages from [1] to [4]
12:57:15.135 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.journal.CassandraJournal - [User_0013] Recovery sending tag progress: [Map()]
12:57:15.137 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.cassandra.journal.TagWriters - Pid [User_0013] set tag progress [Map()]. Tags to reset as not in progress: [Set()]
12:57:15.142 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.journal.CassandraJournal - [User_0013] Recovery is starting before the latest tag writes tag progress. Min progress [1]. From sequence nr of recovery: [1]
12:57:15.143 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.journal.CassandraJournal - [User_0013] starting recovery with tag progress: [Map()]. From [1] to [4]
12:57:15.157 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-25] a.p.c.q.s.CassandraReadJournal - Creating EventByPersistentIdState graph
12:57:15.189 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [User_0013] Query from seqNr [1] in partition [0]
12:57:15.192 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-20] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [User_0013] Query took [3] ms
12:57:15.211 ERROR [ngage-actor-system-akka.actor.default-dispatcher-25] ngage.user.UserEntity - Persistence failure when replaying events for persistenceId [User_0013]. Last known sequence number [0]
java.lang.NullPointerException: null
at akka.persistence.journal.WriteJournalBase.adaptFromJournal(WriteJournalBase.scala:27)
at akka.persistence.journal.WriteJournalBase.adaptFromJournal$(WriteJournalBase.scala:26)
at akka.persistence.cassandra.journal.CassandraJournal.adaptFromJournal(CassandraJournal.scala:54)
at akka.persistence.journal.AsyncWriteJournal$$anonfun$receiveWriteJournal$1.$anonfun$applyOrElse$15(AsyncWriteJournal.scala:147)
at akka.persistence.journal.AsyncWriteJournal$$anonfun$receiveWriteJournal$1.$anonfun$applyOrElse$15$adapted(AsyncWriteJournal.scala:145)
at akka.stream.impl.fusing.Map$$anon$1.onPush(Ops.scala:54)
at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:523)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:409)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:606)
at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:485)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:581)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:749)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$shortCircuitBatch(ActorGraphInterpreter.scala:739)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:765)
at akka.actor.Actor.aroundReceive(Actor.scala:539)
at akka.actor.Actor.aroundReceive$(Actor.scala:537)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:671)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:612)
at akka.actor.ActorCell.invoke(ActorCell.scala:581)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:268)
at akka.dispatch.Mailbox.run(Mailbox.scala:229)
at akka.dispatch.Mailbox.exec(Mailbox.scala:241)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
12:57:15.220 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-25] a.p.cassandra.journal.TagWriters - Persistent actor terminated [Actor[akka://ngage-actor-system/system/sharding/UserEntity/1/User_0013#151794441]]. Informing TagWriter actors to drop state for pid: [User_0013]
^_12:57:29.785 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-4] a.p.cassandra.journal.TagWriters - Update tag scanning [(/sharding/UserFieldUniquenessCheckEntityCoordinator,1),(/sharding/UserEntityCoordinator,1),(/sharding/GroupEntityCoordinator,1)]
12:57:29.800 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-4] a.p.cassandra.journal.TagWriters - Update tag scanning of [3] pids took [14] ms
For minimising the problem, I also try the below scenario:
1- I have a seed node that doesn’t host any entity and I have another node that host three entities. This time, I haven’t created any entity.
2-I did stop everything including seed nodes.
3- I have a script that exports all tables from akka and akka_snapshot keyspace.
4- I have removed previously created akka and akka_snapshot .
5- Loaded akka and akka_snapshots schema and data as well.
6- I started seed node, entity node and another node where there I was sending a command to create an entity
7- Now, I am sending a command and getting below the debug log and no entity has created, those logs seems to retry logs.
14:05:30.545 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.journal.CassandraJournal - [/sharding/UserEntityCoordinator] Recovery is starting before the latest tag writes tag progress. Min progress [1]. From sequence nr of recovery: [1]
14:05:30.545 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.journal.CassandraJournal - [/sharding/UserEntityCoordinator] starting recovery with tag progress: [Map()]. From [1] to [6]
14:05:30.546 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.q.s.CassandraReadJournal - Creating EventByPersistentIdState graph
14:05:30.549 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-32] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/UserEntityCoordinator] Query from seqNr [1] in partition [0]
14:05:30.550 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-32] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/UserEntityCoordinator] Query took [2] ms
14:05:30.554 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-22] a.p.cassandra.journal.TagWriters - Persistent actor terminated [Actor[akka://ngage-actor-system/system/sharding/UserEntityCoordinator/singleton/coordinator#1872241626]]. Informing TagWriter actors to drop state for pid: [/sharding/UserEntityCoordinator]
14:05:36.759 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-32] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] asyncReadHighestSequenceNr [0] [Actor[akka://ngage-actor-system/system/sharding/GroupEntityCoordinator/singleton/coordinator#-1144391645]]
14:05:36.764 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-32] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] Persistent actor starting [Actor[akka://ngage-actor-system/system/sharding/GroupEntityCoordinator/singleton/coordinator#-1144391645]]
14:05:36.764 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-32] a.p.cassandra.journal.TagWriters - Watching pid [/sharding/GroupEntityCoordinator] actor [Actor[akka://ngage-actor-system/system/sharding/GroupEntityCoordinator/singleton/coordinator#-1144391645]]
14:05:36.764 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-32] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] asyncReplayMessages from [1] to [4]
14:05:36.767 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] Recovery sending tag progress: [Map()]
14:05:36.767 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.cassandra.journal.TagWriters - Pid [/sharding/GroupEntityCoordinator] set tag progress [Map()]. Tags to reset as not in progress: [Set()]
14:05:36.767 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] Recovery is starting before the latest tag writes tag progress. Min progress [1]. From sequence nr of recovery: [1]
14:05:36.768 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.journal.CassandraJournal - [/sharding/GroupEntityCoordinator] starting recovery with tag progress: [Map()]. From [1] to [4]
14:05:36.768 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.q.s.CassandraReadJournal - Creating EventByPersistentIdState graph
14:05:36.770 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/GroupEntityCoordinator] Query from seqNr [1] in partition [0]
14:05:36.772 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-24] a.p.c.q.EventsByPersistenceIdStage - EventsByPersistenceId [/sharding/GroupEntityCoordinator] Query took [1] ms
14:05:36.775 DEBUG [ngage-actor-system-akka.actor.default-dispatcher-34] a.p.cassandra.journal.TagWriters - Persistent actor terminated [Actor[akka://ngage-actor-system/system/sharding/GroupEntityCoordinator/singleton/coordinator#-1144391645]]. Informing TagWriter actors to drop state for pid: [/sharding/GroupEntityCoordinator]
Hi @patriknw ,
I have created a simple project by copying akka-sample-persistence-scala from https://github.com/akka/akka-samples/tree/2.5/akka-sample-persistence-scala to produce similar issue. Here is a sample project https://github.com/abdheshkumar/akka-sample-persistence.
Steps to run:
1- src/main/resources/docker-compose up
2- Run PersistentActorExample.scala
3- src/main/resources/cassandra-import.sh //This will import tables on local machine from docker
4- src/main/resources/docker-compose down //For cleaning up
5 - src/main/resources/load.sh //Load cassandra tables into cassnadra
6- Re-run PersistentActorExample.scala
7. Will see an error
Thanks for putting together the example. I couldn’t run the scripts, various problems and I don’t have time to investigate. It must be something wrong or missing with the copy if it doesn’t work after export/import. I hope you can figure it out.