Akka Typed and protobuf serialization

Akka Libraries
1

I’m wondering if someone has done this or thought about a good approach. I’m thinking serialization both for the purpose of persistence and intra-cluster communication.

My usual approach would be to use a tool that generates the case classes based on the protobuf source, using e.g. https://github.com/thesamet/sbt-protoc. In any instance I don’t want to have to serialize by hand.

Using this approach I would then select different base traits for commands / events of each actor - the traits would be defined in code most likely, whereas the selection can be done by feeding an option (for each message) to scalapb.

What’s bugging me a bit is how to best go with regards to namespacing. Using companion objects isn’t an option with generated classes (I think) and so I’d either have to settle for one big flat namespace (meh) or something odd such as one package per actor.

Thoughts?

Thanks,

Manuel

2

Anyone? :slight_smile:

Come to think of it, I probably would end up having on .proto file per actor / actor protocol (which then defines the namespace) and use some scalapb trickery to have them extend a sealed trait (custom traits can’t be sealed). This can be done using the preamble option (see https://scalapb.github.io/customizations.html).

1 Like
3

Thanks for sharing.

4

The way I would go about this is to strictly separate infra and domain models and not mix both. To facilitate the back&forth, I would use some data mapping lib such as https://github.com/scalalandio/chimney

5

I’m facing the exact same questions as OP, also with an emphasis on the namespacing question.

I think that the desire to use Protobuf is a classic case of YAGNI, as I’m going to explain. It stems from a desire to future-proof the actor messages (not bad in itself). We ask, what if we are going to use the same message format later with some other service? What if we send the messages over the network, isn’t protobuf faster then? Protobuf all the things.

I have concluded that using Protobuf as primary source for message definitions is not feasible, because of the way it disintegrates the code, and adds development overhead. Actor messages must be defined closely to the Actor itself, i.e. in the companion object, and using a sealed trait. Having the messages defined in a separate (protobuf) directory structure, far removed from the actor code, and in a different, broader namespace is a price too high to pay. The case would be different if Protobuf messages could be defined inline inside of Scala code, similar to vue.js single file components.

What is needed is really a fast serialization mechanism for Scala case classes.

If Protobuf is needed for communication with other services, then that’s where Protobuf should be used - but not as a universal replacement for all actor messages. The previous poster’s link seems to be helpful in this regard. Using protobuf everywhere “just in case” is a case of YAGNI, introducing unnecessary complexity, clumsy development and non-transparency.

If you only use Protobuf where you actually need it, i.e. where it makes a positive difference, you won’t run into the above problems.

1 Like
6

@ignatius Note that re-use protocols between services is not what Manuel is asking about but specifically generating messages from protobuf for use with typed where you need a shared supertype for the generated class and getting the messages nicely in a namespace for the actor like one would usually do with MyActor.SomeMessage.

Following that detour from the original topic:

The main benefit for protobuf in general is to be able to handle wire compatibility for messages to allow for rolling upgrades of a cluster where old and new nodes can still communicate even to messages has changed and read persisted events that was written from an older version of the application. APIs for the outside world should pretty much always be strictly defined separately from the internal serialization,

Since Akka 2.6 we have the akka-serialization-jackson which is relatively fast, uses reflection so requires low up front developer overhead and supports binary formats (CBOR). I think that checks your wish list for serialization of Scala case classes. See docs here…

2 Likes
7

I asked the author of ScalaPB for his input and he was kind enough to give a helpful reply: Message case classes inside of common object · Issue #751 · scalapb/ScalaPB · GitHub

The easiest thing to do would just to wrap the messages in your proto files within some other message:

message SomeActorProtocol {
  Message A {}
  Message B {}
}

The case classes for A and B will be generated inside an object named SomeActorProtocol.

Sealed traits for the messages are supported with scalapb’s preamble setting (see the docs)

Ping OP @manuelbernhardt

1 Like