Problem Definition
I have a bunch of tasks to be done as part of a job. The jobs will be defined by JSON configuration files and they can have any number of tasks. All the tasks will be run asynchronously (mostly I/O). There is a bunch of different types of actors in the system, each of which are capable of handling a particular task. The tasks will have to be executed sequentially.
Proposed Solution
A single actor which will be the entry point into the system, which will receive the job request and construct a List of tasks. The constructed tasks will have all the information they need to work with, based on the job configuration. The entry point actor will determine the actor who can handle the first task, based on the type of the task and route the List of tasks to it. That actor will complete the task and then determine the actor who can handle the second task and route the rest of the tasks in the list to it. The process is repeated till the list of tasks is empty. If there is an error in one of the steps, the message will be sent to an actor which is capable of handling failures. This approach allows individual tasks to be retried in case of failures. And resumption of the tasks is very easy (if an actor was not able to complete the task at any point of time, we don’t have to start processing from the beginning), as the tasks yet to be done will be still there in the list.
Sample Job configuration
{
"tasks": [
{
....
"taskType": "type",
"config": {...}
},
{
....
"taskType": "type",
"config": {...}
},
...
]
}
Questions
- Is there any name for this pattern of message processing?
- Are there potential problems in this solution?
- Is there a better way to solve this problem?