Awesome
Real-time Map
Real-time Map displays real-time positions of public transport vehicles in Helsinki. It's a showcase for Proto.Actor - an ultra-fast distributed actors solution for Go, C#, and Java/Kotlin.
🔥 Check out the live demo! 🔥
The app features:
- Real-time positions of vehicles.
- Vehicle trails.
- Geofencing notifications (vehicle entering and exiting the area).
- Vehicles in geofencing areas per public transport company.
- Horizontal scaling.
The goals of this app are:
- Showing what Proto.Actor can do.
- Presenting a semi-real-world use case of the distributed actor model.
- Aiding people in learning how to use Proto.Actor.
Find more about Proto.Actor here.
Running the app
Configure Mapbox:
- Create an account on Mapbox.
- Copy a token from: main dashboard / access tokens / default public token.
- Paste the token in
Frontend\src\config.ts
.
Start frontend (requires node.js 17.x):
cd Frontend
npm install
npm run serve
Start Backend:
cd Backend
dotnet run
The app is available on localhost:8080.
Also check out the proto.actor dashboard (alpha) by navigating to localhost:5000.
Kubernetes
In order to deploy to Kubernetes and use the Kubernetes cluster provider, see Deploying to Kubernetes
How does it work?
Prerequisites
To understand how this app works, it's highly recommended to understand the basics of the following technologies:
It would also be great, if you knew the basics the of actor model, Proto.Actor, and virtual actors (also called grains). If you don't you can try reading this document anyway: we'll try to explain the essential parts as we go.
Learn more about Proto.Actor here.
Learn more about virtual actors here.
Also, since this app aims to provide horizontal scalability, this document will assume, that we're running a cluster with two nodes.
One last note: this document is not a tutorial, but rather a documentation of this app. If you're learning Proto.Actor, you'll benefit the most by jumping between code and this document.
Data source
Since this app is all about tracking vehicles, we need to get their positions from somewhere. In this app, the positions are received from high-frequency vehicle positioning MQTT broker from Helsinki Region Transport. More info on data:
This data is licensed under © Helsinki Region Transport 2021, Creative Commons BY 4.0 International
In our app, Ingress
is a hosted service responsible for subscribing to Helsinki Region Transport MQTT server and handling vehicle position updates.
Vehicles
When creating a system with actors, is common to model real-world physical objects as actors. We'll start by modelling a vehicle since all the features depend on it. It will be implemented as a virtual actor (VehicleActor
). It will be responsible for handling events related to that vehicle and remembering its state, e.g. its current position and position history.
Quick info on virtual actors:
- Each virtual actor is of a specific kind, in this case, it's
Vehicle
. - Each virtual actor has an ID, in this case, it's an actual vehicle's number.
- Virtual actors can have state, in this case, vehicle's current position and position history.
- Virtual actors handle messages sent to them one by one, meaning we don't have to worry about synchronization.
- Virtual actors are distributed in the cluster. In normal circumstances, a single virtual actor (in this case a specific vehicle) will be hosted on a single node.
- We don't need to spawn (create) virtual actors explicitly. They will be spawned automatically on one of the nodes the first time a message is sent to them.
- To communicate with a virtual actor we only need to know its kind and ID. We don't have to care about which node it's hosted on.
Note: virtual actors are sometimes referred to as "grains" - terminology originating in the MS Orleans project.
The workflow looks like this:
Ingress
receives an event from Helsinki Region Transport MQTT server.Ingress
reads vehicle's ID and its new position from the event and sends it to a vehicle in question.VehicleActor
processes the message.
Notice, that in the above diagram, vehicles (virtual actors) are distributed between two nodes, however, Ingress
is present in both of the nodes.
Organizations
Let's consider the following feature: in this app, each vehicle belongs to an organization. Each organization has a specified list of geofences. In our case, a geofence is simply a circular area somewhere in Helsinki, e.g. airport, railway square, or downtown. Users should receive notifications when a vehicle enters or leaves a geofence (from that vehicle's organization).
For that purpose, we'll implement a virtual actor to model an organization (OrganizationActor
). When activated, OrganizationActor
will spawn (create) a child actor for each configured geofence (GeofenceActor
).
Quick info on child actors:
- In contrast to virtual actors, we need to manually spawn a child actor.
- Child actor is hosted on the same node as the virtual actor that spawned it. Communication in the same node is quite fast, as neither serialization nor remote calls are needed.
- After spawning a child actor, its PID is stored in the parent actor. Think of it as a reference to an actor: you can use it to communicate with it.
- Since parent actor has PIDs of their child actors, it can easily communicate with them.
- Technically, we can communicate with a child actor using their PID from anywhere within the cluster. This functionality is not utilized in this app, though.
- Child actor lifecycle is bound to the parent that spawned it. It will be stopped when parent gets stopped.
The workflow looks like this:
VehicleActor
receives a position update.VehicleActor
forwards position update to its organization.OrganizationActor
forwards position update to all its geofences.- Each
GeofenceActor
keeps track of which vehicles are already inside the zone. Thanks to this,GeofenceActor
can detect if a vehicle entered or left the geofence.
Viewport, positions and notifications
So far we've only sent messages to and between actors. However, if we want to send notifications to actual users, we need to find a way to communicate between the actor system and the outside world. In this case, we'll use:
- SignalR to push notifications to users.
- Proto.Actor's (experimental) pub-sub feature to broadcast positions cluster-wide.
First we'll introduce the UserActor
. It models the user's ability to view positions of vehicles and geofencing notifications. UserActor
will be implemented as an actor (i.e. non-virtual actor).
Quick info on actors:
- An actor's lifecycle is not managed, meaning we have to manually spawn and stop them.
- An actor will be hosted on the same node that spawned it. Like with child actors, this gives us performance benefits when communicating with an actor. It also enables the actor to manage a local resource (SignalR connection).
- Since an actor lives in the same node (i.e. the same process), we can pass .NET references to it. It will come in handy in this feature.
- After spawning an actor, we receive its PID. Think of it as a reference to an actor: you can use it to communicate with it. Don't lose it!
- Technically, we can communicate with an actor using their PID from anywhere within the cluster. This functionality is not utilized in this app, though.
The workflow looks like this:
- When user connects to SignalR hub, user actor is spawned to represent the connection. Delegates to send the positions and notifications back to the user are provided to the actor upon creation.
- When starting, the user actor subscribes to
Position
andNotification
events being broadcasted through the Pub-Sub mechanism. It also makes sure to unsubscribe when stopping. - When user pans the map, updates of the viewport (south-west and north-east coords of the visible area on the map) are sent to the user actor through SignalR connection. The actor keeps track of the viewport to filter out positions.
- Geofence actor detects vehicle entering or leaving the geofencing zone. These events are broadcasted as
Notification
message to all cluster members and available through PubSub. Same goes forPosition
events broadcasted from vehicle actor. - When receiving a
Notification
message, user actor will push it to the user via SignalR connection. It will do the same withPosition
message provided it is within currently visible area on the map. The positions are sent in batches to improve performance.
Map grid
In the Realtime map sample we also showcase an optimization, which is possible thanks to the virtual actors and the pub-sub mechanism. Instead of broadcasting all positions to all users, we can limit the amount of positions sent to just the ones a particular user has interest in. For this purpose we overlay a "virtual grid" on top of the map.
Each cell in the map becomes a topic in the pub sub system. Topics are implemented as virtual actors so we don't need to create them in an explicit way. We can just start sending messages to them. In the example above we would have a topic for grid cell A1, B1, etc.
Depending on the size and position of the user's viewport, it covers some of the grid cells. This means the user actor needs to subscribe to topics corresponding to the covered grid cells. In the example above, the user is subscribed to topics A1, B1, A2 and B2. When the viewport moves to cover different set of grid cells, the topics corresponding to grid cells that are no longer relevant need to be unsubscribed.
On publishing side, the vehicle calculates what topic to publish to based on current position. Only the topic corresponding to the grid cell the vehicle is currently in receives the position.
This way we're removing a bottleneck in the system that would otherwise force us to send all the messages to all the users, which is not a scalable approach. You could argue, that if the viewport covers all the grid cells, the problem occurs anyway. This is true, but we could prevent users from doing so, for example by stopping the real time updates when the viewport is over some size threshold.
Getting vehicles currently in an organization's geofences
Let's consider the following feature: when a user requests it, we want to list all vehicles currently present in a selected organization's geofences.
This one is quite easy, as actors support request/response pattern.
The workflow looks like that:
- A user calls API to get organization's details (including geofences and vehicles in them).
OrganizationController
asksOrganizationActor
for the details.OrganizationActor
asks each of its geofences (GeofenceActor
) for a list of vehicles in these geofences.- Each
GeofenceActor
returns that information. OrganizationActor
combines that information into a response and returns it toOrganizationController
.OrganizationController
maps and returns the results to the user.
Deploying to Kubernetes
Prerequisites:
- kubectl with configured connection to your cluster, e.g. Docker desktop
- Helm
- nginx ingress controller configured on your cluster
- Redis - used by the pub-sub mechanism
The Realtime Map sample can be configured to use Kubernetes as cluster provider. The attached chart contains definition for the deployment. You will need to supply some additional configuration, so create a new values file, e.g. my-values.yaml
in the root directory of the sample. Example contents:
frontend:
config:
mapboxToken: SOME_TOKEN # provide your MapBox token here
backend:
config:
# since all backend pods need to share MQTT subscription,
# generate a new guid and provide it here (no dashes)
RealtimeMap__SharedSubscriptionGroupName: SOME_GUID
ProtoActor__PubSub__RedisConnectionString: realtimemap-redis-master:6379
ingress:
# specify localhost if on Docker Desktop,
# otherwise add a domain for the sample to your DNS and specify it here
host: localhost
Create the namespace and deploy the sample:
helm upgrade --install -f my-values.yml --namespace realtimemap --create-namespace realtimemap ./chart
NOTE: the chart creates a new role in the cluster with permissions to access Kubernetes API required for the cluster provider.
The above config will deploy the sample without TLS on the ingress, additional configuration is required to secure the connection.