🎃 Create a Minimalist Event Sourcing for Activity Logs in Laravel Apps

Hey guys,

Happy New Year 2024! I hope you all have a good one.

It is my first blogging for this year too, so let's get started 😎

I'm gonna share with you all the minimalist Event Sourcing that I've applied and used in many of my Laravel apps.

Lovely requirement notes: your Laravel version doesn't matter, every Laravel version supports the following:

Event
Eloquent/Database

That's all we need 🥹.

Event Sourcing

Event Sourcing (ES) architecture is one of the most popular architectures in the world.

Some notable points are:

Stuff talks using Events
Maintaining an Event Stream that holds all events of the application
- Every state changes and every action will be recorded
...

Some are applying ES 100%, some are adapting ES just for the Activity Logs.

And we are targeting the second half in this topic 😆
- I might write another one for the first half hehe 👀

So, let's jump into the technical details 🖥️

The Event Store table

The source of truth - the Event Stream of the application, where all Events/Activity Logs are stored.

Table name

MySQL users: events
PGSQL users: event_store

Base structure

id (bigint)
ulid (string - 26 chars - unique)
- I prefer using ulid nowadays, it's cool, cleaner than UUID, and sortable (ascending by default)
name (string): The event name
payload (jsonb): The event payload
metadata (jsonb): The event metadata
- Metadata could be anything related to the requests, which might help you to debug, e.g.:
  - IP Address
  - Request ID
  - User-agent
  - ...
created_at (timestamp)
updated_at (timestamp)
- One of the rules of ES is that events are immutable. But in real life, we just can't always ensure things are going well.
- Having an updated_at column would help us indicate if the events have been changed in the past.

Depending on your use case, you might want to introduce entity-related keys, e.g.:

user_id
business_id
payment_id
...

Notes:

We mustn't add Foreign Key (FK) indexes, ES must be able to store ALL data, even if the data is removed/truncated. Adding FKs will bite us at a later stage.
Nullable by default.

Reasons why I chose dedicated keys over model_id & model_type (morph):

It's more explicit and easy to understand for readers ✅
It is wayy more scalable ✅

P/s: don't forget to create the Event eloquent model afterward 😆.

Bootstrap the Event Sourcing

The Contract

A contract helps us to determine which Events are using ES mode, and they must provide the required information for recording.

Events must implement this Contract

interface EventSourcingContract
{
    // required fields
    public function getName(): string;
    public function getPayload(): array;
    public function getMetadata(): array;

    // your additional entity-related keys
    public function getUser(): ?User;
    public function getBusiness(): ?Business;
}

The Event Definition

For example, after the user logs in, we dispatch the UserLoggedIn event:

class UserLoggedIn implements EventSourcingContract
{
    public function __construct(
        public User $user
    ) {}

    public function getName(): string
    {
        return 'UserLoggedIn';
    }

    public function getPayload(): array
    {
        return [];
    }

    public function getMetadata(): array
    {
        return [];
    }

    public function getUser(): ?User
    {
        return $this->user;
    }

    public function getBusiness(): ?Business
    {
        return null;
    }
}

Additionally, you can create a Trait helper to help you deal with basic fallback:

trait UseDefaultEventSourcingInfo
{
    public function getName(): string
    {
        return class_basename(static::class);
    }

    public function getMetadata(): array
    {
        return [];
    }

    public function getUser(): ?User
    {
        return null;
    }

    public function getBusiness(): ?Business
    {
        return null;
    }
}

The Listener

The Listener will create a new event store record every time we dispatch the event.

Create a new listener

class EventSourcingHandler
{
    public function handle(EventSource $event): void
    {
        EventRecord::create([
            // base
            'name' => $event->getName(),
            'payload' => $event->getPayload(),
            'metadata' => array_filter([
                ...$event->getMetadata(),
                'ipAddress' => $this->getIpAddress(),
                'browser' => $this->getUserAgent(),
            ]),

            // entities
            'user_id' => $event->getUser()?->id,
            'business_id' => $event->getBusiness()?->id,
        ]);
    }

    private function getIpAddress(): ?string
    {
        return app()->runningInConsole()
            ? 'Console'
            : request()->getClientIp();
    }

    private function getUserAgent(): ?string
    {
        return app()->runningInConsole()
            ? 'Console'
            : request()->userAgent();
    }
}

Define the listener in the EventProviderService

protected $listen = [
    EventSourcingContract::class => [
        EventSourcingHandler::class,
    ],
];

P/s: Not only you can listen to the exact classes but also interfaces 😎

Dispatching Events

Use the Event facade from Laravel 🔥

Event::dispatch(new UserLoggedIn($user));

The Activity Logs

API

Now that we have an up-and-running Event Sourcing under the hood, we can just query directly to the Event Store table and get the records.

For example, getting all Events related to a User and showing up on the Activity page:

class ActivityLogController extends Controller
{
    public function ofUser(User $user): JsonResponse
    {
        $records = EventRecord::query()
            ->where('user_id', $user->id)
            ->orderBy('created_at', 'DESC')
            ->paginate(20);        

        return EventRecordResource::collection($records)->response();
    }
}

Simple as cake. 😎

Frontend

Personal preference, I'm using zod for defining schema & parsing the Backend's responses into a structured & typed value in TypeScript.

I'll add a lot of event schema and a discriminatedUnion like this:

const userLoggedInEvent = z.object({
    name: 'UserLoggedIn',
    payload: z.unknown(),
});

const userUpdatedEvent = z.object({
    name: 'UserUpdated',
    payload: z.object({
        name: z.string(),
        email: z.string(),
    }),
});

export const eventSchema = z.discriminatedUnion('name', [
    userLoggedInEvent,
    userUpdatedEvent,
    // .. and more
]);

export type EventSchema = z.infer<typeof eventSchema>;

Simple axios request to get the activity logs and parse to the structured data:

type ActivityLogsResponse = AxiosResponse<{
    logs: unknown[];
}>;

// I'll have a Promise<EventSchema[]>
const getActivityLogs = (userId: string) 
    => axios.get(`api/users/${userId}/logs`)
        .then(
            (res: ActivityLogsResponse) => eventSchema.array().parse(
                res.data.logs
            )
        );

And now, I can render the activity logs with strictly typed event data 😎.

// reference in Vue 3, event = ref<EventSchema>({...});

const eventInfo = computed(() => {
    switch (event.value.type) {
        case 'UserLoggedIn':
            // event.value.payload here is unknown
            return 'You logged into the system';
        case 'UserUpdated':
             // event.value.payload here is an object { name, email }
            const { name, email } = event.value.payload;
            return `You updated your info. New name: ${name} | New email: ${email}`;
    }
});

Conclusion

Some are applying ES 100%, some are adapting ES for Activity Logs.

With ES, we would be able to record any state change related, thus in a later stage, it would help us to:

Debug production issues faster ❤️ Every movement is tracked.
Generate analytics data FROM THE PAST (since we hold the data from the past til now) for business purposes 🔥 (data & business intelligence team will love it)
We have a lovely Activity Logs page showing up to users, thus they know what they did in the past

Just neats.

Thanks for reading and until next time ☺️!