Hi all,
One of my previous works for my OSS Organization - ShipSaaS - The Laravel Priority Queue Driver.
So I'd like to share it with you guys today ๐
Normal Prioritization in Laravel Queue
By default, Laravel Queue only has an alternative approach for Queue Prioritization, which create several dedicated queues and push your job there.
For example, I'm using DB driver for a quick example:
default
high
low
Then I'll push the jobs like this:
TransferJob::dispatch($data)->onQueue('high');
UpdateDataJob::dispatch($data)->onQueue('low');
KybJob::dispatch($data); // default
Problems
Imagine you have 1000 TransferJob and 10 worker processes, a single job takes 30s
So Overall, it costs 5 mins to solve all Queue Msgs without any prioritization.
In reality, the number of jobs will continually increase, and certain constraints will arise, such as prioritizing transactions for VIP customers.
And create more dedicated queues will increase the complexity of the codebase. You have to manage them all in both code & infra.
Not so fun and scalable, right?
Meets Priority Queue Driver
Repository: https://github.com/shipsaas/laravel-priority-queue
PROS:
You can compute the "weight" for each job, based on your constraints ๐ฅ.
Using the "DB" driver, with indexes, the job will be popped super fast ๐๏ธ.
High visibility, under DB, your DB โ๏ธ.
Needs to solve msgs faster? Increase the worker processes ๐โโ๏ธ.
(Optional) You can create dedicated queues if you wanted to.
Installation
Check this out: https://github.com/shipsaas/laravel-priority-queue#installation
Usage
Pretty straightforward and the same as Laravel - Push Job:
MyJob::dispatch($data)->onConnection('database-priority');
// QueueFacade
Queue::connection('database-priority')
->push($myJob);
Define the $weight
- build time:
class MyJob
{
public int $weight = 950;
}
On runtime:
use ShipSaasPriorityQueue\Traits\UseJobPrioritization;
class MyJob implements ShouldQueue
{
use UseJobPrioritization;
public function __construct(public User $user)
{
// you can do this too
$this->onConnection('database-priority');
}
public function getJobWeight() : int
{
return $this->user->isPro()
? 1000
: 500;
}
}
Testing
It is covered by:
Unit Testing
Integration Testing (
queue:work
)
Conclusions
With this driver, we hope to elevate your Laravel Queue Prioritization to a whole new level ๐ฅ
If this package is helpful, please give it a โญ๏ธโญ๏ธโญ๏ธ. Thank you!
Thanks for reading!