原文地址:http://fideloper.com/ubuntu-beanstalkd-and-laravel4
Note: TL;DR version at the bottom!
Queues are a great way to take some task out of the user-flow and put them in the background. Allowing a user to skip waiting for these tasks makes our applications appear faster, and gives us another opportunity to segment our application and business logic out further.
For example, sending emails, deleting accounts and processing images are all potentially long-running or memory-intensive tasks; They make great candidates for work which we can off-load to a queue.
Laravel can accomplish this with its Queue package. Specifically, I use the Beanstalkd work queue with Laravel.
Here’s how I set that up to be just about production-ready.
Note: I use Ubuntu for development and often in production. The following is accomplishsed in Ubuntu 12.04 Server LTS. Some instructions may differ for you depending on your OS
Here’s what we’ll cover:
- Laravel and Queues
- Installing Beanstalkd
- Churning through the queue with Supervisor
Laravel and Queues
Laravel makes using queues very easy. Our application, the "producer"
, can simply run something likeQueue::push('SendEmail', array('message' => $message));
too add a "job"
to the queue.
On the other end of the queue is the code listening for new jobs and a script to process the job (collectively, the "workers"
). This means that in addition to adding jobs to the queue, we need to set up a worker to pull from the stack of available jobs.
Here’s how that looks in Laravel. In this example, we’ll create an image-processing queue.
Install dependencies
As noted in the docs, Laravel requires the Pheanstalk package for using Beanstalkd. We can install this using Composer:
$ composer require pda/pheanstalk:dev-master
Create a script to process it
Once our PHP dependency in installed, we can begin to write some code. In this example, we’ll create aPhotoService
class to handle the processing. If no method is specified, laravel assumes the class will have a fire()
method. This is half of a worker – the code which does some processing.
<?php namespaceMyapp\Queue;classPhotoService{
publicfunction fire($job, $data){
// Minify, crop, shrink, apply filters or otherwise manipulate the image}}
Push a job to a Queue
When a user uploads an image, we’ll add a job to the queue so our worker can process it.
In Laravel, we’ll create a job by telling the Queue library what code will handle the job (in this case thefire()
method inside of Myapp\Queue\PhotoService
as defined above) and give it some data to work with. In our example, we simply pass it a path to an image file.
Queue::push('Myapp\Queue\PhotoService', array('image_path'=>'/path/to/image/file.ext'));
Process the jobs
At this point, we have code to process an image (most of a worker), and we’ve added a job to the queue. The last step is to have code pull a job from the queue.
This is the other half of a worker. The worker needs to both pull a job from the queue and do the processing. In Laravel, that’s split into 2 functionalities – Laravel’s queue listener, and the code we write ourselves – in this case, the PhotoService
.
Laravel has some CLI tools to help with queues:
// Fire the latest job in the queue $ php artisan queue:work // Listen for new jobs in the queue// and fire them off one at a time// as they are created $ php artisan queue:listen
When not working with the “sync” driver, these tools are what you need to use in order to process the jobs in your queue. We run the queue:listen
command to have laravel listen to the queue and pull jobs as they become available.
Let’s install Beanstalkd to see how that works.
By default, laravel will run queue jobs synchronously – that is, it runs the job at the time of creation. This means the image will be processed in the same request that the user created when uploading an image. That’s useful for testing, but not for production. We’ll make this asynchronous by introducing Beanstalkd.
Beanstalkd
Let’s install Beanstalkd:
# Debian / Ubuntu: $ sudo apt-get update $ sudo apt-get install beanstalkd
Note: You may be able to get a newer version of Beanstalkd by adding this PPA. Ubuntu 12.04 installs an older version of Beanstalkd.
Next, some quick configuration. The first thing we need to do is tell Beanstalkd to start when the system starts up or reboots. Edit /etc/default/beanstalkd
and set START
to “yes”.
$ sudo vim /etc/default/beanstalkd > START yes # uncomment
Then we can start Beanstalkd:
$ sudo service beanstalkd start # Alternatively: /etc/init.d/beanstalkd start
Now we can setup Laravel. In your app/config/queue.php
file, set the default queue to ‘beanstalkd’:
'default'=>'beanstalkd',
Then edit any connection information you need to change. I left my configuration with the defaults as I installed it on the same server as the application.
'connections'=> array('beanstalkd'=> array('driver'=>'beanstalkd','host'=>'localhost','queue'=>'default',),),
Now when we push a job to the queue in Laravel, we’ll be pushing to Beanstalkd!
Installing Beanstalkd on a remote server
You may (read: should) want to consider installing Beanstalkd on another server, rather than your application server. Since Beantalkd is an in-memory service, it can eat up your servers resources under load.
To do this, you can install Beanstalkd on another server, and simply point your “host” to the proper server address, rather than localhost
.
This leaves the final detail – what server runs the job? If you follow all other steps here, Supervisord will still be watching Laravel’s listener on your application server. You may want to consider running your job script (or even a copy of your application which has a job script) on yet another server whose job is purely to churn through Beanstalkd queue jobs. This means having a listener and working listener/job code on yet another server.
In fact, in a basic distributed setup, we’d probably have an application server (or 2, plus a load-balancer), a database server, a queue server and a job server!
Supervisord
Let’s say you pushed a job to Beanstalkd:
Queue::push('Myapp\Queue\PhotoService', array('image_path'=>'/path/to/image/file.ext'));
Now what? You might notice that it goes to Beanstalkd, but Myapp\Queue\PhotoService@fire()
doesn’t seem to be getting called. You’ve checked your error logs, you see if the image was edited, and found that the the job is just “sitting there” in your Beanstalkd queue.
Beanstalkd doesn’t actually PUSH jobs to a script – instead, we need a worker to check if there are jobs available and ask for them.
This is what $ php artisan queue:listen
does – It listens for jobs and runs them as they become available.
If you run that command, you’ll see your job being sent to code. If all goes well, your image will beproperly manipulated.
The question then becomes: How do I make php listen at all times? We need to avoid having to “supervise” that process manually. This is where Supervisord comes in.
Supervisord will watch our queue:listen
command and restart it if it fails. Let’s see how to set that up.
First, we’ll install it:
# Debian / Ubuntu: $ sudo apt-get install supervisor
Next, we’ll configure it. We need to define a process to listen to.
$ sudo vim /etc/supervisor/conf.d/myqueue.conf
Add this to your new conf file, changing file paths and your environment as necessary:
[program:myqueue] command=php artisan queue:listen --env=your_environment directory=/path/to/laravel stdout_logfile=/path/to/laravel/app/storage/logs/myqueue_supervisord.log redirect_stderr=true
We now have a process called “myqueue” which we can tell Supervisord to start and monitor.
Let’s do that:
$ sudo supervisorctl > reread # Tell supervisord to check for new items in /etc/supervisor/conf.d/> add myqueue # Add this process to Supervisord> start myqueue # May say "already started"
Now the “myqueue” process is on and being monitored. If our queue listener fails, Supervisord will restart the php artisan queue:listen --env=your_environment
process.
You can check that it is indeed running that process with this command:
$ ps aux | grep php # You should see some output like this: php artisan queue:listen --env=your_environment sh -c php artisan queue:work --queue="default"--delay=0--memory=128--sleep --env=your_environment php artisan queue:work --queue=default--delay=0--memory=128--sleep --env=your_environment
Wrapping up
Now we have a full end-to-end queue working and in place!
- We create a script to process a queued job
- We installed Beanstalkd to act as the work queue
- We use Laravel to push jobs to our queue
- We use Laravel
queue:listen
to act as a worker and pull jobs from the queue - We wrote some code to process a job from the queue
- We use Supervisord to ensure
queue:listen
is always listening for new jobs
Notes
- You might want to consider setting up log rotation on the Laravel and Supervisord logs
- You can read here for more information on setting up Supervisord on Ubuntu.
- Read the Laravel docs on queues to learn how and when to
release
ordelete
jobs.
TL;DR
For reference, just copy and paste the whole process from here:
$ sudo apt-get update $ sudo apt-get install -y beanstalkd supervisor $ sudo vim /etc/default/beanstalkd > START yes # uncomment this line $ sudo service beanstalkd start $ sudo vim /etc/supervisor/conf.d/myqueue.conf
Enter this, changing as needed:
[program:myqueue] command=php artisan queue:listen --env=your_environment directory=/path/to/laravel stdout_logfile=/path/to/laravel/app/storage/logs/myqueue_supervisord.log redirect_stderr=true
Start Supervisord:
$ sudo supervisorctl > reread # Get available jobs> add myqueue > start myqueue
Read more on Supervisord here for info on supervisorctl.
另外还有一篇相关的不错的文章:http://www.lornajane.net/posts/2014/working-with-php-and-beanstalkd
转载于:https://www.cnblogs.com/argb/p/3729084.html
发布者:全栈程序员-用户IM,转载请注明出处:https://javaforall.cn/109830.html原文链接:https://javaforall.cn
【正版授权,激活自己账号】: Jetbrains全家桶Ide使用,1年售后保障,每天仅需1毛
【官方授权 正版激活】: 官方授权 正版激活 支持Jetbrains家族下所有IDE 使用个人JB账号...