Laravel 4 Queue with Beanstalk and Supervisor

Queue  is great tools in laravel for implementing delayed job. Queue allowing system to put task in background so it can become faster and make users skip waiting until tasks is finished.
Queue can be implemented to handle big process such as generating reports, image processing, sending email, or even heavy CRUD process.

Tool Kit

• Laravel 4.2 or 4.3
• Queue
• Beanstalk
• Supervisor

Configuration

Installing Queue

In your application add this to your "require" :{} composer.json

    "pda/pheanstalk": "2.0.*"

Scripting

You can create new class or use controller/model/any class to execute function with Queue. Below is function in controller class executed by Queue.

<?php

class ReportsController extends \BaseController {
  function fire($job,$data){
    //Function Process
  }
}

To execute this method with queue use this command

Queue::push('ReportsController', $data);

Other alternative is to execute function without fire method

<?php

class ReportsController extends \BaseController {
  function generateReport($job,$data){
    //Function Process
  }
}

To execute this method with queue use this command

Queue::push('ReportsController@generateReport', $data);

Queue::push() command can be use in Route, Controller, or even Command.

Running Queue

To running Queue we need Beanstalk as a worker. To activate beanstalk running this command in console

  // To fire last job
  php artisan queue:work

  // To fire running job and listen it
  php artisan queue:listen

  // add listener with flag (you can choose or implement all of this)
  php artisan queue:work --timeout=0 --queue="default" --delay=0 --memory=128 --sleep=3 --tries=0

Install and Configure Beanstalk

Install Beanstalk

sudo apt-get update
// Install in Debian or ubuntu
sudo apt-get install beanstalkd

Configure Beanstalk

sudo nano /etc/default/beanstalkd

// Add or uncomment this line
START yes

Start Beanstalk

sudo service beanstalkd start

Configure Beanstalk in Our Laravel App

Set default queue in app to Beanstalk. Go to app/config/queue.php

// Change This
'default' => 'beanstalkd',

// This is default beanstalk configurations
'connections' => array(

    'beanstalkd' => array(
        'driver' => 'beanstalkd',
        'host'   => 'localhost', 
        'queue'  => 'default',
        'ttr'    => 60,
    ),

),

Supervisor

We use php artisan queue:work or php artisan queue:work as listener, but we know that impossible for us to running this command every time we push job to Queue or when we start servers.

To overcome this we use Supervisor to listen all of active worker. Supervisor will listen all of our queue:work --daemon and restart it if failed.

Install Supervisor

// Install Supervisor in Debian or ubuntu
sudo apt-get install supervisor

// Adding configurations
sudo nano /etc/supervisor/conf.d/reportqueue.conf

[program:reportqueue]
command=php artisan queue:work --daemon --env=your_environment
directory=/path/to/MYAPP
stdout_logfile=/path/to/MYAPP/app/storage/logs/myqueue_supervisord.log
redirect_stderr=true
autostart=true
autorestart=true

Add Configuration to Supervisor

sudo supervisorctl
reread # Tell supervisord to check for new items in /etc/supervisor/conf.d/
add reportqueue       # Add reportqueue process to Supervisord
start reportqueue     # Let's Rock

Check Supervisor

// In Console
ps aux | grep php

# You should see some output similarish this:
php artisan queue:work --daemon --env=your_environment
sh -c php artisan queue:work  --queue="default" --delay=0 --memory=128 --sleep --env=your_environment
php artisan queue:work --queue=default --delay=0 --memory=128 --sleep --env=your_environment

Read More