Asynchronously sending email using Django, Celery, Django-mailer

It's good to blog in a beautiful Sunday. This is the second one.
A popular situation a webapp can face is to send thoundsands of emails to the users. This action can block the system a while, and this is annoying! To solve this problem, there are many solutions. One of them is sending email to a queue and send them later in the background (automatically), so the system don't have to be freezed. Pretty cool huh?
I will setup a message queue in Django with:
A. django-celery: Celery will help you to execute periodical tasks. And, django-celery is the package you need to use celery in django. Follow the instruction here to install django-celery (celery will be install automatically as a dependence) (http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html):

1. Install the django-celery library:

   $ pip install django-celery

2. Add the following lines to settings.py:

   import djcelery djcelery.setup_loader()

3. Add djcelery to INSTALLED_APPS.
4. Create the celery database tables.

   For those who are not using south, a normal syncdb will work:

   $ python manage.py syncdb

B. rabbitmq: a message broker prefer by Celery: (http://www.rabbitmq.com/install-debian.html)

1. Add the following line to your /etc/apt/sources.list:

   deb http://www.rabbitmq.com/debian/ testing main

   (Please note that the word testing in this line refers to the state of our release of RabbitMQ, not any particular Debian distribution. You can use it with Debian stable, testing or unstable, as well as with Ubuntu. We describe the release as "testing" to emphasise that we release somewhat frequently.)
2. (optional) To avoid warnings about unsigned packages, add our public key to your trusted key list using apt-key(8):

   wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc sudo apt-key add rabbitmq-signing-key-public.asc

3. Run apt-get update.
4. Install packages as usual; for instance,

   sudo apt-get install rabbitmq-server

C. Django-mailer: A reusable Django app for queuing the sending of email (https://github.com/jtauber/django-mailer). Download the packace and install it (and you can copy the mailer folder in the package to your project folder)
Now, It's time to start the show:
I. Create your task in tasks.py file. Place tasks.py in the same place with your settings.py :

from celery.decorators import periodic_task 

from mailer.engine import send_all 

from datetime import timedelta   

# this will run every 1 seconds 

# send all emails in the mailer queue 

@periodic_task(run_every=timedelta(seconds=1)) 

def email_tasks(): 

    send_all()

II. And in your desired views.py, call send_mail() or send_mass_mail() (those 2 function actually send the mail to queue in the database):

from mailer import send_mail, send_mass_mail def myview(request):      ...     send_mail('My subject', 'This is the content of the mail', 'dangtrinhnt@gmail.com', ['someone@example.com'])return render_to_response('mytemplate.html', locals()

III. Settings: add the following to your settings.py (check http://www.atsnippets.com/development/django-celery-rabbitmq.html to know how to great user and vhost for rabbitmq broker)

BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "ptc_user"
BROKER_PASSWORD = "1q2w3e"
BROKER_VHOST = "myvhost" 

update May 3, 2013: replace all of the above settings with:

BROKER_URL = 'amqp://ptc_user:1q2w3e@localhost:5672/myvhost'

# List of modules to import when celery starts.
CELERY_IMPORTS = ("tasks",)
# smtp settings for email
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = ''
EMAIL_HOST_PASSWORD = ''
EMAIL_USE_TLS = False
DEFAULT_FROM_EMAIL = 'dangtrinhnt@gmail.com'

IV. Run the magic:
- Start rabbitmq:

$ sudo rabbitmq-server 

- Start Celery in beat mode (note: do not run the following command line with sudo), cd to you project directory:

$ python manage.py celeryd -v 2 -B -s celery -E -l INFO

- Access your view to activating send_mail function, and see the magic!

Enjoy!

Trinh