Thursday, February 13, 2014

Redmine - How the Email fetching from IMAP server task works

As I mentioned in the previous blog post, you can make the redmine a support ticketing system by running the rake task to fetch email from IMAP server:

http://iambusychangingtheworld.blogspot.com/2014/02/redmine-email-notifications-are-not.html

I have no ruby experience, but as far as I understand this is how the imap rake task works.

First of all, let take a look at the command:

rake -f /path/to/redmine/appdir/Rakefile --silent redmine:email:receive_imap RAILS_ENV=production host=imap.foo.bar username=redmine@somenet.foo password=xx project=myproject unknown_user=create no_permission_check=1 no_account_notice=1 

1. The rake task: /path/to/redmine/lib/tasks/email.rake

    task :receive_imap => :environment do
      imap_options = {:host => ENV['host'],
                      :port => ENV['port'],
                      :ssl => ENV['ssl'],
                      :username => ENV['username'],
                      :password => ENV['password'],
                      :folder => ENV['folder'],
                      :move_on_success => ENV['move_on_success'],
                      :move_on_failure => ENV['move_on_failure']}

      Redmine::IMAP.check(imap_options, MailHandler.extract_options_from_env(ENV))
    end

When the task is executed, It will take the imap options to fetch emails, and other options used for issue creation and notification purposes, and then call the IMAP.check method.


2. IMAP utitilities: /path/to/redmine/lib/redmine/imap.rb

      def check(imap_options={}, options={})
        host = imap_options[:host] || '127.0.0.1'
        port = imap_options[:port] || '143'
        ssl = !imap_options[:ssl].nil?
        folder = imap_options[:folder] || 'INBOX'

        imap = Net::IMAP.new(host, port, ssl)
        imap.login(imap_options[:username], imap_options[:password]) unless ima$
        imap.select(folder)
        imap.uid_search(['NOT', 'SEEN']).each do |uid|
          msg = imap.uid_fetch(uid,'RFC822')[0].attr['RFC822']
          logger.debug "Receiving message #{uid}" if logger && logger.debug?
          if MailHandler.receive(msg, options)
            logger.debug "Message #{uid} successfully received" if logger && lo$
            if imap_options[:move_on_success]
              imap.uid_copy(uid, imap_options[:move_on_success])
            end
            imap.uid_store(uid, "+FLAGS", [:Seen, :Deleted])
          else
            logger.debug "Message #{uid} can not be processed" if logger && log$
            imap.uid_store(uid, "+FLAGS", [:Seen])
            if imap_options[:move_on_failure]
              imap.uid_copy(uid, imap_options[:move_on_failure])
              imap.uid_store(uid, "+FLAGS", [:Deleted])
            end
          end
        end


This method will take imap options to connect to IMAP server and fetch emails. Then, for each email it receives, call MailHandler.receive method to process the issue creation and/ or notify users.


3. MailHanler's receive method: /path/to/redmine/app/models/mail_handler.rb

class MailHandler < ActionMailer::Base
  include ActionView::Helpers::SanitizeHelper
  include Redmine::I18n

  class UnauthorizedAction < StandardError; end
  class MissingInformation < StandardError; end

  attr_reader :email, :user

  def self.receive(email, options={})
    @@handler_options = options.dup

    @@handler_options[:issue] ||= {}

    if @@handler_options[:allow_override].is_a?(String)
      @@handler_options[:allow_override] = @@handler_options[:allow_override].split(',').collect(&:strip)
    end
    @@handler_options[:allow_override] ||= []
    # Project needs to be overridable if not specified
    @@handler_options[:allow_override] << 'project' unless @@handler_options[:issue].has_key?(:project)
    # Status overridable by default
    @@handler_options[:allow_override] << 'status' unless @@handler_options[:issue].has_key?(:status)

    @@handler_options[:no_account_notice] = (@@handler_options[:no_account_notice].to_s == '1')
    @@handler_options[:no_notification] = (@@handler_options[:no_notification].to_s == '1')
    @@handler_options[:no_permission_check] = (@@handler_options[:no_permission_check].to_s == '1')

    email.force_encoding('ASCII-8BIT') if email.respond_to?(:force_encoding)
    super(email)
  end

...

The class method receive(email, options={}) will manipulating options and then call the instance method receive(email) to process the incoming emails:

  # Processes incoming emails
  # Returns the created object (eg. an issue, a message) or false
  def receive(email)
    @email = email
    sender_email = email.from.to_a.first.to_s.strip
    # Ignore emails received from the application emission address to avoid hell cycles
    if sender_email.downcase == Setting.mail_from.to_s.strip.downcase
      if logger
        logger.info  "MailHandler: ignoring email from Redmine emission address [#{sender_email}]"
      end
      return false
    end
    # Ignore auto generated emails
    self.class.ignored_emails_headers.each do |key, ignored_value|
      value = email.header[key]
      if value
        value = value.to_s.downcase
        if (ignored_value.is_a?(Regexp) && value.match(ignored_value)) || value == ignored_value
          if logger
            logger.info "MailHandler: ignoring email with #{key}:#{value} header"
          end
          return false
        end
      end
    end
    @user = User.find_by_mail(sender_email) if sender_email.present?
    if @user && !@user.active?
      if logger
        logger.info  "MailHandler: ignoring email from non-active user [#{@user.login}]"
      end
      return false
    end
    if @user.nil?
      # Email was submitted by an unknown user
      case @@handler_options[:unknown_user]
      when 'accept'
        @user = User.anonymous
      when 'create'
        @user = create_user_from_email
        if @user
          if logger
            logger.info "MailHandler: [#{@user.login}] account created"
          end
          add_user_to_group(@@handler_options[:default_group])
          unless @@handler_options[:no_account_notice]
            Mailer.account_information(@user, @user.password).deliver
          end
        else
          if logger
            logger.error "MailHandler: could not create account for [#{sender_email}]"
          end
          return false
        end
      else
        # Default behaviour, emails from unknown users are ignored
        if logger
          logger.info  "MailHandler: ignoring email from unknown user [#{sender_email}]"
        end
        return false
      end
    end
    User.current = @user
    dispatch
  end

Based on the options, this method will parse the email message to get user information. If the user exists, but is inactive, ignore the message. If the user does not exist, check the unknown_user option to proceed to a appropriate action (create new user account, accept the message as anonymous, or ignore the message by default...). Then call the dispatch function to create a new issue (default, receive_issue) or a new update to an issue (if the subject of the mail has #{id} of an issue, receive_issue_reply).


Check out the mailer_handler source code to understand more (/path/to/redmine/app/models/mail_handler.rb):

https://github.com/redmine/redmine/blob/master/app/models/mail_handler.rb


Cool!