Posts

Showing posts from March, 2014

OpenLDAP - alock package is unstable

This morning, After making some changes, I fail to start the slapd service, and this is the error log:

Mar 31 10:20:30 SRVR-UParentLDAP slapd[16175]: hdb_db_open: database "dc=my,dc=domain,dc=com": alock package is unstable.                                         
Mar 31 10:20:30 SRVR-UParentLDAP slapd[16175]: backend_startup_one (type=hdb, suffix="dc=my,dc=domain,dc=com"): bi_db_open failed! (-1)                           
Mar 31 10:20:30 SRVR-UParentLDAP slapd[16175]: slapd stopped. 

This error's related to the file permission of ldap db in /var/lib/ldap/. So, I just change the owner of those files to openldap:

# cd /var/lib/ldap/
# chown openldap:openldap -R ./*

Then, start the slapd service again:

# service slapd start


It works like a charm!



Reference: http://serverfault.com/questions/28270/openldap-error-alock-package-is-unstable




Life's Manifesto

Image
(source: http://www.holstee.com/pages/manifesto)



“This is your life. Do what you want and do it often.  If you don't like something, change it.  If you don't like your job, quit. If you don't have enough time, stop watching TV. If you are looking for the love of your life, stop; they will be waiting for you when you start doing things you love. Stop over-analysing, life is simple. All emotions are beautiful. When you eat, appreciate every last bite. Life is simple. Open your heart, mind and arms to new things and people, we are united in our differences. Ask the next person you see what their passion is and share your inspiring dream with them. Travel often; getting lost will help you find yourself. Some opportunities only come once, seize them. Life is about the people you meet and the things you create with them, so go out and start creating. Life is short, live your dream and wear your passion.”


Free

Image

Vietnamese input in Skype with ibus-bogo

ibus-bogo is a Vietnamese input engine targeting IBus, an input method manager in GNU/Linux environments created by a group of talented Vietnamese developers. You can install it in Ubuntu by the following command:

$ sudo apt-get install ibus-bogo

Its github repo: https://github.com/BoGoEngine/ibus-bogo-python

It works really well in the current release of  xUbuntu (13.10, 14.04) in comparison with ibus-unikey which is buggy. But,  there are still some annoying issues with ibus-bogo. One of them is Vietnamese input in Skype. The official documentations of ibus-bogo instructs us how to solve this problem by adding these lines to ~/.profile:

export GTK_IM_MODULE=ibus
export QT4_IM_MODULE=xim
export QT_IM_MODULE=xim
export XMODIFIERS=@im=ibus

Log out and log in again. Then, restart the ibus-daemon:

$ ibus-daemon -xdr

Notes: In Xubuntu 14.04, I have to do this every time the system startup/reboot or turn off / turn on Skype.

Debugging djangosaml2

I'm using the djangosaml2 app to add the Single Sign On functionality using SAML2 to my Django projects. I followed the official manual at https://bitbucket.org/lgs/djangosaml2/overview to make my project work with a SimpleSAMLphp instance as an IdP but I got this error:

cannot serialize IdpUnspecified('No IdP to send to given the premises',) (type IdpUnspecified)

I have no idea why this error showed up?! I have a very little knowledge about SAML2 and the PySAML2 module. The only thing I can think about is to enable logging for the djangosaml2 app. So, I added these following lines in the settings.py of my project:

import logging
logging.basicConfig()


I reloaded the website and a logging message appeared in the console:

ERROR:djangosaml2:Unable to know which IdP to use


Yessss, that is the problem!

Great lessons From Winnie The Pooh

Image

Upgrade to Xubuntu 14.04 (beta) from 13.10

Image
Right now, March 24, 2014, you can try the Trusty Tahr release of Ubuntu. But, for the Xubuntu distro, I did the do-release-upgrade and found no new release. So, here is the trick to upgrade to Xubuntu 14.04:


1. Open the source list file and change saucy to trusty:

$ sudo nano /etc/apt/sources.list

deb http://archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty universe
deb http://archive.ubuntu.com/ubuntu/ trusty-updates universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates universe
deb http://archive.ubuntu.com/ubuntu/ trusty multiverse
deb-src http://archive.ubuntu.com/ubuntu/ trusty multiverse
deb http://archive.ubuntu.com/ubuntu/ trusty-updates multiverse
deb-src http://archive.ub…

I miss you - Guitar cover by Sungmin Lee

Just an amazing song

Credit: unknown

Error: symbol 'grub_term_highlight_color' not found. Entering rescue mode.

Image
Yesterday, I upgraded my system to the latest Ubuntu 14.04 core, Trusty Tahr, from Xubuntu 13.10. And after rebooting, I got this error:

Error: symbol 'grub_term_highlight_color' not found.
Entering rescue mode.
grub rescue >



To fix this, the only way I know is to re-install grub2. So, I booted into a live cd and used the boot-repair utilities:

sudo add-apt-repository ppa:yannubuntu/boot-repair && sudo apt-get updatesudo apt-get install -y boot-repair && (boot-repair &)

I just used the Recommended repair option from the above window. The utilities led me through several steps including instructed me to run some commands to remove grub, then re-install it. After the restoring process finished, I restarted and everything back to normal. \m/

For more information about boot-repair, please read this manual page: https://help.ubuntu.com/community/Boot-Repair

Introduction to SAML

Image
Recently, I have a chance to work on a SSO system using SAML standard. Here are some good resources to get started with SAML:


The SAML use case, SAML Web Browser SSO:  this is the workflow of authenticating a user via a SSO system using SAML data format.




Here is a visual explanation of SAML by Ping Identity:





This is another video about SAML by Paul Moore, Centrify CTO and co-founder:






And to differentiate between OpenID and SAML (from StackOverflow):
They are two different protocols of authentication and they differ at the technical level.

From a distance, differences start when users initiate the authentication. With OpenID, a user login is usually an HTTP address of the resource which is responsible for the authentication. On the other hand, SAML is based on an explicit trust between your site and the identity provider so it's rather uncommon to accept credentials from an unknown site.

OpenID identities are easy to get around the net. As a developer you could then just accept users comi…

Install M2Crypto with pip in a Virtualenv in Ubuntu 13.10

To be able to install M2Crypto with pip in a Virtualenv in Ubuntu 13.10, you have to install swig first:

$ sudo apt-get install swig

then

$ source /path/to/myvirtualenv/bin/activate
(myvirtualenv)$ pip install M2Crypto



The Book Thief OST by John WIlliams

Image
"One small fact: you are going to die. Despite every effort, no one lives forever"





“I wanted to tell the book thief many things, about beauty and brutality. But what could I tell her about those things that she didn't already know? I wanted to explain that I am constantly overestimating and underestimating the human race-that rarely do I ever simply estimate it. I wanted to ask her how the same thing could be so ugly and so glorious, and its words and stories so damning and brilliant.”

~Death, The Book Thief by Markus Zusak





"I have seen a great many things. I have attended all the world's worst disasters, and worked for the greatest of villains. And I've seen the greatest wonders. But it's still like I said it was: no one lives forever. When I finally came for Liesel, I took selfish pleasure in the knowledge that she had lived her ninety years so wisely. By then, her stories had touched many souls. Some of whom I came to know in passing. Max, whose friendsh…

edX-platform - An Introduction to XBlock development

Image
Yesterday at Delft Hackathon, Ned Batchelder (@nedbat), one of the core developers of edX platform, gave a presentation on what the XBlock is and How to develop XBlocks:



This is very useful if you want get the most out of the edX-platform.

Cross-domain ajax request

Normally, if you send an ajax request to a server script stays in a different domain server, you will get this error:

XMLHttpRequest cannot load http://my.external.domain/myscript.php. Origin http://my.external.domain is not allowed by Access-Control-Allow-Origin.

So, to allow other domains call this cript, just set the 'access-control-allow-origin' header equal '*'.

For example:

http://my.external.domain/myscript.php

<?php header('content-type: application/json; charset=utf-8');
header('access-control-allow-origin: *');

$date = date('m-d-Y h:i:s a', time());

$json = json_encode($date);

echo isset($_GET['callback'])
    ? "{$_GET['callback']}($json)"
    : $json;
?>


And in my current site, send the ajax request to myscript.php:

var mydate = '';
jQuery.ajax({ url: 'http://my.external.domain/myscript.php', crossDomain: true, type: 'GET' }).done(function(res){ console.log(res); mydate =…

edX-platform - To run the CMS at port 80 along side with the LMS

By default, the CMS of the edX production stack will run at port 18010 which may be blocked by corporate's network. To make it run at port 80 and can be accessible via the url studio.your.domain, you only need to change the nginx server block of the CMS at /edx/app/nginx/sites-available/cms:

upstream cms-backend {
            server 127.0.0.1:8010 fail_timeout=0;
    }

server {
  # CMS configuration file for nginx, templated by ansible

listen 80 ;


  server_name ~^((stage|prod)-)?studio\..*;
...
Then, restart the nginx service:
$ sudo service nginx restart
Cool!

edx-platform - Using the Standford theme for your production stack

Image
To apply the Standford theme for your edX production stack:

1. Open/create the file /edx/var/edx_ansible/server-vars.yml, and add the following lines:

edxapp_use_custom_theme: true
edxapp_theme_name: 'stanford'
edxapp_theme_source_repo: 'git://github.com/Stanford-Online/edx-theme.git'
edxapp_theme_version: 'HEAD'


2. Then run the provisioning script:

$ sudo /edx/bin/update edx-platform master




Notes and troubleshooting:

1. To customize the theme, fork the Standford theme's repository, change the style, images...or whatever you want, then edit the server-vars.yml file again:

edxapp_use_custom_theme: true
edxapp_theme_name: 'mycustomtheme'
edxapp_theme_source_repo: 'git://github.com/myaccount/edx-theme.git'
edxapp_theme_version: 'HEAD'

Save and exit. Repeat step #2 to apply the new theme to your edx instance.

2. server-vars.yml does not make any changes after running the update bash script

Take a look at this line in the /edx/bin/update bash script…

edX-platform - Updating edX production stack's versions using edX repos

To update your Open Edx instance to the latest version from a repository (master branch of edx-platform, mybranch,... ) run the following command:

$ sudo /edx/bin/update edx-platform master

After the updating finish successfully, run the db migration command:

$ cd /edx/app/edxapp/edx-platform/
$ sudo -u www-data /edx/app/edxapp/venvs/edxapp/bin/python ./manage.py lms syncdb --migrate --settings aws

Finally, restart the edx's services:

* LMS/CMS: sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf restart edxapp:
* Workers: sudo /edx/bin/supervisorctl -c /edx/etc/supervisord.conf restart edxapp_worker:


References: https://github.com/edx/configuration/wiki/edX-Managing-the-Production-Stack

Using SQLAlchemy and Django FormSet to work with an external database

Manipulating an external database with SQLAlchemy and Django FormSet is not as easy as Form (http://iambusychangingtheworld.blogspot.com/2013/07/django-make-use-of-django-forms.html). It's even more complicated when you want to do the CRUD. I had to do some hacks to achieve that.

For example, I have an external database table as following:

merge table:
=======================
|| id  || original     || translated     ||
=======================
|| 1   || course123 || courseABC ||
|| 2   || course456 || courseABC ||
|| 3   || course234 || courseABC ||
|| ...  || ...              || ...                 ||
=======================

1. First, define the merge table model with SQLAlchemy (MyProject.utils.merge_models):

class Merge(Base):
__tablename__ = 'merge'

id = Column(Integer, primary_key=True)
original = Column(String(100))
translated = Column(String(100))

def __init__(self, original, translated):
self.original = original
self.translated = translated

def __repr__(self):
return "<…

SQLAlchemy - Get and delete object

These are some utility methods to make a query with SQLAlchemy: I gonna use the MyDatabaseConnector class from the previous blog post:


db = MyDatabaseConnector()

* Get object:

def get_obj(db, pk):
session = db.get_session()
try:
obj = session.query(MyObjModel).get(pk)
except:
print "Object not found"
finally:
session.commit()
db.close()

* Delete object:

def delete_obj(db, pk):
session = db.get_session()
try:
obj = session.query(MyObjModel).get(pk)
except:
print "Object not found"
else:
session.delete(obj)
finally:
session.commit()
db.close()


Reference:http://docs.sqlalchemy.org/en/rel_0_9/orm/query.html

Django Form - A shorter way to work with form

I've just found a shorter way to work with Django form in a view with POST request. Instead of:

def home(request):
if request.method == 'POST':
myform = MyForm(request.POST, request.FILES)
if myform.is_valid():
myform.save()
return redirect('home')
else:
myform = MyForm()

return render_to_response('home.html', locals(), context_instance=RequestContext(request))


I can do something like this:

def home(request):
myform = MyForm(request.POST or None)

if myform.is_valid():
myform.save()
return redirect('home')

return render_to_response('home.html', locals(), context_instance=RequestContext(request))



SQLAlchemy - DetachedInstanceError

I got this error when trying to retrieve all the database records in a table:

DetachedInstanceError: Instance <CoursesMerger at 0x37fa950> is not bound to a Session; attribute refresh operation cannot proceed

and this is how I query the database:

rs = session.query(MyTable).all()

To avoid this error, remember to set expire_on_commit = False when initialize the session:

from sqlalchemy import *
from sqlalchemy.orm import *

DB_HOST = 'localhost'
DBMS = 'mysql'
DB_PORT = '3306'
DB_NAME = 'mydatabase'
DB_USER = 'myuser'
DB_PWD = 'mypassword'

DB_ENGINE_URL = '%s://%s:%s@%s:%s/%s' % (DBMS, DB_USER, DB_PWD, DB_HOST, DB_PORT, DB_NAME)

class MyDatabaseConnector(object):
engine = None
Session = None

def __init__(self):
self.connect()

def connect(self):
if self.engine is None:
self.engine = create_engine(DB_ENGINE_URL, echo=False, \
pool_size = 100, pool_recycle=3600)

if self.Session is None:
self.Session = scoped_session(sessionmaker(bind=self.engine,expi…

DBeaver 2.3.6 and 2.3.7 : no more wizard for new connection

If you are using DBeaver version 2.3.6 or 2.3.7, you will encounter this error when you create a new connection:

problem Occured, Problem Openning Wizard, The Selected wizard could not be started. in details :
The selected wizard could not be started. 
Plug-in "org.jkiss.dbeaver.core" was unable to instantiate class "org.jkiss.dbeaver.ui.dialogs.connection.NewConnectionWizard".
org.jkiss.dbeaver.ui.dialogs.connection.NewConnectionWizard

It is a bug. So, just update DBeaver to 2.3.8 or higher, this issue will be fixed.

Reference:http://dbeaver.jkiss.org/forum/viewtopic.php?f=2&t=1160

Google SMTP Error: 454 4.7.0

This morning, I tried to send around 1/2 thousand of emails using an Google App Email account from my Django app along with celery and django-mailer. But after 100 messages were sent, the celery task cannot send email, and I got this error:

Google SMTP Error: 454 4.7.0 Too many login attempts, please try again later

There are something I need to do to solve this:

1. Add a DKIM record in Google Apps: following this stackoverflow question:

http://serverfault.com/questions/543007/google-smtp-error-454-4-7-0-too-many-login-attempts-please-try-again-later

In order to add a DKIM record in Google Apps, you need to do the following:

Go to the Admin ConsoleClick on "Google Apps"Click on "Gmail"Scroll down until you see "Authenticate Email" and click thatSelect the domain you wish to add DKIM toWhen it asks what prefix you want to use, simply use the default of 'google'
You will then see a TXT record in two parts, one piece has the domain and the other has the ac…