Posts

Showing posts from July, 2014

Oracle error - ORA-10873 file 1 needs to be either taken out of backup mode or media recovered

If you got this error when trying to open an Oracle database

SQL> alter database open;
ORA-10873: file 1 needs to be either taken out of backup mode or media recovered
...


As the error message said, you need to move the file out of backup mode:

SQL> startup  mount ; //if the db is not mounted yet
SQL> alter database end backup ;

then you will be able to open the Oracle db:

SQL> alter database open ;

Oracle error - ORA-01109 database not open

This morning, when I was trying to troubleshoot the Oracle database using sqlplus console I got this error:

ORA-01109 database not open

So I need to open the database:

C:\> sqlplus /@PSPRODDB as sysdba
SQL> select status from v$instance;

the result should be MOUNTED. If not, try:

SQL> startup  mount;

Then

SQL> alter database open;



References:https://community.oracle.com/thread/320757?tstart=0

edx-platform - How to enable EDX-SGA xblock

Image
To enable Staff Graded Assignment xblock in your Open EdX instance:

1. Install EDX-SGA:

* Install sga to edx-platform:


sudo -H -u edxapp bashsource /edx/app/edxapp/edxapp_envcd /edx/app/edxapp/venvs/edxapp/srcgit clone https://github.com/mitodl/edx-sgacd edx-sgapython setup.py install cp -R templates/ /edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_sga-0.2.0-py2.7.egg/

* Add 'edx_sga' to INSTALLED_APPS in cms/env/common.py and lms/env/common.py


* Migrate the db:

LMS: sudo -u www-data /edx/bin/python.edxapp ./manage.py lms syncdb --migrate --settings awsCMS: sudo -u www-data /edx/bin/python.edxapp ./manage.py cms syncdb --migrate --settings aws

2. In edx-platform/cms/envs/common.py and edx-platform/lms/envs/common.py, uncomment:

# from xmodule.x_module import prefer_xmodules
# XBLOCK_SELECT_FUNCTION = prefer_xmodules

3. In edx-platform/cms/envs/common.py, change:

'ALLOW_ALL_ADVANCED_COMPONENTS': False,

to:

'ALLOW_ALL_ADVANCED_COMPONENTS': True,


4. Add t…

edx-platform - EDX-SGA's "NoAuthHandlerFound" error

Image
On July 20th, MIT's Office of Digital learning has released an XBlock for a Staff Graded Assignment (SGA). When placed in a course, this XBlock allows students to upload documents which can be viewed and graded by course staff.

Git repository: https://github.com/mitodl/edx-sga

But after I enabled the XBlock successfully, I could not upload the assignment file to the sga xblock (stop at 100%):



I checked the log and found this:

File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/s3/connection.py", line 174, in __init__    validate_certs=validate_certs)  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/connection.py", line 557, in __init__    host, config, self.provider, self._required_auth_capability())  File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/boto/auth.py", line 728, in get_auth_handler    'Check your credentials' % (len(names), str(names)))NoAuthHandlerFound: No handler…

Using nmap to find network devices (IP, MAC) in your LAN

Image
If someday you plug a device which does not have a display (something like a monitor) into your LAN network and don't know where it is on the network (IP address), E.g my Beaglebone Black, you can use nmap. Nmap is a free network scanner utility.

$ sudo nmap -sn 172.18.0.0/16

(scan the 172.18.0.0 network, my LAN network, to find the device)

The results are something like:


I can see one Texas Instruments device which is my Beaglebone Black's network chip manufacturer. So, that is my board.



Accessing PowerSchool Extension Table (custom fields)

In the past, It was not easy to get the custom fields data from PowerSchool's Oracle database because of the way it architected and organized custom fields data (they are stored as views in Oracle). With the newer version of PowerSchool (v7.9 and above), you can create custom fields to the extension database table. You can even migrate all the legacy custom fields (students and teachers/ school staffs) to that new location. It makes my life much easier when I want to manipulate those custom fields data because from now on, all the students custom fields data will be stored in 1 place, and the same for teachers's custom fields.

* Student custom fields data will be stored in U_STUDENTSUSERFIELDS table
* Teacher custom fields data will be stored in U_SCHOOLSTAFFUSERFIELDS table
For example, you can use this script (python, SQLAlchemy, cx_Oracle) to export students data from PowerSchool along with any custom fields data you want:



PowerSchool DATABASE Connector:



How to make rabbitmq runs on localhost only

If you don't care about rabbitmq clustering like I do, you can setup rabbitmq to listen connections from localhost only:


$ sudo nano /etc/rabbitmq/rabbitmq-env.conf


RABBITMQ_NODE_PORT=5672
RABBITMQ_NODE_IP_ADDRESS=127.0.0.1
RABBITMQ_NODENAME=rabbit@myhostname

(RabbitMQ 3.3.4)
References: 
[0] http://serverfault.com/questions/235669/how-do-i-make-rabbitmq-listen-only-to-localhost [1] http://gigisayfan.blogspot.com/2012/06/rabbit-mq-clustering-python-fabric.html

Using Boris REPL to test your PHP code

Image
Love the Python REPL interface? Wanna do something like that in PHP? Let's try Boris, the missing PHP REPL:

https://github.com/d11wtq/boris



To install boris in Ubuntu (I'm using v14.04), the easiest way is to  clone the git repository:

$ git clone git://github.com/d11wtq/boris.git
$ cd boris
$ sudo ln -s /path/to/boris/bin/boris /usr/local/bin/boris

Now you can test-drive your PHP code in command line without writing all the code at one.






How to install OCI8 PHP extension in Ubuntu 14.04 to work with Oracle

To work with Oracle DBMS using PHP you need to install oci8 extension.

0. Install Oracle client instant:

http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html

The Oracle instant client will be installed at /usr/lib/oracle/12.1/client64

Export the ORACLE_HOME and $LD_LIBRARY_PATH environment variable:

$ export ORACLE_HOME=/usr/lib/oracle/12.1/client64
$ export LD_LIBRARY_PATH=$ORACLE_HOME/lib

Read my blog post about oracle 12c instant client at http://iambusychangingtheworld.blogspot.com/2014/07/install-cxoracle-513-in-ubuntu-1404.html

1. Install php-pear (needed by pecl) and php5-dev:

$ sudo apt-get install pear
$ sudo apt-get install php5-dev


2. Install oci8 extension using pecl:

$ sudo pecl install oci8

This will ask for Oracle Home Directory – give the path of the instant client:

'instantclient,/usr/lib/oracle/12.1/client64/lib'


3. Enable oci8 in php.ini:

$ sudo echo "extension=oci8.so" >> /etc/php5/fpm/php.ini 
$ sudo echo "extension=oci8.so"…

edx-platform - Why didn't the THIRD_PARTY_AUTH feature work for me?

Image
Yesterday, I's trying to enable the THIRD_PARTY_AUTH feature of the OpenEdX following these article of the author:

Overview of the feature:
http://johnmcox.blogspot.com/2014/05/getting-started-with-edx-third-party.html

To understand how the feature works in OpenEdX:
http://johnmcox.blogspot.com/2014/05/understanding-edx-third-party.html

I did everything the author said to enable the third party authentication feature to allow my users at UnSchool using their GMail or LinkedIn account to log in to our OpenEdX, including:

1. I did enable the feature in the /edx/app/edxapp/lms.env.json:

...
    "FEATURES": {
"ENABLE_THIRD_PARTY_AUTH": true,
        "AUTH_USE_OPENID_PROVIDER": true,
        "AUTOMATIC_AUTH_FOR_TESTING": false,

...

2. Create Google and LinkedIn apps, then fill in the /edx/app/edxapp/lms.auth.json with keys and secrets of these apps (at the end of the json):

...
    "THIRD_PARTY_AUTH": {
        "Google": {
            &qu…

How is the PowerSchool admin login form processed?

Understand how PowerSchool works is very important if you're working as a developer at a school which is using PowerSchool as the Student Information System. PowerSchool is a big web application with a lot of components working together. It will take a huge amount of time to master all of them, so to get started, I will go through the admin login process of PowerSchool.

1. The first time you access the admin login page (http://mydomain.com/admin/pw.html), PowerSchool will give you a random token, pstoken input field of LoginForm:

<input type="hidden" name="pstoken" value="20233460462xtw69pPuoIBNJ1RyLBnKsfuykVNlJzd">

2. The admin user input username and password into the form (in the form of <username;password>), and press submit button.


3. Before the form's data is sent to PowerSchool server, the password is hashed using base64 md5 and then hashed again with the token provided in step (1)

If the user is a ldap user, the ldappassword fie…

Install cx_Oracle 5.1.3 in Ubuntu 14.04 with oracle-instantclient12.1

To be able to install cx_Oracle 5.1.3 (to manipulate an Oracle 12c database with python) in Ubuntu 14.04 with oracle-instantclient12.1 succesfully, you have to do following things:

1. Download these instant client packages from http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html:

oracle-instantclient12.1-tools-12.1.0.1.0-1.x86_64.rpm
oracle-instantclient12.1-odbc-12.1.0.1.0-1.x86_64.rpm
oracle-instantclient12.1-devel-12.1.0.1.0-1.x86_64.rpm
oracle-instantclient12.1-sqlplus-12.1.0.1.0-1.x86_64.rpm
oracle-instantclient12.1-jdbc-12.1.0.1.0-1.x86_64.rpm
oracle-instantclient12.1-basic-12.1.0.1.0-1.x86_64.rpm

(the packages that are in red are required)

2. Convert all rpm packages to deb using alien:
$ sudo alien -d *.rpm

3. Install all those deb packages:
$ sudo dpkg -i *.deb

4. Create the ORACLE_HOME environment variable:
Create a file: /etc/ld.so.conf.d/oracle.conf and in the first line of the script, write (it depends on your system):
/usr/lib/oracle/12.1/client64/lib
Run the command…

Fix error BDB0091 DB_VERSION_MISMATCH when running alien on Ubuntu

I was trying to convert a rpm package to deb package using alien and I got this error:

$ sudo alien -d mypackage.rpm

error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch
error: cannot open Packages index using db5 -  (-30969)
error: cannot open Packages database in /home/trinh/.rpmdb
...

It is because this is the first time I run alien on Ubuntu and the system did not have any rpm database. So, to get rid of the error, run the following command to install RPM DB:
sudo rpm --rebuilddb

Automate MySQL defragmentation with mysqlfragfinder and crontab

After a while, your MySQL databases will be fragmented, and you need to run the MySQL optimization to defrag it. A guy name Phil Dufault had written a shell script which can find all the fragmented tables in your databases and defrag them for you:

The author's repo: https://github.com/pdufault/mysqlfragfinder



Create a crontab to run that script periodically on a given schedule. For example:

# crontab -e

...
22 4 * * 0 /usr/local/bin/mysqlfragfinder.sh  --user root --password mySuperPassword
...

That crontab will run the mysqlfragfinder.sh at 4:22 AM every Sunday.

edx-platform - AMQPConnectionError of XQueue

I saw this error in the xqueue's log: /edx/var/log/xqueue/edx.log:

Traceback (most recent call last):
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 111, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 20, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/edx/app/xqueue/venvs/xqueue/local/lib/python2.7/site-packages/statsd.py", line 97, in wrapped
    result = func(*args, **kwargs)
  File "/edx/app/xqueue/xqueue/queue/ext_interface.py", line 39, in get_queuelen
    job_count = queue.producer.get_queue_length(queue_name)
  File "/edx/app/xqueue/xqueue/queue/producer.py", line 62, in get_queue_length
    return push_to_queue(queue_name)
  File "/edx/app/xqueue/xqueue/queue/producer.py", line 37, in push_to_qu…

edx-platform - Comments service error caused by auth user disappeared

Image
This morning, the comments service of our OpenEdX instance stopped working. Noone could post comment or replied in a thread:


 Because the forum data is stored in mongodb, so I first try to look into that database, found out that the user I use for the comment service was disappeared?!! So, I tried to create that user and hopefully, the forum worked again:
1. Login to mongodb cli by typing in "mongo" in your linux console ( not sure how to do it in windows or MacOSX)
$ sudo mongo

2. In mongodb cli, switch to the forum db:
> use mydb

3. Check the existence of myuser user using for the forum service:

> show users



4. If there is no user names myuser, create one (mongodb 2.4, http://docs.mongodb.org/v2.4/reference/method/db.addUser/#db.addUser):
> db.addUser({user: "myuser", pwd: "mypassword", roles: ["readWrite", "dbAdmin"] })

5. You may have to restart the forum service (or even reboot the whole machine):

$ sudo /edx/bin/superviso…