Tuesday, August 5, 2014

edx-platform - Default ElasticSearch configuration caused me trouble

A couple weeks ago, our OpenEdX instance's network interface which was deployed in a DigitalOcean's VPS was blocked by the company. They said that there was a large flood of traffic coming out from our server that was disrupting the normal traffic flow for other users. We were still able to ssh to the server using the web interface of DO to troubleshoot the issue but It was really annoying. I tried many things such as turn off or reconfigure services of the OpenEdX instance but it seemed like the problem was still there and our server was still blocked.

After a while, the DO engineer suggested us to check our ElasticSearch configuration. There is a vulnerability of ElasticSearch if we use the default configuration:

http://bouk.co/blog/elasticsearch-rce/#how_to_secure_against_this_vulnerability

I configure our ElasticSearch service as the article suggested, add these two line in /etc/elasticsearch/elasticsearch.yml:

1. Disable dynamic scripting and prevent remote code execution:
script.disable_dynamic: true

2. Only binding ElasticSearch to localhost:
network.bind_host: 127.0.0.1


In addition, I took that chance to optimize all other services such as RabbitMQ, Nginx... then restart the server.

Finally, the issue seemed to be solved and DigitalOcean turned on the network interface for us.

I asked around the OpenEdX community and realized that the error was fixed in the latest release of OpenEdX's configuration repo. So, I should have plan to upgrade our system soon to avoid errors like this one.