Tag Archives: sqlite

[update 2012-01-18] postgres has been updated to 9.1.2; the latest version as of today.

[update 2012-01-17] feel free to ignore my comments about Lua. While Lua might sit in an interesting place between Python and Java in an embedded/scripting place. The fact that lunatic-python does not compile and lupa depends on LuaJIT2 which is compatible with Lua 5.1 and the current Lua version 5.2 was recently released… and the comment from the LuaJIT team about adoption was a little snarky. I gotta think about something else.

[update 2011-12-29] I forgot to add twitter’s bootstrap CSS/JS. I’ll cover that in a future post when I also discuss modern-package-template

It’s pretty simple to set things up. There are some prerequisites and some basic install packages that need root access but the intent is to get the config in userspace as soon as possible. This article covers VM slices at Rackspace using Ubuntu 11.10.

First: Install and update:

• allocate the OS
• select the OS and wait for it to complete.  You’re going to receive an email with the root password
• login, change the root password
• create an “admin” privileged user (usually my name or “builder”)
• add this user to the sudo
• change it’s password
• edit /etc/ssh/sshd_config and disable root login
• update the package definitions (apt-get update)
• upgrade the packages (apt-get upgrade)
• reboot

Second: Install the required roo packages

• postfix – when prompted select the default values
  • apt-get -y install postfix
• apt-get -y install python-setuptools daemontools daemontools-run python-dev mailutils mutt build-essential uuid-dev python-nose vim htop sysstat dstat ifstat screen locate apache2-utils unzip siege python-virtualenv bwm-ng libcairo2-dev libglib2.0-dev libpango1.0-dev libxml2-dev fail2ban openssl libssl-doc openvpn libssl-dev libgcrypt11-dev lighttpd lighttpd-dev libevent-dev libcurl4-openssl-dev  libreadline6-dev beanstalkd tree
• apt-get install postgresql-9.1 postgresql-client-9.1 postgresql-doc-9.1 postgresql-plperl-9.1 postgresql-plpython-9.1 postgresql-server-dev-9.1
• easy_install pip
• easy_install mercurial
• easy_install pycurl
• pip install virtualenvwrapper

That’s it, essentially, for the second layer, however, here’s an explanation of the modules from a macro perspective:

• python-setuptools – make the installer, easy_install, available
• daemontools daemontools-run – there are so many ways to implement a ‘daemon’ this tools make it simple to make daemon deployment simple
• python-dev python-nose python-virtualenv – basic prereqs for python development. virtualenv is needed so that packages can be installed in userspace
• mailutils mutt – generate emails
• build-essential uuid-dev  - basic developer tools
• vim screen – editor and console tool
• htop sysstat dstat ifstat locate unzip bwm-ng – debug /monitoring tools
• libcairo2-dev libglib2.0-dev libpango1.0-dev libxml2-dev – libs used when rendering usage graphics
• fail2ban – detect login attempts and put the IP in time-out
• openssl libssl-doc openvpn libssl-dev libgcrypt11-dev libcurl4-openssl-dev - crypto
• lighttpd lighttpd-dev – web server that should be in front of the framework
• libevent-dev – kevent, kpoll libs
• apache2-utils siege – performance simulation tools
• beanstalkd – message queue

And finally the third layer, the userspace framework layer. But before you start installing packages you need to create the virtual environment:

• cd ${HOME}
• mkdir -p src
• cd ${HOME}/src
• virtualenv currentenv
• . ./currentenv/bin/activate

• pip install tornado
• pip install flask
• pip install flask-rest
• easy_install pip
• pip install pycurl
• pip install simplejson
• pip install tornado
• pip install Fabric
• pip install PasteDeploy
• pip install PasteScript
• pip install modern-package-template
• pip install requests
• pip install gevent
• pip install pystache
• pip install nose
• pip install redis-py
• pip install pymongo
• pip install hoover
• pip install pyzmq
• pip install pyyaml
• pip install beanstalkc
• pip install django
• pip install django-redis-cache
• pip install clint
• pip install djangorestframework
• pip install pyparsing
• pip install flup

I’m hoping that there is a practical use-case for embedding Lua in Python. There are an few interesting projects like lunatic-python and lupa. Normally I would not consider Lua for anything beyond “hello world”, however, the redis team is embedding Lua, it seems like a very lightweight codebase, it can be embedded in just about any language (do a google search).

• cd ${HOME}
• mkdir -p tmp
• cd tmp
• wget http://www.lua.org/ftp/lua-5.2.0.tar.gz
• tar zxvf lua-5.2.0.tar.gz
• cd lua-5.2.0
• make linux
• sudo make install

NOTE: the lunatic project does not compile under Lua 5.2. So this thread is postponed for now.

• pip install lunatic-python

Alternatively I tried [lupa] but that requires LuaJIT 2.0 which is currently in beta (version 9)

• cd ${HOME}
• mkdir -p tmp
• cd tmp
• wget http://luajit.org/download/LuaJIT-2.0.0-beta9.tar.gz
• tar zxvf LuaJIT-2.0.0-beta9.tar.gz
• cd LuaJIT-2.0.0-beta9
• make
• sudo make install
• sudo ldconfig

Then install lupa.

In closing, I would like to include a few more libraries, however, the current version in apt-get is too old. I’d prefer installing them from scratch. They are necessary packages so for the time-being I’m just going to list them. They should be installed when installing the first layer and by the root user (or sudo)

• ZeroMQ - trivial to build and deploy if you follow the instructions
  • cd /tmp
  • wget http://download.zeromq.org/zeromq-2.1.11.tar.gz
  • tar zxvf zeromq-2.1.11.tar.gz
  • ./configure
  • make
  • make install
  • ldconfig
• MongoDB – (can actually be installed in userspace)
  • cd /tmp
  • wget http://fastdl.mongodb.org/osx/mongodb-linux-x86_64-2.0.2.tgz
  • mkdir -p ${HOME}/bin
  • cd ${HOME}/bin
  • tar zxvf mongodb-linux-x86_64-2.0.2.tgz
  • sudo mkdir -p /data/db
  • sudo chown `id -u` /data/db
  • ./mongodb-linux-x86_64-2.0.2/bin/mongod
  • … or …
  • cd ${HOME}/bin
  • find ./mongodb-linux-x86_64-2.0.2/bin/ -type f -exec ln -s {} \;
  • ./mongod
• Redis – trivial to build and deploy if you follow the instructions
  • cd /tmp
  • wget http://redis.googlecode.com/files/redis-2.4.6.tar.gz
  • tar zxvf redis-2.4.6.tar.gz
  • cd redis-2.4.6
  • make
  • make install
• SQLite – a simple SQL DB
  • cd /tmp
  • wget http://www.sqlite.org/sqlite-autoconf-3070900.tar.gz
  • tar zxvf sqlite-autoconf-3070900.tar.gz
  • cd sqlite-autoconf-3070900
  • ./configure
  • make
  • make install
  • ldconfig
• ISO8583 – ISO8583 lib
  • # will use virtualenv if it is configured
  • cd /tmp
  • http://pypi.python.org/packages/source/I/ISO8583%20Module/ISO8583%20Module-1.2.zip#md5=259c5eb63bb36e3376f8a430a5b4a092
  • unzip ISO8583\ Module-1.2.zip
  • cd ISO8583\ Module-1.2
  • python ./setup.py install

Good luck!

PS: You should consider scripting this installation so that the deploy can be automated.  Specially via Fabris, chef, or puppet.