RSS

Tag Archives: erlang

Celery 3.0 – congrats, so what you’re still fat.

07 Jul

Celery is HUGE. The dependencies include RabbitMQ which requires erlang. And while erlang offers some killer features it’s one big-ass VM. Given how Celery is meant to be used I’m not convinced that RabbitMQ is a good tool. There are so many other MQs out there. Make sure you check the features you really want and need. I happen to like speed, persistent queueing, multiple workers, flexible topology. Much of this is implemented in ZMQ but even that can be heavy winded. Some folks are using redis, others mongodb for this exact function.

I guess this is why Cerlery offers some options to the RabbitMQ broker.

 
Leave a comment

Posted by on 2012/07/07 in architecture

 

Tags: , ,

mnesia sucks – give me SQL any day!

14 Apr

I started programing SQL some 15 years ago. I do not particularly like it because it reminds me of COBOL and I most certainly do not like it either. But that was yesterday’s news.

In recent weeks I have had this notion that well written code is better than well written documentation and poorly written code. That said I can see the value in the “values” presented by COBOL. Sadly, SQL is not as expressive, however, it’s syntax reads a lot better than the existing query syntax for the likes of Riak, MongoDB, Mnesia, Cassandra (although Cassandra is trying to address that).

The missing link is that there are no REAL tools for administer these databases or applications. And at the extreme end Mnesia demands that the administrator write their own scripts to manage the database. (The so called emacs-IDE for erlang is not the answer and never will be)

As a demonstration of this frustration I posted a question on StackOverflow. It’s a long and involved question but it clearly demonstrates that administering an erlang application and a mnesia database cannot be performed by a standard DBA or SYSADMIN. It requires highly paid and highly skilled programmers. It’s the sort of thing that cannot and will not scale in an enterprise.

Until this changes I will not have much use for erlang. But keep this message in mind when you start your next project. If people/administrators who are paid to manage your application do not have to tools they need to detect and resolve production problems then you have not performed well for your client or employer or profession.

PS: If anyone can answer my question I’d appreciate that.

 
Leave a comment

Posted by on 2012/04/14 in architecture, database, ProgLang

 

Tags: ,

Your Next Web Application Framework?

12 Apr

Suppose you are the person who has to make the decision as to what language and framework your startup is going to use to deploy it’s application.  What would you choose? There are so many interesting and qualified frameworks that are already powering a good portion of the internet.

(You don’t have to know the language and framework but enough to argue what makes it idea.)

What would it be?

For example: I read an article several years ago that strongly recommended Erlang. At the time it was a great idea. The author suggested that using Erlang would attract smart people and keep the actual number of respondents to something manageable. Since then I implemented several applications that in hindsight: (a) impossible to attract new talent. (b) the more time that passes the more fragile the app gets because my detail recollection is fading (c) and it lacks common tools that would make allow generalized apps to give access to “operators” instead of programmers.

Other examples include: perl, ruby, python… mojolicious, rails and sinatra, flask, tornadoweb, and django.

I have my ideas… what are yours?

 
Leave a comment

Posted by on 2012/04/12 in architecture, future, web

 

Tags: , , , , , , , , , , ,

Response to “Seven Languages in Seven Weeks”

20 Dec

Bruce Tate is a good writer and recently he published a book titled: “Seven Languages in Seven Weeks”. I do a lot of career development so I completely agree with the premise, however, the first place I get lost is the selection of languages:

  • Ruby,
  • IO
  • Prolog
  • Scala
  • Erlang
  • Clojure
  • Haskell

Initially there is nothing wrong with this selection. Tate tells the reader that the choices were made my asking his readers. And at first glance this might makes sense (blame the reader), however, it’s more dubious than that.

As I rebuked a recent blog for it’s survey results pertaining to Agile because the sample group were in relative social circles to the author. I believe the same can be said here. About the only thing in Tate’s favor, however, is that the words “practical” or “pragmatic” were omitted. Had they been present then I believe that the language selection might have echoed github’s language survey.

In hindsight I should have read the TOC before I purchased the book. I already had a cursory knowledge of Ruby, I’ve been coding erlang professionally for 3 years, prolog was deprecated when Borland’s Turbo Prolog was decommissioned, I’ve reviewed Haskell and it’s of no general interest… I think erlang got it right. And as the saying goes, “lipstick on a pig, it’s still a pig” when it comes to Scala and Clojure.

If it were my book I think the list would have been a little different:

  • go – the languages solves some concurrency and messaging issues in many other languages, it’s also statically linked.
  • erlang – lightweight processes, fast, has momentum
  • python 3 & perl 6 and PHP(Facebook) - These updates have been in development a long time. It’s critical to understand whether the new versions are worth the mid share or if they should be deprecated.
  • modern C or C++ -
  • groovy – It’s java lite and while I do not have any practical experience with it, since I do a lot of development in python, perl and ruby this makes sense in the JVM environment.
  • serverside javascript (NodeJS, MongoDB, etc) – another up and comer. This is probably more like a 1/2 week experience, however, just because you know browser javascript does not mean that you’ll be successful on the server side.
  • R – The google-ites and Facebook-ies are going crazy with analytics and now that the “social” aspect has entered just about every website tools that render information about the business are becoming critical. R has a great many tools to help out. Hopefully one does not need a Phd in math to be successful.

What sets my list apart from Tate’s is that they are look to the future. “Where are we going?” not “What’s slipped between the cushions?”

As a sidebar, I have another list that I think might be interesting: “Seven Frameworks in Seven Days”. You’re not going to become an expert in seven days but you might know enough to make a choice for your next project based on that experience:

  • TornadoWeb or Cyclone (python) – very capable frameworks but they are even driven.
  • Mojolicious (perl) – another event driven framework.
  • Sinatra (Ruby) – something to attract ruby-ists. It’s as capable as those above.
  • Limonade (PHP) – PHP is powering back up thanks to Facebook’s compiler.
  • Orbit (Lua) – Lua was conceived in the vacuum of Brazil and has an adopted home in World of Warcraft. At some point those programmers are going to want to break out of the game into the real world.
  • Snap (Haskell) – It’s fast.
  • Nitrogen (erlang) – interesting GUI, comet, baked right in.

One reason for the entries in this list is that the language portion of the exercise is trivial. Micro frameworks are not capable of running an enterprise but it’s low cost of entry is going to get things started so that your burn rate is smaller.

While Snap and Nitrogen are interesting in their own right that’s about it. They will not likely be here in 2 or 3 years but the ideas are great.

Thanks for reading.

 
Leave a comment

Posted by on 2011/12/20 in future

 

Tags: , , , , , , , , , , , ,

Connecting to MongoLab – perl and python

13 Oct

[update 2011.10.13] I thought I would add the following quote from the MongoLab support pages: “If you connect to your database from outside EC2 or Rackspace your data is less secure. While your database does require username / passord authentication, you are potentially vulnerable to others “sniffing” your traffic. We are currently exploring ways to provide for more secure methods of connecting to MongoLab databases from outside the cloud.”

I’m working on a mojolicious project as currently mentioned on this site. The next logical step for the application is a connection to the DB. I was originally going to deploy a mongoDB instance on my own server… and that would be great. But I’ve decided to use MongoLabs instead.  I suppose I could also use MongoHQ and try them independently and for comparison. That’s a story for another day.

Connecting to MondoLab was pretty simple:

  • create an account
  • create a database
  • create a collection
  • create a user for interacting with the collection
That’s it.  The nice thing is that MongoLab gives you the mongo-cli interface sample so you know exactly what’s what. From here you can test the connection on your PC if you have the mongoDB client installed.
From outside of Rackspace:
   mongo dbh04.mongolab.com:27047/mydb -u <username> -p <password>
From within Rackspace:
   mongo 10.183.5.47:27047/mydb -u <username> -p <password>

Looking at the CPAN help for MongoDB (previously installed in mojolicious part 1) So now we have to test the connection with this sample code (I got it from the CPAN and then made some corrections).

use MongoDB;

my $connection = MongoDB::Connection->new(host => 'mongodb://dbh99.mongolab.com:27999');
$connection->authenticate('mydb', 'username', 'password');
my $database   = $connection->mydb;
my $collection = $database->get_collection(my_collection);
my $id         = $collection->insert({ some => 'data' });
my $data       = $collection->find_one({ _id => $id });

Once I executed the program I verified that the data was written to the DB by logging into the webGUI and checking the collection. The data was there and ready.

Then I took this program and made a few modifications so that I could dump the record I just inserted. The code looks like this:

use MongoDB;

my $connection = MongoDB::Connection->new(host => 'mongodb://dbh99.mongolab.com:27999');
$connection->authenticate('mydb', 'username', 'password');
my $database   = $connection->mydb;
my $collection = $database->get_collection(my_collection);
my $data       = $collection->find_one();
while (($key, $value) = each(%$data)){
     print $key.", ".$value."<br />";
}

It’s not a very sophisticated dumper and there are some good libs for that sort of thing, however, my mission was to dump the data and so I did.

I’d like to take a sidebar moment to mention that I recently read an article “why perl”. The take away from the article was that perl programmers are ‘A’ and that perl programs are ‘B’. Granted there is no real evidence of this, however, there is a corollary. If you want to hire smart people who take an interest in their craft and you do not want to go through throngs of java resumes post an erlang position. So what I’m saying is that perl is a edge language where python, ruby, javascript, java are in the median space and therefore “mostly” attracting median skilled programmers.(let the trolling begin)

So I implemented the same exact program (to pull the data from the DB) in python. It took half the time because I was already familiar with the syntax etc… as there is some nuance in perl that I’ve flushed from my cache I wanted to make a connection to my my python side.

from pymongo import Connection
connection = Connection('dbh99.mongolab.com',27999)
db = connection.mydb
db.authenticate('username','password')
collection = db.mycollection
print collection.find_one()

Worked like a charm. The output was prettier because python makes that easy. I’m looking forward to part 3. In the meantime I’m going to try this against mongoHQ.

 
Leave a comment

Posted by on 2011/10/13 in database, ProgLang, Tools

 

Tags: , , , , ,

Clojure and Scala… again

09 Aug

[Update 2011-08-09: I am returning from a failure to install scala and clojure. I was able to install OpenJDK from the Ubuntu packages, however, I wanted to build ant from source. And that does not appear to be possible. Ant depends on JUnit and JUnit depends on Ant. This makes it impossible to install everything from source. What's worse is that the build instructions are so much worse than I remember. There was a time when I would install all my Java tools except the JDK and my commercial JDBC drivers by hand.]

I just watched a video presentation from one of the Twitter geeks and he was going on and on about the JVM. What make it interesting is that Twitter is moving from Ruby to the JVM, in fact everything is now written in Java or Scala; and there are some research projects in Clojure.

I was looking over my resume recently as I was reviewing my Java experience for a potential position. And with about 10 practical years experience from the first version of Java (1.02) until a few years ago… it was an uphill battle with most managers. Now “Java is the new Cobol” or “nobody ever got fired for using Java”.

I find myself looking back at dynamic languages with a little more interest. Perl and I have a long history and Python and I are getting there.

But I have some criticisms:

  • Eclipse was a wonderful project once. Now it’s total chaos.
  • NetBeans is a nice platform but with a little FUD I’m concerned about Oracle and eating your own dog food. We might not be getting the best tools from them.
  • There are other Java IDEs but they are expensive or single purpose. (IntelliJ’s Idea is $199 for an individual license; but it supports Groovy, Clojure, and Scala.)
  • I tried to install Scala and Clojure using packages, however, the version numbers were so far back that it’s scary. I have heard “please upgrade” too many times.

On the positive side I have a couple of books already. I’m sure they are good enough. I’m still a little bitter about the dependency on Java libs for Scala and Clojure but if you’re a functional programmer I think that Scala and Clojure are going to be easier to sell than Erlang or Haskell… as they are incremental steps (baby steps) and they still use the JVM.

So what is the plan or roadmap?

  • deploy jars and wars
  • instrument deliverables in a consistant way
  • use event driven design with MQs
  • watch your TPS rates in terms of events
  • log in a way that makes sense
  • plan to deploy in a cluster (see MQ)
  • plan to use 3 types of databases (OLTP, OLAP, DW)
  • use one-way replication in a rollup fashion
  • Keep it simple by making sure that the unit of work is something the average programmer can pick up.
  • Stay native. If you’re writing a Scala app then use Scala libs. And when you cannot, write it yourself. And when that’s not possible then use a Java version; and plan to write it when you can.

Scala and Clojure are going to get another shot real soon.

PS: and so I’ll probably launch a jar framework for application monitoring. Anyone want to play?

 
2 Comments

Posted by on 2011/08/09 in beta, ProgLang

 

Tags: , , , ,

What is the attraction to Function Programming?

01 Aug

In computer science, functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data. It emphasizes the application of functions, in contrast to the imperative programming style, which emphasizes changes in state. -Wikipedia

I’ve been a programmer for 25+ years and back in the earliest days a lot of PC-based code was implemented in x86 assembler. Later on ‘C’ became the de facto standard yet we still counted CPU instructions and IO operations.

As the web evolved and programming languages have come and gone many of the oldies are still in play. In fact many of the benefits of functional programming languages are limited or obscured by the non-functional OS and systems they rely on. Both scala and clojure rely on Java’s JVM and they also depend extensively on standard old java libs, which are not functional, instead of replacing everything with native versions as one would expect.

Haskell and Erlang, on the other hand, while they are basically clean room implementations with few dependencies other than GCC and many of those libs; at some point in the stack they touch the libs for the OS. Erlang has the benefit that the libs and overall packaging seems contained and professional where Haskell seems more chaotic and scientific. Not that one is actually better than the other… Haskell’s Snap seems to be a better platform than Erlang’s yaws, webmachine or nitrogen.

Recently there was a post on slashdot in which an anonymous coward indirectly suggested the smartest programmers don’t work for Google. I’m sure there is truth to the statement but the description that this person gives of his day to day activities are no different than I have experienced depending on the circumstances. In fact it takes more balls and bravado than high IQ to hack trading systems in the way he described. In fact I dare say that the SEC might consider an investigation into this individual; while he might be smart, some day he’s going to write a patch that his bank cannot cash.

What initially attracted me to Erlang was it’s sigma references. It seemed to me that if a phone switch was to operate at 9-sigma then my bank software should too. Now that years and several applications have passed, all written in Erlang, I have come to the conclusion that sigma is not a good enough justification for a business application. And that it should be a business decision as to which language one implements the business in.

I think there are two main reasons why functional programming is popular: 1) because it is different. 2) because the smart guys are making a lot of money with it.

Just because you say you are “different” does not make you different wen you are standing in a crowd of people who are all different too.

In response to being “different”. There is no real justification here other than curiosity. There was a time when C and Pascal, forth competed for mindshare. In fact P-Code almost became the foundation for an operating system/platform but it never had the performance or mindshare. Once serious development began on *nix ‘C’ took hold although MS-DOS was still written in x86 assembler. And so were a number of desktop environments like TopView, DesqView and many others. C was low enough to let the programmers tinker and high enough to be useful for cross platform development. And easier to debug than assembler.

And as far as what the smart guys are doing… it also reminds me of a time when I almost went into smalltalk development. At the time consultants were making 100 – 150% more than the average programmer at the time. And while a lot of good work had been accomplished at the time. Where are these guys now? Where are their applications? Who is maintaining them?

As a side note; part of the reason why Microsoft invested in .NET was partly because they wanted to give their platform a name. Some way they could bolster OS, it’s development platform and the collective ego of the .NET certified programmer. Microsoft’s choice to alter java with J++ and moving everything to their CLR was all about mindshare.

I recall interviewing for a company that did datamining of personal information. I liked the idea and complexity of the problem, however, after making it through the interview process I was told that the position required that the programmer use their internal programming language. It was internal and proprietary. Needless to say I never accepted the position.

Yahoo is primarily a PHP shop. a) to be different from the other web properties. b) to keep their employees from jumping ship.

Google is primarily a python shop.  a) to be different from the other web properties. b) to keep their employees from jumping ship.

These companies have functional applications too. But why? Is it really worth the effort? Should we really re-tool for functional programming? Consider the amount of Java code that is out there. Why would anyone rewrite it in the functional fashion? Consider all of the non-functional code that we have come to rely on. Should we really rewrite it all in a functional environment (this is the only reason I like scala and clojure)

This topic makes my head ache.  There is no rational justification for the attention to functional programming. It’s just a game.

 
6 Comments

Posted by on 2011/08/01 in for hire, ProgLang, Uncategorized

 

Tags: , , , , , , , , , ,

PEP-8 is an awful document

11 Jul

Ordinarily I do not comment on RFC or PEPs and the like. They are written by people who are either experts in their fields (hopefully languages) or people who are clearly students of the art and craft of software development. That’s not to say this I follow these guidelines to the letter… for example “do not code defensively” from the erlang best practices document makes my skin crawl.

So as I re-review PEP-8 I’m reminded of the first time it was introduced. It was not a pretty day or a fun time. It was everything that a good software development manager tells you can and will go wrong when the inmates run the asylum. So while I think it’s generally a good idea… I hate the PEP-8 partly because it’s poorly written and partly because the way the team deployed it. Maybe they are one in the same. (I don’t think anyone referred to Knuth’s work on documentation or documented code)

So here is my summary of PEP-8 in order to cool things down and make it workable.

NOTE 1: code is often read more than it’s written.

NOTE 2: style guide is about consistency.

NOTE 3: use your best judgement.

Indentation: 4 spaces, no tabs. when wrapping lines use at least 8 spaces but keep the like items together.

Line Length: limit all line lengths to 79 chars.

Blank Lines: use sparingly. 2 between classes and 1 between methods.

Encoding: ASCII or Latin-1, and with Python 3 use UTF-8

imports: each import statement on it’s own line. group the imports by “standard python”, “3rd party :”, and “app specific”

whitespace: good but do not go crazy. Add a single space in assignments but not in function declarations.

single line: do not put multiple statements on a single line.

NOTE 4: keep your comments up to date

block comments: the same indent level as the code

inline comments: use sparingly.

NOTE 5: see PEP-257 about docstrings

version: this only applies if you use CVS or subversion.  Since I use git and hg, they do not apply.

naming conventions: whatever you use make it obvious and distinguishable. Avoid ambiguous lettering like ‘l’, ‘o’ .

package or module:

class:

exception:

global:

function:

function arguments:

method names:

instance variables:

I’m into this for an hour already and I’m bored out of my mind. This just seems so obvious to me.

design for inheritance:

programming recommendations:

So good luck with this. With any luck you’ll have a strong manager or boss. You’ll have a decent understanding and it’s won’t be confused with anything you used the other most recent language you’ve mastered. And that your intuition is something you can rely on. (remember reverse hungarian notation in OS/2 and Windows?)

 
Leave a comment

Posted by on 2011/07/11 in ProgLang

 

Tags: , ,

Perl is Better than Python – The Killer App.

05 Jul

I just started working on rewriting a fully operational acquirer gateway. I originally wrote the application in erlang and now I convinced my client to implement it in Python. I happen to like python a lot and now that perl 6 seems to be more of a fork than a version python is even more tasty. And that’s when it happened!

I wanted to “over” document the new system. Partly because the erlang version was very well undocumented but I also wanted to code the documentation inline… and I was thinking perldoc all the way. If you have not used perldoc then you don’t know perldoc. It’s the easiest way to document perl applications and I like it.

So naturally I wanted the same syntax or better for Python. Unfortunately it takes perldoc to a new level. So much so that I hate it and I want to try to reconsider my python decision. I may have to try a module or two in perl just for fun.

Perl has it’s CPAN and Python has it’s setuptools(easy_install) but nothing compares to perldoc. perldoc may be the killer app. Long live perldoc!

 
1 Comment

Posted by on 2011/07/05 in ProgLang

 

Tags: , , ,

Concurrency on the JVM

05 Jul

I just purchase a couple of books from PragProg; Programming Clojure, Programming Scala, and Programming Concurrency on the JVM. (I’m starting to think that I already purchased the Scala book some time ago and that now I have  a dupe.)

As I’m about to embark on Clojure and Scala for the 2nd or third time I’m beginning with many of the questions I had when I started on Erlang and LUA. What does it mean to me? What is the current mindshare out there? Are there any real projects? Is there enough full-time and contract work out there to warrant even the cost of the books and the time lost reading them?

And that’s when I get really frustrated because Scala has a section on “intermixing with java” and then my head starts to spin as I remember that Lift-web (a webserver written in scala) is not really written in scala. It’s actually a combination of Scala and intermixed java (notably Jetty).

And right about there is when I lose my steam. When you are a fully vested java programmer and you know all 100B jar file filled APIs with all that it means to you… it’s hard to implement anything 100% native. So that’s the dark cloud hanging over Clojure and Scala. How can they implement all the functional goodness without getting twisted up by all that legacy poison? And so I read on.

[UPDATE: Scala and Clojure seem to intermix with Java a lot. I find this painful because I be not believe that there is a way to determine where the warts are automatically. It sure would be nice if there were some sort of dependency checker so that the graph provided some illumination before projects were implemented. Or at least some developers might contribute where the holes are. I'm about to put these books on the stack again.]

 
Leave a comment

Posted by on 2011/07/05 in ProgLang, Tools

 

Tags: , , , , , , , , ,

← Older posts
 
One Page Docs

Creating a library one page at a time.

One Page Bugs

Reducing the friction of writing and fixing bugs or features.

Follow

Get every new post delivered to your Inbox.

Join 223 other followers