RSS

Monthly Archives: November 2011

breadcrumbs, where did we come from?

30 Nov

Some author wrote a column “Moving on from Rails” (I strongly dislike self important authors who attribute their publications to their “handle” instead of their actual person). It’s mostly a well written article, however, the author did not not allow for feedback or comments.

My only complaint about the article is that it seems that this person does not have the perspective of having been through the “history repeating itself” or “waterfall” that is our industry.

For example: as servers were more expensive and more powerful “we” shifted the processing to the server. And as memory and personal computers became cheaper and more powerful “we” moved processing to the client-side. Additionally the core of the application has shifted. At one time there was a single database and now there are farms … and so the core application has shifted and the layers are stacking.

Actually, the layers are not stacking at all. They are the same as they have always been. They are simply more granular in some cases. The OSI network model still stands as a description of what we have architected.

As to why the author is moving on from RoR (Ruby on Rails) has probably more to do with “me too” rather than any real technical justification. This behavior typically happens when a programmer is starting to burn out (either with the framework or the job) and so it is not an inditement of the technology but the person.

me thinks he doth complain to much

As to the conclusion that RoR answers fewer questions is not much of a conclusion. The same can be said of any language or framework and it is terribly subjective. It is also based on the weighted important by the author and not any real metric. Which is why I think the author is unhappy.

As for me and my projects, languages and frameworks. I’m mostly client driven. I like to guide clients toward technology that I like or I’m familiar with, however, they are all general purpose enough that they could complete the job easily enough. For me the question has less to do with the tools than it does the architecture. And with that in mind it’s a better idea to have more tools in the toolbox than one tool you think is crappy.

 
Leave a comment

Posted by on 2011/11/30 in response, web

 

New Perl Monger In Weston

29 Nov

I was looking for a PM group in Weston or one of the surrounding cities. Sadly both the Miami and Delray/Deerfield groups seem to be inactive and since I live half way in between I decided to open my own group. I hope this was a good idea. We will see.

 
1 Comment

Posted by on 2011/11/29 in open source

 

Tags:

Mojolicious or TornadoWeb?

26 Nov

It depends on what you are really trying to accomplish.  In a current project I’m using TornadoWeb to get the transaction through the ZeroMQ broker to a worker. (standard request/response broker implementation).

Since I’m using TornadoWeb I needed to implement about 50 lines of glue code and I needed to hijack TornadoWeb’s IOLoop instance with the IOLoop from ZeroMQ. And since the work is only a few dozen lines of code … it’s all up for debate.

In retrospect I think I should have gone with Perl+Mojolicious+beanstalk. It would have completely eliminated my own broker .. another 100 lines of code. Then there was the direct access to redis in tornadoweb.

MEH! While I’m pragmatic that if I did not have to wtote an extra 150 lines of code then I would nto have to debug them either.

This race between the two platforms is so close! (on any given Sunday)

 
Leave a comment

Posted by on 2011/11/26 in web

 

Tags: ,

Django-App-Engine is not Django

24 Nov

I’m not going to give these guys any more press than they deserve.  I tweeted that “django-app-engine development halted”. As I tweeted the message I thought they were referring to the DjangoProject, however, I was wrong! Seems the Django project is still going strong, I don’t know who these other guys are or what their endgame is…

Anyway, Django is a strong platform. I do not use it much these days because I’m focusing on TornadoWeb and Mojolicious but it is rock solid. It’s admin tools can bootstrap just about app you might consider building.

 
Leave a comment

Posted by on 2011/11/24 in web

 

Tags: , ,

Projects and more projects

24 Nov

I have been working on my resume lately and there are about as many different formats and general approaches. For the time being I’m going with a better narrative and a list of employers. But then I got to thinking about the specific projects I’ve worked on and why it won’t fit on one page. So here is a list, even if it’s just for myself:

Florida Freelance IT LLC; System Architect / Programmer, Owner 1/2009-

  • China Union Pay – issuer integration
  • POS – PinPad integration
  • Big data reporting, audit, reconciliation and billing system for a VOIP arbitrage system
  • devops for VOIP system (resource planning and production monitoring)
  • PCI-DSS audit
  • Implement new features (bill payment internals)
  • Online merchant implementation
  • Merchant acquiring system from scratch
  • Personel allocation for outsourced merchant restocking
  • Social media spider for FaceBook and LinkedIn
  • Local Lis Pendens search tools
  • 24×7 operations support

Insight Cards Director of Platform Development 6/2010-8/2010

  • Query optimization and report validation
  • DB schema normalization
  • managed internal and external resources
  • met with VC, executives and clients

MetaVentures System Architect / Programmer 1/2008-6/2009

  • implement new features
  • designed and implemented a POS from scratch
  • designed and implemented an HSM (encryption) from scratch
  • designed and implemented a merchant acquiring system from scratch
  • 24×7 operations support

eDiets Senior Programmer III 2007-1/2008

  • implemented bug fixes
  • implemented client customizations
  • assisted in the ATG conversion

Secure Computing Senior Programmer 2005-2007

  • designed and implemented custom TCL code for SnapGear firewall
  • integrated SnapGear hardware into universal command center tool

Ricoh Corporation Engineering Manager / Programmer 2005

  • managed the QA, Dev, Operations and support teams
  • designed PaaS system upgrade for increased volume
  • designed Desktop application and delivery system
  • SQL performance tuning and optimization
  • presentations to management team and regional superiors
  • presentation at sales meetings

Premier Global Senior Programmer 2004-2005

  • designed and implemented a reporting sub-system
  • rebuilt the build system
  • bug fixes and new feature implementations

WildCard Systems Software Development Manager / Programmer 1998-2004

  • designed and implemented a general purpose authorization system
  • certified with Visa, MasterCard, American Express, Discover
  • designed and implemented 2 SQL Server replication applications
  • designed and implemented association settlement system (Visa, MasterCard, American Express, Discover)
  • designed and implement a reporting script language for special purpose reporting system.
  • 24×7 operations support
  • DBA and SysAdmin
  • visited with clients for customization or integration

First Data Corporation Team Lead 1995-1998

  • Communications TSR for hospitality system
  • designed and implemented a GiftCard authorization system
  • designed and implemented a plastics manufacturing system
  • designed 2 and implemented 1 help desk application
  • designed proprietary changes to the GiftCard magstripe so that the POS did not have to change
  • designed and implemented a “sparse” card manufacturing process
  • worked with sales to prioritize work
  • visited with clients for customization or integration
  • 24×7 operations support
  • DBA and SysAdmin
  • Supported NCR debit system

Misc Contract Programmer, Full-time & Part-time 1983-1994

  • debugged OS/2 video device drivers. Both local and remote applications
  • trained the level 1 and other level 2.5 support teams
  • developed and trained others on strategies for converting OS/2 Presentation Manager (GUI) from 16-bit to 32-bit
  • architected a replacement for the 16-bit thinking layer (dives the entire window architecture)
  • converted 2.5M lines of code including 500K lines of assembly from 16-bit to 32-bit
  • kernel level debugging in order to verify operation
  • I certified a desktop app as CUA compliant
  • designed and implemented a special purpose scripting language for testing special purpose computers
  • built a suite of regression tests for release verification of firmware
  • built a hardware and software solution for simulating signal input for the special purpose computer
  • built second generation of hardware for use in the manufacturing process for FVT(function verification testing)
  • implemented a simulation of the special purpose computer (in assembly language) for testing the desktop app
  • implemented BIOS and POST (power on self test) for the artic coprocessor
  • implemented HDD and Tape device driver for OS/2
  • provided on-site Object-Oriented training
  • developed training materials for Object-Oriented training class
  • implemented and supported commercial copy protection removal
  • file manager for DOS with 256 byte filename support
  • operating room scheduling system
  • vehicle preventative maintenance system
  • produce warehouse inventory tracking system
  • custom MailMerge application for WordStar

I’m pretty sure this is everything.

 
1 Comment

Posted by on 2011/11/24 in business, for hire, management, updates

 

Tags:

AnyEvent and memory leaks

21 Nov

Recently twitter kraih posted a gist link that demonstrated a simple memory leak. I say it’s simple now because I’ve spent enough time analyzing it. To say that it’s a beginner’s error is a bit arrogant but such as it is the bug points out a bigger issue when using AnyEvent (whether inside a Mojolicious app or not) it’s own has potential to  cause memory leaks.

In the example:

# Very common leak
my $foo;
$foo = sub {
 my ($i, $j) = @_;
 return if $i >= $j;
 say $i++;
 $foo->($i, $j);
};
$foo->(1, 10);

The bug here is caused because $foo is declared and set outside the scope of the virtual function. Then within the function the code access the variable again. That’s where the variable reference count is incremented. One way to correct this function looks something like:

# Very common leak
my $foo;
$foo = sub {
 my ($foofoo, $i, $j) = @_;
 return if $i >= $j;
 say $i++;
 $foo->($foofoo, $i, $j);
 undef $foofoo;  #this line is optional
};
$foo->($foo, 1, 10);

The corrected example is nothing special but it’s not pretty (so maybe aesthetics should be #5 or #6 on the list).

Where this is a problem for AnyEvent is that it uses a virtual callback function in just about every API. This means that if the function does not push the data you need into the callstack of the callback that you have to get it by triggering access to the variable though the scope.

my $retval = undef;
$ctx->do_somthing_cool(1, 2, 3, cb => sub { $retval = 'done'; }).recv;

The above code is going to leak memory for the same reason that the first example gives. Now there is some good and bad news. The bad news is that there is additional work to do. The good news is that it’s pretty standard and not that big of a deal. Just more aesthetics.

my $retval = undef;
$retval = $ctx->do_somthing_cool(1, 2, 3, 
                    cb => sub { my $myretval = 'done'; 
                                return $myretval}).recv;
 In AnyEvent, recv() returns the value in the “return” function inside the callback. One place I have not experimented with yet is what happens when you have more than one function. For example:
my $retval2 = undef;
my $retval2 = undef;
$ctx->do_somthing_cool(1, 2, 3, 
 cb => sub { my $myretval = 'done'; 
              return $myretval}); 
$ctx->do_somthing_cool(1, 2, 3, 
 cb => sub { my $myretval = 'really'; 
              return $myretval}); 
[$retval1, $retval2] = $cv->recv()

In this example I do not know what ->recv() is going to return and if it does return everything; what order will the results be in.

The most important thing is making sure that you do not increase the number of references unnecessarily. Voila.  I’m not a beginner any more.  *sniff*

 
2 Comments

Posted by on 2011/11/21 in web

 

Tags: , ,

Memory Leaks? Really?

21 Nov

I don’t know anything about the specifics on this one… but when the project leader makes a statement like:

I have yet to see one non-trivial #perl web application that doesn’t leak memory, wish we had cycle detection like #python does since 2000.

Ya gotta wonder whether or not this is a good idea? Is the framework the problem or the language?

When I did a search for “memory leak mojolicious” there was mention of some problems going back to version 1.56… but I’m so disgusted with the state of things that I hope he/they make a correction or statement addressing the issues… I also hope it’s not sarcasm.

I cannot wait to update this post!!!

[update #1] “Take this little example, how would you explain what’s going on to a #perl beginner? gist.github.com/1381643″ [ref: twitter: kraih]

[update #2] “Perl’s memory management is well-hidden, but is based on reference counts and garbage collection. It also has mortal variables, whose lifetimes are limited to the current context. It is possible to free(1) the memory(2) assigned to variables (including arrays) explicitly, by undef-ing the only reference to them.” (ref: http://bit.ly/tH2XJt)

[update #3] since perl uses a reference counter in order to determine what memory to release, this is not much different than Java and a host of other memory managers. It’s easy to cause a leak by having a circular linked list reference and then dropping all of the userspace references to that list. That can easily happen within a function that declares some local memory for a list of some kind and then exits without cleaning up. This is a perfect justification for functional programming style. “no side effects”

 
Leave a comment

Posted by on 2011/11/21 in web

 

Tags: , ,

Simple Economics

19 Nov

I’m not an economics major and I hated statistics but I noticed something that I think is interesting and scary at the same time; and I’m not talking alien scary.

  • my grandparents survived the great depression
  • my parents, having learned their lessons, did better than them
  • the baby boomers are entering into retirement
  • US economic policy makers are taxing the crap out of us
  • those same policy makers want us to spend spend spend in order to stimulate the economy and no incentive to save
  • I am clearly not better off than my parents
  • and as I spend over half my income on things that sustain life (food, water, insurance)

I can only hope and prey that my children do better than I am.

 
Leave a comment

Posted by on 2011/11/19 in Uncategorized

 

Tags:

Getting Mojolicious and AnyEvent::Beanstalkd to work

18 Nov

I have been having a hard time getting Mojolicious and AnyEvent::Beanstalkd to integrate. Given the commonality of AnyEvent one would think that it would a) work off the shelf; and b) there would plenty of documentation or examples. Sadly neither are true and I had to test several permutations, give up, and then try one last method (which worked).

It basically comes down to this. $cv->recv() does not work inside a Mojolicious. It causes a recursion error that AnyEvent just barely explains. The first alternative, that worked, was a matter of nesting all of the callback functions and as long as the stack is shallow that would work but it’s not pretty and it’s certainly unmaintainable.

In the end the code was replaced with $txn->cb(sub { $_[0]->recv }); and then the beanstalkd commands were flattened; and everything seems to be working nicely now. There was no explanation as to why this worked.

After looking at the AnyEvent code I think the code was able to “falsely” detect the $cv->revc() as a recursive call where the other format is undetectable.  I’m not certain whether this is good or bad but it works.

 . . .
 my $txn = AnyEvent::condvar;
 $txn->begin;
 $client->use('restmq', sub {$txn->end;});
 $txn->begin;
 $client->watch($guid, sub {$txn->end;});
 $txn->cb(sub { $_[0]->recv });

 $txn = AnyEvent::condvar;
 $txn->begin;
 my $job2 = $client->put( $msg, sub {$txn->end;});
 $txn->begin;
 $client->reserve( sub { my $job = shift;
   my $decode = $job->decode;
   my $uuid = (@$decode)[0][0]->{'uuid'};
   my $retval = sprintf "data> %s, %s\n", $job->data, $uuid;
   $client->delete($job->id);
   $client->ignore($guid);
   $client->quit;
   $self->render(text => $retval);
   $txn->end;
 });
 $txn->cb(sub { $_[0]->recv });
 $self->render_later;

The complete code is here. I’ll be posting my scratch code shortly and I’ll be updating and commenting the code too.

 
Leave a comment

Posted by on 2011/11/18 in web

 

Tags: , , ,

Benchmark – Hello World – TornadoWeb vs Mojolicious

17 Nov

I was running a simple hello world benchmark. It was meant to be as simple as possible. Mojo clearly wins the LOC comparison but TornadoWeb wins the race.  Mojo produced 20TPS and TW produced 22TPS. I used the basic run commands and I used siege from JoeDogs.

Mojo:

#!/usr/bin/env perl
use Mojolicious::Lite;
# Simple delayed rendering
get '/' => sub {
 my $self = shift;
 $self->render_text('3 seconds delayed!');
};
app->start;

TornadoWeb:

 
#!/usr/bin/env python
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
 def get(self):
 self.write("Hello, world")
application = tornado.web.Application([
 (r"/", MainHandler),
])
if __name__ == "__main__":
 application.listen(8888)
 tornado.ioloop.IOLoop.instance().start()
 
Leave a comment

Posted by on 2011/11/17 in beta, web

 

Tags: , ,

← Older posts
 
One Page Docs

Creating a library one page at a time.

One Page Bugs

Reducing the friction of writing and fixing bugs or features.

Follow

Get every new post delivered to your Inbox.

Join 223 other followers