Fleegix.org

Major Geddy updates

2012-08-23 19:10:09

In the most recent round of updates, Larz Conwell has done a shit-ton of amazing work on templates: making the templating system play nicely with various formats: EJS, Mustache, and Jade. It can now happily figure out what sort of template from the file-extension, and you can even mix and match different formats together in layouts/partials.

He's also implemented Rails-like view-helpers like urlFor and linkTo, making your views simpler and more DRY.

Last but not least, he's vastly improved the generators to support full scaffold-generation, not just minimal resources.

What else is changing?

There's a lot of work happening on Geddy -- mostly breaking the project apart to make it more modular.

Utilities

The large set of utility functions (most of which originated in the now-ancient Fleegix.js library) have been pulled out into their own library, Utilities.

This classic collection of JavaScript utilities is of course available on NPM. String-manipulation, datetime stuff, mixin, filesystem utilities -- there's a ton of useful shit in there.

Model

Even more importantly, the Geddy model code now resides in a separate project, simply called Model, which is a general-purpose, datastore-agnostic ORM library in JavaScript.

I built Postgres support first, but back-porting Geddy's existing support for MongoDB, and building a Riak adapter, are next in line. Model is also installable via NPM, but documentation hasn't moved over from the Geddy project, so it's still rather DIY.

When is this happening?

The libraries are usable as-is, but the Geddy work for this lives in a branch -- but as you see, it's solid enough to build real applications with, so it will be landing soon. I'm pretty excited to have a solid framework for building Web apps like this in JavaScript.


Fleegix on Geddy

2012-08-23 02:59:00

It's taken some time, a lot of late-night hacking, and tons of work by contributors like Daniel Erickson and Larz Conwell, but the Geddy Web framework is starting to grow up and become pretty fucking solid.

It's finally far enough along I can replace my old blog application, which has been running on the Merb-based PmpknPi blog-engine for ages (anybody remember Merb?), with a very simple Geddy app. I've actually waited a really long time for this.


Older programmers with families considered harmful?

2012-02-03 11:24:00

At a recent meetup, a guy in the community (a guy whom I generally have a lot of respect for) said one of the stupidest things I've heard in a long time (and that's saying something, given the silly shit I hear coming out of my own mouth day to day).

We were talking about an open-source project I started working on quite a while back, and why development on it kind of stalled for a while. He was totally convinced that the reason it had stalled was because I'm an older guy with a family. I told him no, it was because my work on Yammer takes up most of my time. We're pretty busy these days at Yammer.

Now, this guy is at a startup, so I would assume he'd totally get this, but bizarrely, he acted as if he hadn't even heard me (we were at a noisy bar; I guess it's possible somehow he actually didn't hear me), and continued to hold forth, saying, with a big grin, "that's why I'd never hire an older guy with a family."

Yeah, sure, I'm rolling my eyes at this because I'm one of those older dudes with a family. But you can see this same silliness in other places: "I'd never hire a woman, because she might get pregnant and have to quit." "I'd never hire a person without a comp-sci degree from an elite univerity and a high GPA, because we want smart people." "I'd never hire a person from Uzbekistan, because fluctuations in their currency's exchange-rate might distract them from their work." "I'd never hire a person who's lactose-intolerant, because we'd have to keep that special milk in the office." None of these make any fucking sense either.

As wrong as it is, I guess I can understand where the idea comes from -- it seems obvious that younger folks have a lot more time to hack. That is, unless they're spending their free time rock-climbing, or snow-boarding, hanging out at bars, or even, say, recovering from hangovers (okay, I still do this last one from time to time, too).

A previous startup where I worked was populated overwhelmingly with younger dudes who prided themselves on their ability to code for days and days at a time. They produced prodigious amounts of code -- and it was some of the most appallingly bad code I've ever seen. Piles and piles of it.

It's possible that older guys with families work fewer hours (although that hasn't been my experience in startups so far), but it's also highly likely that with their experience they spend more time writing the right code. Just sayin'.


Logan, client- and server-side JavaScript test runner

2010-07-17 22:28:00

Some of the work I'm doing at Yammer involves writing JavaScript code that runs in different environments. We're using geddy-model code in the browser, as well as with TheRubyRacer (which embeds the V8 JavaScript interpreter in a Ruby process).

This code is part of the Geddy web framework I have been building for Node.js -- so this means three different environments, and and four different JavaScript interpreters, that this code has to work within.

Multi-environment testing? Dual-sided? X-env?

I've been talking to people about ways to solve this -- in particular, Anders Conbere, who is working with similar issues at his job with Estately.

Testing multi-environment JavaScript code is an annoying problem. I looked around, I really didn't find any good solutions.

(I also didn't find anything nice and snappy to call JS code that has to run on both client and server. What's the "Ajax" of that? "Dual-sided"? "Multi-environment"? "JS-everywhere"?)

Meet Logan

I sat down to see what would be required to run the same tests in all these different environments, and it turned out to be pretty reasonable to build something that can run both brower- and server-side tests (thanks, Node.js!).

Despite having built Windmill's complex JavaScript API (or perhaps because of it), I've come around to Mikeal's way of thinking on testing: I want something minimal, without a lot of frameworky baggage -- something that just sets up some conditions and does some asserts.

What I've ended up with is Logan, a very small test runner that lets you run your tests in browsers, in Node.js, and in embedded V8 with TheRubyRacer.

The name is probably kind of obvious to anybody who's familiar with old sci-fi movies -- Logan is a runner), of course.

Logan uses a version of Node's assert module that Anders patched to remove the V8-specific code. I would love to see these changes pushed upstream to Node, so this fork doesn't have to persist (assuming there's a minimal performance impact).

Logan requires Node.js, and of course if you want to test out tests in TheRubyRacer, you need to have it installed.

Sharing code between sides

For Logan's testing purposes, "server-side" means in Node.js, which uses CommonJS modules. "Client-side" includes the browsers, and TheRubyRacer, where there is no facility for loading code modules from within the runtime. Logan uses plain requires for the server-side, and eval/script-append in the clients.

I should take a second here to call out very specifically one of the requirements is synchronous loading of dependencies. (This is why I went the route of source-code transformation.)

Once you get past some initial hackery, it's not that bad loading code in these different enviroments. (After you've done enough client-side JavaScript, I suspect your threshold for hackery gets a little higher.)

To make this stuff work, you do have to follow some specific rules in both app-code, and testing code. (The constraints for the app code are needed if you plan to run your code on both client and server at all, not specifically for running Logan tests.)

1. Use namespaces in your modules

Use a namespace object inside the module file -- you won't have module scope when you load it in the browser. Name it the same name as the variable you'll be setting it to when you load it via CommonJS require.

2. Check for the need to export

Only export your namespace in the CommonJS environment.

For example, in the module file, foo.js:

if (typeof foo != 'undefined') { foo = {}; }

foo.bar = new function () {
  this.a = 1;
  this.b = function () {}; 
}();

if (typeof module != 'undefined') { module.exports = foo.bar; }

Or, if you are exporting a constructor:

if (typeof foo != 'undefined') { foo = {}; }

foo.Baz = function () {
  this.a = 2;
  this.b = []; 
};  

if (typeof exports != 'undefined') { exports.Baz = foo.Baz; }

3. Set require result to namespace name

Use the same name as the namespace you created in the module for the variable name of the require result.

4. Create top-level namespaces non-destructively.

In your tests, if you're creating top-level namespaces to hang your required objects on, use something to create them that checks for their existence before creating.

Logan includes a utility function, logan.namespace, to help with this. In your own app's code, you'll either have to use a utility function for doing this, or simply check with a typeof check for undefined.

Here's how to load and use the previous two examples:

logan.namespace('foo');
foo.bar = require('./path/to/foo');
foo.Bar = require('./path/to/bar').Bar;

var fooTests = new function () {
  this.testFooBarAIsTruthy = function () {
    assert.ok(foo.bar.a);
  };

  this.testFooBazAIsNumber = function () {
    var barInstance = new foo.Bar();
    assert.equal(typeof barInstance.a, 'number');
  };

}();

logan.run(fooTests);

Details

The reason for these special requirements is the difference between the way that code loads on the client and the way it loads via CommonJS.

In the client environment (including TheRubyRacer), there is no module-level scope, and the mechanism for loading code doesn't return any value, so namespace creation has to happen inside the module file, and has to be careful not to rewrite anything other modules might have done.

Logan makes this work with CommonJS require syntax in your tests by doing source-code transformation -- it uses the require statements to know what code to load, and rewrites the statements so they don't stomp on the results of the previous eval operation.

If you're at all interested in the icky rewriting tricks necessary, have a look at the source for the browser runner.

The process for running tests in TheRubyRacer is pretty weird too -- Node shells out to a Ruby script that then turns right back around and embeds V8. It's a crazy world we're living in.

What's next

Logan is very new. I'm sure there are better ways to deal with the problems of X-env testing. (How about that? Yeah, "X-env"? It has the letter "X" in it like Ajax, right?) At least I hope there are better ways. I'm very interested in getting feedback on how to make improvements.


Geddy models/validations for Node.js

2010-04-26 22:53:00

The last couple of weeks have been a blast.

I've been settling into my new job at Yammer, getting to know the guys on the team, and enjoying the insanely delicious food. Yes, we have a chef that comes in; it's pretty ridiculous.

By happy accident, my team at Yammer is using the work I've done with Geddy on models and validations -- but in the browser.

It's kind of mind-bending to see code written for Node.js running in IE6, but that's a real-world example of the pure awesome we get with server-side JavaScript. No more writing the exact same input-validation in two different languages.

Geddy has an easy, intuitive way of defining models, with a full-featured set of data validations. The syntax is very similar to models in Ruby's ActiveRecord or DataMapper, or Python's SQLAlchemy.

As I mentioned, the model module works great both in Node, and in the browser, so it's very easy to share model and input validation code in your app between client and server.

Here is an example of a model with some validations:

var User = function () {
  this.property('login', 'String', {required: true});
  this.property('password', 'String', {required: true});
  this.property('lastName', 'String');
  this.property('firstName', 'String');

  this.validatesPresent('login');
  this.validatesFormat('login', /[a-z]+/,
      {message: 'Subdivisions!'});
  this.validatesLength('login', {min: 3});
  this.validatesConfirmed('password', 'confirmPassword');
  this.validatesWithFunction('password', function (s) { 
      // Something that returns true or false
      return s.length > 0;
  });
  // Can define methods for instances like this
    this.someMethod = function () {
      // Do some stuff
    };
};

// Can also define them on the prototype
User.prototype.someOtherMethod = function () {
  // Do some other stuff
};

// Server-side, commonjs
exports.User = User;
// Client-side
// model.registerModel('User');

Creating an instance of one of these models is easy:

var params = {
  login: 'alex',
  password: 'lerxst',
  lastName: 'Lifeson',
  firstName: 'Alex'
};
var user = User.create(params);

Data-validation happens on the call to create, and any validation errors show up inside an errors property on the instance, keyed by field name. Instances have a valid method that returns a Boolean indicating whether the instance is valid.

// Leaving out the required password field
var params = {
  login: 'alex',
};
var user = User.create(params);

// Prints 'false'
sys.puts(user.valid());
// Prints 'Field "password" is required'
sys.puts(user.errors.password);

I've been writing a bunch of tests so you can see some other examples.

Really cool seeing code that runs equally well on the server and in the browser. A credible server-side JavaScript plaform like Node really opens up a ton of new possibilities. Looking forward to seeing where it all goes.


What's next is Yammer

2010-04-05 17:16:00

I've just come back from a meeting at the offices of my new employer, Yammer, Inc. I've accepted a position there as Senior Front-End Engineer, doing development work primarily in JavaScript and Ruby.

I start next week, after enjoying a little family time this week -- coincidentally it's spring break for my boys.

For those who don't know, Yammer is a microblogging service for the enterprise. Think Twitter, or Facebook's News Feed, but for use in businesses.

Yammer is an excellent fit -- I've been a heavy user of microblogging and 'activity streams' since the fairly early days of Twitter.

It's easy to see the impact this new medium of communication has had on people's personal lives. I'm looking forward to helping invent how it impacts people in the the workplace.

The Yammer office is at Townsend and 5th in downtown San Franciso, very near South Park and the baseball stadium, so my commute remains a breezy 15-minutes up 280.

If you happen to find yourself in that part of town, hit me up -- I'd love to go have lunch.


REST easy with Geddy for Node.js

2010-03-25 01:20:00

Geddy is a small Web-app development framework for Node.js. It uses a router syntax similar to Rails or Merb, and now has tools for quickly creating RESTful, resource-based routes as well.

Building and installing Geddy

Prerequisites: An up-to-date version of Node.js. I'm developing Geddy on Linux (and I'm lazy), so I'm not sure yet what adjustments might be needed for it to work smoothly on OS X. I hope one of you Mac guys can tell me what's needed.

Get Geddy from GitHub and install:

git clone git://github.com/mde/geddy.git cd geddy make && sudo make install

There's currently no configure step, because the installation is really just packaging stuff up, and a little file-copying. The installation sticks Geddy into .node_libraries/ in your home directory.

Creating a Geddy app

Geddy comes with a utility called geddy-gen you can use to create an app:

mde@localhost:~/work$ geddy-gen app bytor Created app bytor. mde@localhost:~/work$ cd bytor mde@localhost:~/work/bytor$ geddy Server running at http://127.0.0.1:8000/

Go to http://localhost:8000/, and you should see:

Attention all planets of the Solar Federation

Pretty simple.

What's in a Geddy app? Well, geddy-gen starts you off with a simple app stub. The directory contents look like this:

mde@localhost:~/work$ find bytor bytor bytor/config bytor/config/router.js bytor/app bytor/app/controllers bytor/app/controllers/main.js bytor/public

There's just a single, simple default route in router.js:

var Router = require('geddy/lib/router').Router;

router = new Router();
router.match('/').to({controller: 'Main', action: 'index'});

exports.router = router;

This has the the / path handled by the index action on the Main controller.

Adding resources

Change directories into your new app directory, and use geddy-gen resource to create one:

mde@localhost:~/work/bytor$ geddy-gen resource snow<em>dogs [ADDED] ./app/controllers/snow</em>dogs.js resources snow_dogs route added to ./config/router.js

Restart Geddy, and you'll see the new route working. Hit your new route -- for example, http://localhost:8000/snow_dogs.json, and you should see something like this:

{"method":"index","params":{"extension":"json"}}

Depending on how your browser is set up to deal with the JSON MIME type, it may want to open the response in a separate application.

Resource controllers

Take a look at the new file created, app/controllers/snow_dogs.js to see what actions a resource-based controller has. The actions are methods called on the controller, and get passed a copy of the parsed parameters from the URL and query string.

Okay, so REST isn't quite the same as CRUD, but Geddy's resource-based routes do take the expedient course of creating url/request-method mappings for easy CRUD operations like this:

GET /snow_dogs
(SnowDogs controller, index action)

GET /snow_dogs/add
(SnowDogs controller, add action, for any new-resource template -- "new" is not usable as a JavaScript action name)

POST /snow_dogs
(SnowDogs controller, create action)

GET /snow_dogs/:id
(SnowDogs controller, show action)

PUT /snow_dogs/:id
(SnowDogs controller, update action)

DELETE /snow_dogs/:id
(SnowDogs controller, remove action)

Content-negotiation

Geddy has some built-in ability to perform content-negotiation based on the abilities of the client, and the requested filename-extension. Right now, this feature is very bare-bones, supporting only JSON and plaintext, but can be easily extended to support XML, Atompub, or whatever.

If you have a JSON-serializable JavaScript object you want to return in JSON format, all you have to do is set up a resource with geddy-gen resource, and in the appropriate controller action, pass your JavaScript object to the respond method.

Something like this:

this.respondsWith = ['text', 'json'];

this.show = function (params) {
  // (Fetch some item by params.id)
  item = {foo: 'FOO', bar: 1, baz: false};
  this.respond(item);
};

Geddy will automatically serialize item to JSON, and set the content-type appropriately -- assuming the client can accept JSON.

Eventually it should be possible to register handlers for new content-types on a per-controller, or per-model basis. (Doing this by way of defining a 'to_[format]' method on the objects to be returned isn't a workable solution in JavaScript, when you're likely to be returning built-in types.)

If you want to use a specific format, you can pass it as the second argument to the respond call -- even if it's not one of the listed formats in respondsWith.

Perhaps something like this:

this.show = function (params) {
  // (Fetch some item by params.id)
  item = {foo: 'FOO', bar: 1, baz: false};

  // Serve up some html
  if (params.extension == 'html') {
    var html = '';
    html += '<html><body>';
    html += '<h3>' + item.foo + '</h3>';
    html += '</body></html>';
    this.respond(html, 'html');
  }
  // Otherwise return formatted data
  else {
    this.respond(item);
  }
};

Geddy uses simple strings for the identifiers of the formats, rather than an enumeration, because it's just cleaner-looking. I prefer simply 'json' to this.formats.JSON or something like that.

What's next?

Next is work on easy handling of templates. I'm not sure how far down the HAML/SASS road I want to go initially, but basic EJS, and Moustache seem like a good place to start.


Geddy: Web framework for Node.js

2010-03-15 22:46:00

So I've posted previously about the embarrassment of riches that we have with Node.js modules, specifically Web frameworks.

I count no less than 14 frameworks and micro-frameworks for building Web-apps with Node right now on the list of Node modules.

I had actually hoped to use one of these to build a small Node app, but as it turns out, many of these are either half-finished, or are broken with current versions of Node. Pretty understandable, given how quickly Node is changing.

Express looks good, but I'm a little more interested in something that will let me build complex apps -- and the Sinatra-style routing, with the verb and everything kind of inline there, seems a little verbose. I prefer the simpler DSL of Rails-/Merb-style routes. To me it's just more expressive, flexible, and powerful.

So I started building a basic router, looking at a lot of the ideas in Bomber.js, and remembering back on how simple and hackable the Merb framework was to use.

The result is Geddy. It's a small Web-app development framework that's still very embryonic -- but it does routing and invoking controller actions, and serves static files. I'm having a blast hacking on it. It's crazy fun doing all this server-side code in JavaScript.

Of course, it's named in honor of Geddy Lee, the bassist for Rush.

Routing

Routes in Geddy look like this:

r = new Router();
r.match('/').to({controller: 'Main', action: 'index'}).name('main');
r.match('/users/:userid/lerxstizer/:lerxstid').to(
  {controller: 'Users', action: 'lerxstizer'});

This should look pretty familiar to anybody who's used Merb or Rails. The controller here is really just an execution context for the action, which is really just a handler method.

The advantage of thinking of these as 'controller' and 'action' is that it allows the execution context of the handler method to inherit a bunch of helpful methods from a base Controller.

It also leverages people's understanding of MVC so the on-ramp to building something is minimal.

Controllers and actions

On startup, Geddy scans the app/controllers directory for JavaScript files, and loads them as available controllers.

A sample controller file looks like this (in file app/controllers/users.js):

var Users = function () {
  this.lerxstizer = function (params) {
    // params should have userid, lerxstid, and any query-string params
    this.render(JSON.stringify(params));
  };
};

exports.Users = Users;

The controllers are all required and instantiated based on convention, but I'm not going all meshugah with the Railsy pluralization rules.

The controller files should be all lowercase names, with underscores separating words. The constructor function in the file should be a camel-cased, initial-capital-letter version of the same name.

Some examples:

File Constructor
app/controllers/snow_dogs.js SnowDogs
app/controllers/priests_of_syrinx.js PriestsOfSyrinx

The action is a method then called on the controller instance, and passed a params object that contains any params pulled off by matching the pattern in the router -- and any query-string params.

So for example, in the case of the 'lerxstizer' example above, if you hit the following URL:

/users/alex/lerxstizer/2112?code=YYZ

The params object will look like this:

{
  userid: 'alex',
  lerxstid: '2112',
  code: 'YYZ'
}

Static files

It's impossible to develop a Web application without serving the static assets needed. I took a look at how Paperboy and other frameworks did it, and added static-file support for Geddy.

In development-mode, if an URL matches no routes, Geddy will serve up matching filenames in the public/ directory rather than responding with a 404. Obviously this directory path needs to be configurable at some point.

Stuff to do

There's a long list of stuff to hack in:

  • Resource-based routes
  • Content negotiation
  • Templating support
  • Better error handling
  • Controller-generator script
  • Kiwi package

I also need to separate out the framework code from what will eventually be generated app code. Lots of stuff to work on, and hopefully lots of opportunity to get to know Node better.


About

This is the blog for Matthew Eernisse. I currently work at Yammer as a developer, working mostly with JavaScript. All opinions expressed here are my own, not my employer's.

Related

Previous posts

All previous posts ยป

This blog is a GeddyJS application.