Recent Updates RSS Hide threads | Keyboard Shortcuts

  • Server Side Javascript Continued – Node.js (plus example)

    britg 8:00 am on July 1, 2009 | Comments Permalink | Log in to leave a Comment
    Tags: comet, , long-polling, node,

    In my previous post on server-side javascript (SSJ) I took a quick look at Jack, a project that aims to implement the Rack/WSGI pattern for javascript. I still think this approach is great as it opens the door for more traditional Rails/Django-esque frameworks for SSJ.

    But, lets face it, the next gen web is all about real-time interactivity, and current popular environments and servers just aren’t ideal for that. It’s not their fault, up until recently we only cared about getting that request handled and out the door as quickly as possible with nothing shared between requests. Unfortunately, it’s no longer just about number of requests/sec — we now need high concurrency, long-lasting connections, and shared persistence over these connections.

    Node.js

    Enter node.js – a high performance javascript project built ontop of Google’s V8 runtime. From the author’s description:

    Node’s goal is to provide an easy way to build scalable network programs. … Each connection is only a small heap allocation. This is in contrast to today’s more common model where OS threads are employed for concurrency.

    Nice, but does this pan out in implementation? After spending a few days with Node, I truly believe that this will be the go-to project for the future of the real-time web.

    A Simple Game Lobby

    Let’s take a look at a simple example I put together. The following is a very basic game lobby that is based on a more complex project I’m working on with node. (You can checkout this script from github as well).

    The script accepts new players through a url like /join?player=joebob. Then, the client can long-poll the URL /wait and receive a notification in real-time when new players join!

    First, lets define a couple of Arrays that will hold our persistence in-memory.

    // our in-memory list of player
    var players = [];
    
    // our in-memory list of players waiting
    var waiting = [];
    

    Next, lets define a set of URLs our server will respond to. Notice that the /wait response does not take place immediately. Instead, the response is captured in a callback function that is held in-memory until it is called. These callbacks are called whenever a new player hits the /join URL.

    // define a set of paths that respond to requests
    var paths = {
    
      /**
       * Requests to /join add players to our
       * player list, and fire off a notification
       * to all our waiting players
       **/
      "/join": function(req, res) {
    
        // extract the player from the request
        var newPlayer = req.uri.params.player;
    
        // respond to this request with a list of players
        // already in the lobby
        server.respond(200, players);
    
        // add this player to list of players
        players.push(newPlayer);
    
        // notify all of our waiting players that
        // a new player has joined
        while(waiting.length > 0) {
          waiting.shift().callback.apply(this, [newPlayer]);
        }
      },
    
      /**
       * Requests to /wait holds the connection
       * open until another player joins
       **/
      "/wait": function(req, res) {
    
        // define our waiting player and the notification
        // callback to trigger when another player joins
        var waitingPlayer = {
          "player": req.uri.params.player,
          "callback": function(newPlayer) {
             server.respond(200, newPlayer);
           }
        };
    
        waiting.push(waitingPlayer);
      }
    };
    

    Finally, we define our server. We tell the server to map requests to the paths we defined above, and to listen on port 8000.

    // Define a new HTTP Server
    var server = node.http.createServer(function (req, res) {
    
      // tell our server to look at the paths definition above
      // for a responder to the request
      paths[req.uri.path].apply(this, [req, res]);
    
      // respond to a request
      function respond(status, obj) {
        var body = JSON.stringify(obj);
        res.sendHeader(status, [ ["Content-Type", "text/json"]
                             , ["Content-Length", body.length]
                             ]);
        res.sendBody(body);
        res.finish();
      }
    });
    server.listen(8000);
    puts("The game lobby has started!");
    

    To run the script, first download and build node, and then download this script from my repo. Execute the script with:

    > node gamelobby.js

     
    • Hagen 2:11 pm on July 1, 2009 Permalink

      Nice article!

      But a very bad idea to show HTTP example code completely ignoring request verbs. People will copy that :-/ And don't make people join on GET, use PUT to join, DELETE to unjoin.

    • britg 2:34 pm on July 1, 2009 Permalink

      True, I plan to make some updates, the first of which is to use POST on the /join URL and enforce it. Also, I will implement a reaper process to clear stale joins.

    • TJ Holowaychuk 5:28 pm on July 1, 2009 Permalink

      check out http://github.com/visionmedia/express/tree/master
      might like ;) , I just started it but contribution would be great, building a framework on node.js is sexxxxy

    • britg 5:47 pm on July 1, 2009 Permalink

      Very cool, will watch its progress on github :) I like the Sinatra-esque lightweight approach.

  • Nginx Proxies with FirePHP

    britg 10:34 am on June 16, 2009 | Comments Permalink | Log in to leave a Comment
    Tags: firephp, nginx

    FirePHP_Large_WhiteFirePHP is a great plugin for php+ajax devs — in essence, it captures error and logging output and sends it over HTML headers instead of the content body. These logs show up in your firebug console alongside your javascript logs, making it great for ajax development. You want your output clean but your error and logging info accessible, and lets face it, tail -f is just not as sexy as firebug.

    If you’re running your development environment behind an nginx proxy, you may run into some 502 Bad Gateway errors after enabling FirePHP.

    [error] 103#0: *839 upstream sent too big header while reading response header from upstream

    This happens because FirePHP’s output can get pretty large, and nginx by default tries to cap header output. The fix is simple, in your nginx.conf simply add the following lines:


    proxy_buffer_size 16k;
    proxy_buffers 4 16k;

    You can bump up the header size (the last parameter) to whatever is required by your app to stop throwing errors.

     
    • nlsmith 4:38 am on June 20, 2009 Permalink

      Your last article talked about server-side js, you can use firephp on that, too: http://nlsmith.com/projects/console/ (and rails: http://github.com/smith/firephp_rails/tree/master)

      I ran into the same type of bug on Jaxer and to a lesser degree, ASP.

      I'm planning on making it Jack middleware, too.

    • britg 1:25 pm on June 20, 2009 Permalink

      Very cool project you put together! I will definitely give it a try — I'm finding that I can't live without FirePHP now. I'm sure that will carry over to my server-side JS projects and this sounds like the perfect solution. Cheers!

    • britg 1:30 pm on June 20, 2009 Permalink

      And yes, I think Jack support is a great move for the project — I think it'll find more adoption on Jack than Jaxer. Nothing wrong with Jaxer, but I have a feeling that Jack will be widely adopted going forward just because of it's WSGI/Rack approach.

  • The Brave New World of Server-Side Javascript

    britg 7:00 am on June 8, 2009 | Comments Permalink | Log in to leave a Comment
    Tags: jack, javscript, , narwhal,

    Not the B Team

    Not the B Team

    Every once in a while I hear whispers of server-side javascript and I get just as excited as the next person. But, I’ll admit that until recently a phrase like “Javascript on Rails” was on the same level as a phrase like “McGyver vs. The A Team – The Movie.” Fictonal. But awesome in theory.

    (Don’t worry, they eventually join forces in the sequel)

    But what about Jaxer?

    My limited experience with server-side javascript (SSJ) has been through Jaxer, from Aptana.  It touts some cool functionality, like loading the DOM server-side and built-in database drivers for MySQL, with the former allowing for DOM manipulation and integration with libraries like jQuery.

    The extremely original Gear logo

    Ubiquitous "Gear" Logo

    But, in its attempts to blur the lines between client and server execution, Jaxer has made sacrifices in its extensibility. It is meant to be plugged into another system. For instance, I don’t know of an easy way to expose data through a RESTful JSON/XML API (like so many sites are doing nowadays) using Jaxer alone.

    What I’m really looking for in SSJ is something akin to the ecosystem surrounding languages like Ruby and Python.  That’s why I was pleasantly surprised to stumble accross a few projects that aim to do just that.

    Narwhal – Javascript Standard Library and Interactive Console

    The foundation of any good ecosystem is a standard library.  Enter Narwhal, “a flexible javascript standard library.”

    There are a lot of things to like about this project, but the primary benefit is its attempts to play nice with multiple javascript runtimes like Rhino and V8cgi.  Also, this project is embracing Mozilla’s ServerJS standard which I think is important for its longevity and interoperability.

    Setup is a breeze, you can simply clone the project from github:

    git clone git://github.com/tlrobinson/narwhal.git

    Once you add narwhal/bin to your $PATH, you can use the convenient symlink js to enter an interactive javascript console similar to irb or python.

    Jack – Javascript’s “Rack”

    Interestingly enough, Narwhal came into existence as it’s author, Tom Robinson, was working on Jack.

    JSGI is a webserver interface for JavaScript inspired by Ruby’s Rack (http://rack.rubyforge.org/) and Python’s WSGI (http://www.wsgi.org/).

    Jack is an implementation of JSGI compatible handlers (to connect to web servers), middleware (to intercept and manipulate requests to add functionality), and utilities (to make using JSGI easier).

    Setup is also breezy. Simply clone Jack into the same directory you cloned Narwhal into:

    git clone git://github.com/tlrobinson/jack.git

    Since you’ve already added narwhal’s bin directory to your path, lets just make a symlink to jackup from there.

    From your narwhal/bin directory:
    ln -s [path/to/jack]/bin/jackup jackup

    Run jackup -h for usage.

    So, what’s next? Where’s Javascript on Rails?

    With Narwhal and Jack, you can start writing basic web apps. Looking at the example.js script in jack/examples we can see the basic request/response structure. Looks familiar huh?

    var Jack = require("jack");
    
    var map = {};
    
    // an extremely simple Jack application
    map["/hello"] = function(env) {
        return [200, {"Content-Type":"text/plain"}, ["Hello from " + env["SCRIPT_NAME"]]];
    }
    
    // apply the URLMap
    var app = Jack.ContentLength(Jack.URLMap(map));
    //...
    

    While these two pieces are a good start to a vibrant server-side javascript ecosystem, there’s still a long ways to go to before it’ll be on par with Ruby and Python.

    What SSJ needs is its Django/Pylons/Rails/Merb. There are a couple of projects I’ve found in the wild that are using Jack and Narwhal, but they appear to still be in very rapid flux.

    Nitro

    Nitro provides a library of carefully designed middleware and utilities for creating scalable, standards-compliant Web Applications with JavaScript.

    Helma NG

    Helma NG consists of several components that can be used together or alone:

    1) A compact JavaScript runtime environment based on Mozilla Rhino. It adds
    to Rhino a reloading module system that is compatible to the ServerJS
    Securable Module proposal.

    2) An interactive shell with support for autocompletion and history.

    3) A module library implemented in JavaScript, covering basic functionality
    such as extensions to the built-in objects, file I/O, logging, persistence,
    client and server side HTTP support and more.

     
    • Weekly Digest for June 11th » Brit Gardner :: Web Developer :: Dallas, Texas 12:06 am on June 11, 2009 Permalink

      [...] The Brave New World of Server-Side Javascript — 12:00pm via [...]

    • nlsmith 2:22 am on June 19, 2009 Permalink

      Nice post.

      You can do REST/JSON with Jaxer (see http://www.aptana.com/jaxer/guide/develop_web_s...), though it's a pain to get running, and Jack would be much better suited to this type of thing.

      As far as the JS on Rails type of thing, check out ActiveJS (http://activejs.org/.) It has a way to go yet, though any contributions would be appreciated. It does have code to make it a ServerJS module, though I haven't seen it in use. All of the modules either work with Jack and Jaxer or could be made to with a little work.

      Another project that might be of interest to you is Persevere (http://persvr.org/.)

      Server-side JavaScript has been around for over 10 years, but it just now is starting to get exciting. In fact, in my day job I work on “classic” ASP in JavaScript. Exciting times.

      Thanks,

      Nathan

    • britg 2:51 am on June 19, 2009 Permalink

      Interesting, thanks for pointing me to those two projects. Perservere looks very interesting, and I immediately associated it with CouchDB because of the RESTful HTTP interface and JSON storage. But two bullets that jumped out at me that I don't know that CouchDB can tout yet is (plus I'm sure there are many more differences):

      # Comet-based data monitoring capabilities through HTTP Channels with Bayeux transport plugin/negotiation support

      # Data-centric capability-based object level security with user management, Persevere is designed to be accessed securely through Ajax with public-facing sites

    • Server Side Javascript Continued – Node.js (plus example) » Brit Gardner :: Web Developer :: Dallas, Texas 8:04 am on July 1, 2009 Permalink

      [...] Recent Comments britg on Nginx Proxies with FirePHPbritg on Nginx Proxies with FirePHPnlsmith on The Return of the HREF – The Sammy Javascript Microframeworknlsmith on Nginx Proxies with FirePHPbritg on The Brave New World of Server-Side Javascript [...]

  • Weekly Digest for June 4th

    britg 12:00 am on June 4, 2009 | Comments Permalink | Log in to leave a Comment

    googlereader (feed #4)
    Shared 2 links.
    2:25pm via Google Reader
    twitter (feed #3)
    Interesting that Github:FI is a compiled JRuby app, does that mean no tweaking the source? – http://fi.github.com [#]
    3:14pm via Twitter
    twitter (feed #3)
    also would be nice to have the gist.github app inside a private network – I’m assuming github:fi does not include gist? [#]
    3:16pm via Twitter
    delicious (feed #2)
    5:38pm via Delicious
    delicious (feed #2)
    6:45pm via Delicious
    blog (feed #1)
    10:35pm via britg.com
    twitter (feed #3)
    Pan Seared Salmon with Mushroom Risotto and Baby Spinach Salad http://britg.com/?p=942 [#]
    10:37pm via Twitter
    delicious (feed #2)
    10:46pm via Delicious
    twitter (feed #3)
    HAML in CakePHP – http://bit.ly/155oDX [#]
    3:02am via Twitter
    delicious (feed #2)
    6:15pm via Delicious
    twitter (feed #3)
    I absolutely love the wordpress theme http://mixcss.com is using. Need to find a project to use it on. Theme is: http://bit.ly/vpQJ9 [#]
    6:19pm via Twitter
    delicious (feed #2)
    8:22pm via Delicious
    twitter (feed #3)
    APE Project looks like Comet-made-easy – Ajax Push pluggable into Apache http://www.ape-project.org/en/about.html [#]
    8:53pm via Twitter
     
  • Pan Seared Salmon with Mushroom Risotto and Baby Spinach Salad

    britg 5:35 pm on June 2, 2009 | Comments Permalink | Log in to leave a Comment
    Tags:

    Pan Seared Salmon with Mushroom Risotto

    Pan Seared Salmon with Mushroom Risotto

     
  • How To Apply Full CSS Reset on an Arbitrary Selector with Compass

    britg 6:42 pm on May 30, 2009 | Comments Permalink | Log in to leave a Comment
    Tags: compass, css, sass

    compassCompass comes with a very handy utility for completely resetting the styling accross all browsers. In the author’s own words on the github wiki:

    The Reset module provides mixins that allow you to reset your html so that every element has browser-supplied no styling

    The default compass stylesheet comes with this module included, i.e. the @import compass/reset.sass you see at the top, and spits out the reset styles into the global namespace.

    But, lets say you have a separate stylesheet that is to be included on a third party domain and that is namespaced under a selector. You can still take advantage of the Compass reset styling through the following method:

    @import compass/utilities/general/reset.sass
    
    #hh
      +global-reset
    

    The result is the complete resetting of every child element of your namespace, i.e:

    ...
      #hh div, #hh span, #hh object, #hh iframe, #hh h1, #hh h2, #hh h3, #hh h4, #hh h5, #hh h6, #hh p,
      #hh pre, #hh a, #hh abbr, #hh acronym, #hh address, #hh code, #hh del, #hh dfn, #hh em, #hh img,
      #hh dl, #hh dt, #hh dd, #hh ol, #hh ul, #hh li, #hh fieldset, #hh form, #hh label, #hh legend, #hh caption, #hh tbody, #hh tfoot, #hh thead, #hh tr {
        margin: 0;
        padding: 0;
        border: 0;
        font-weight: inherit;
        font-style: inherit;
        font-size: 100%;
        font-family: inherit;
        vertical-align: baseline; }
    
    ...
    
     
  • Using a Location Proxy for URL Bindings in the Sammy Javascript Framework

    britg 1:21 am on May 25, 2009 | Comments Permalink | Log in to leave a Comment
    Tags: , sammy

    As I hinted in my previous post, the Sammy javascript framework is a great organization layer ontop of javascript heavy apps because it brings web developers back to familiar ground – the URL. But, there are occasions where we don’t want the actual URL in the browser to change, including the hash parameters.

    Why? Maybe we don’t want Back-button accessibility, or as in my case, we are running our scripts on a 3rd party domain and we want to minimize our impact.

    So, how can we maintain the organization that URLs provide, but prevent the actual URL from changing? Use a proxy!

    You can find my fork of the Sammy project here.

    Usage

    
    var app = $.sammy(function(){ with(this) {
    
      // denote that we will be using a location proxy
      use_location_proxy = true;
    
      get('#/home', function() {
        //... do something
      });
    
      //...
    });
    

    And the corresponding link would look like:

    
    Home
    

    Even though we are preventing the URL from changing in the browser (with return false), the Sammy framework still activates the ‘#/home’ routing.

    Some things to note: if you define use_location_proxy in your application, it will no longer listen to the browser URL. Future versions may have more flexibility with this.

     
  • Erlang Supervisor Behavior Gotcha – Worker Init Params

    britg 9:26 am on May 23, 2009 | Comments Permalink | Log in to leave a Comment
    Tags: , supervisor

    When starting up a behavior like supervisor, the start_link func looks something like:

    start_link() ->
      supervisor:start_link({local, ?MODULE}, ?MODULE, []).
    

    Which in turn calls the initialization callback:

    init(_Args) ->
      %% do some stuff.
    

    Notice that the initialization callback takes 1 argument event though the params list (the third parameter) is empty in your start_link func.

    So, naturally (am I alone in this?) I assumed that when defining a Child Spec for a Supervisor, the {M, F, A} definition would follow the same pattern:

    Childspec = {child1,
                {child1, start, []},
                permanent, brutal_kill, worker, dynamic},
    %% ...
    

    But the above definition does not call child1:start/1, where the one parameter is the emtpy list. Instead, the list items are extracted as individual parameters. So {child1, start, [p1, p2, p3]} would call child1:start/3, and {child1, start, []} calls child1:start/0. Just something to look out for if you’re new to erlang and the supervisor behavior like me.

     
  • britg 5:56 am on May 23, 2009 | Comments Permalink | Log in to leave a Comment

    • “When a man runs from fatherhood, he’s not really running from responsibility, he’s running from the guilt of a mediocre life”http://bit.ly/ #
    • last tweet’s link: http://bit.ly/gU6y4 #

    Powered by Twitter Tools.

     
  • britg 5:56 am on May 22, 2009 | Comments Permalink | Log in to leave a Comment

    • Voxtrot: Blood Red Blood http://dora.fm/Xu #dora #
    • when my commit messages turn into grunts, it’s time to take a break. latest commit message: “blah” #

    Powered by Twitter Tools.

     
c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
esc
cancel