Nostalgia for the Future

February 8, 2010 11:07 am

This morning the Endeavour orbiter launched from pad 39A at the Kennedy Space Center carrying people and equipment into space. Only four more shuttle launches will occur and the fleet of orbiters will be retired in mid-September. When that happens, America will lose the ability to put a man in space.

Many will argue that putting people in space or on the moon is/was pointless and a waste of money. Those with this attitude will kindly turn their attention to the list of NASA spin-off technologies. This list includes (among many others) LEDs, in-ear thermometers, and improved water purification systems. There are also countless other improvements in technologies driven by NASA’s need for more efficient and more reliable equipment.

President Obama’s budget plan for 2011 removes all funding for the Constellation Program, which would have led to the development of a new vehicle for transporting humans in to space. If the cancellation of funds occurs (which is pretty close to a sure thing) then the United States will not be able to put people in space for the foreseeable future. This saddens me. We’ve given up on exploring space in any serious manner. We haven’t been to the moon since 1972. All the astronauts still living that walked on the moon are in their 70s. Another ~20 years and there will be no person alive who walked on the moon. Isn’t that kind of sad?

I wasn’t alive when any of the astronauts walked on the moon; but I am still filled with wonder at the idea of stepping foot on a different celestial body. While Star Trek is fiction, I think its tag-line embodies the sense of awe several generations have held about being able to put people into space: “…to explore strange new worlds; to seek out new life and new civilizations; to boldly go where no man has gone before”. It expresses the desire to seek out knowledge for the sake of learning.

The mission wasn’t “to find and commercialize new civilizations and technologies, to open new markets for our products, to boldy make a buck where no man has made a buck before.” Unfortunately, that seems to be mantra of our society. NASA operates on a shoestring budget, the National Labs have (essentially) been privatized to focus on making money (goodbye long-term original research projects to discover more about the universe), public education is about rubber stamping every kid’s high-school diploma, and being educated is nerdy and undesirable.

I miss the future we were promised as kids. A world where space exploration was common. A world where man’s search for knowledge and understanding overcame petty differences of culture and societal status.

——

I’ve always wanted to watch a shuttle launch and time is running out. Jess and I are hoping to take a trip to see the final flight of the Atlantis orbiter in May.

On September 24, 2010, when the Discovery lands, marking the end of U.S. manned spaceflight, I hope everyone can pause for a moment to reflect on what we’ve lost.

Grails: Dynamically building criteria queries

October 26, 2009 10:53 am

** Update: Detached Criteria are probably the better way of achieving this type of DRY principle now.  They became available in Grails 2.0; I wrote this post while using Grails 1.x. **

** Update 2: DetachedCriteria have some limitations. I wrote a cleaned up and expanded version of what this post discusses: Criteria Aggregator: Dynamic Criteria Queries in Grails **

Ok, I’ve been exploring using Groovy and Grails at work to make our team more efficient. Java is great and all, but it’s a terrible language for web development. For some quick info: A group of people decided Java could be a good web development language if it weren’t so Java-y. So they wrote Groovy. It’s fully compatible with Java, but is actually a dynamically-typed scripting language built on top of Java. Great, now we have a good web language with all the support and power of Java. Groovy compiles to the same bytecode as Java, because Java is Groovy is Java.

Well, then Ruby took the web development world by storm with the Rails architecture. Finally web developers could focus on the actual interesting part of the job rather than continually writing frameworks to run their sites. It was such a good idea that it was very quickly mirrored into other languages. Python got Django (my personal favorite and the future of the 100 Hour Board) and Groovy got Grails.

Anyway, if you’ve found this post by Googling you probably know all that so let’s get to the meat of the matter.

In my effort to learn Groovy and Grails I wanted to be able to build up Criteria Queries dynamically. Django lets you do this incredibly easily by passing around queryset objects that have lazy loading. Grails interfaces with Hibernate using some convenient and easy closure work with access to the Hibernate Criteria Builder. So you can easily and quickly define your query but only within that closure that gets passed to the builder. I wanted to be able to piece my query together from other pieces in order to adhere to the Don’t Repeat Yourself (DRY) principle.

In my searching I came upon a blog post by Geoff Lane at zorched.net entitled “DRYing Grails Criteria Queries“. His method involves extending the metaclass of the HibernateCriteriaBuilder object using Groovy’s expando abilities. This is a decent solution, but not incredibly flexible. I needed something better.

For 3 days I relentlessly scoured the Internet searching for answers, I coded all sorts of tests, I digested the source code of the Grails HibernateCriteriaBuilder class, I poked, I prodded, I made wild accusations. All in vain.

I decided that one of the following conclusions must be true:
1. It was so obvious that I just couldn’t see it.
2. I was too stupid to come up with a good solution.
3. I was asking the wrong question.
4. I needed more experience with Groovy and working with closures.

Now, I’m not entirely convinced that #3 isn’t true, but #4 also happened to be true. I found my solution this morning when I actually paid attention to what the error message was really telling me rather than just “it didn’t work”.

So, without further ado, here is how you can achieve DRYness with Grails when dynamically building criteria queries, without dropping all the way back into Java.

For the sake of this example let’s assume we have an Account class with the properties “accountType”, “accountAction”, and “owner”.

Our automatically generated code to list objects would look something like this (in our AccountController.groovy file):

def list = {
  params.max = Math.min( params.max ? params.max.toInteger() : 20,  100)
  params.offset = params.offset ? params.offset.toInteger() : 0

  def results = Account.createCriteria().list(max:params.max, offset:params.offset) {
    order(params.sort ? params.sort : "id", params.order ? params.order : "asc")
  }
  [ accountInstanceList: results, accountInstanceTotal: results.totalCount ]
}

That’s all well and good, but let’s add the ability to sort based on the name of the account owner, and display only accounts belonging to a specific user:

def list = {
  params.max = Math.min( params.max ? params.max.toInteger() : 20,  100)
  params.offset = params.offset ? params.offset.toInteger() : 0
  def req_owner = User.get(params.id)

  def results = Account.createCriteria().list(max:params.max, offset:params.offset) {
    if (params.sort == 'userName') {
      owner {
        if (req_owner) {eq("id", req_owner.id)}
     order("lastName", params.order ? params.order : "asc")
     order("firstName", params.order ? params.order : "asc")
      }
    } else {
      if (req_owner) {owner {eq("id", req_owner.id)}}
      order(params.sort ? params.sort : "id", params.order ? params.order : "asc")
    }
  }
  [ accountInstanceList: results, accountInstanceTotal: results.totalCount ]
}

Still fine, but what if we want to build off of this for different controller methods? If I want a page that does the same thing, but limits the display to only accounts of a certain account type I can use the method described in the link above by Geoff Lane. But suppose I want to further provide another page that limits it further or several pages that provide just slightly different listings based on criteria changes. Perhaps there is a great easy way to do this in Grails, and if so, please comment and tell me, because I couldn’t find it.

Here’s how I do it. First let’s pull the code we have to a reusable location. We make a self-standing closure and then curry the “params” variable to it:

def account_list = {params ->
  def req_owner = User.get(params.id)
  if (params.sort == 'userName') {
    owner {
      if (req_owner) {eq("id", req_owner.id)}
      order("lastName", params.order ? params.order : "asc")
      order("firstName", params.order ? params.order : "asc")
    }
  } else {
    if (req_owner) {owner {eq("id", req_owner.id)}}
    order(params.sort ? params.sort : "id", params.order ? params.order : "asc")
  }
}

def list = {
  params.max = Math.min( params.max ? params.max.toInteger() : 20,  100)
  params.offset = params.offset ? params.offset.toInteger() : 0

  def results = Account.createCriteria().list(max:params.max, offset:params.offset, account_list.curry(params))
  [ accountInstanceList: results, accountInstanceTotal: results.totalCount ]
}

The trouble is when we try to combine self-standing closures.

Everytime I tried something that seemed like it should work I’d get errors about “AccountController.isNotNull not applicable for arguments…” or whatever it said. I didn’t think hard enough about what it was really saying to me. The additional closure pieces were getting the wrong scope for whatever reason and instead of calling methods on the CriteriaBuilder object they were calling them on the AccountController object.

Here’s the solution:

def crit_a = {crit ->
  crit.accountAction { 
    ne("name", "None")
  }
}

def crit_b = {crit ->
  crit.accountAction {
    ne("name", "Renew")
  }
} 

def account_list = {crit, params ->
  def req_owner = User.get(params.id)
  if (params.sort == 'userName') {
    crit.owner {
      if (req_owner) {eq("id", req_owner.id)}
      order("lastName", params.order ? params.order : "asc")
      order("firstName", params.order ? params.order : "asc")
    }
  } else {
    if (req_owner) {crit.owner {eq("id", req_owner.id)}}
    crit.order(params.sort ? params.sort : "id", params.order ? params.order : "asc")
  }
}

def list = {
  params.max = Math.min( params.max ? params.max.toInteger() : 20,  100)
  params.offset = params.offset ? params.offset.toInteger() : 0

  def critBuilder = Account.createCriteria()
  def crit_closure = {
    account_list.curry(critBuilder, params)()
    crit_a.curry(critBuilder)()
    crit_b.curry(critBuilder)()
  }

  def results = critBuilder.list(max:params.max, offset:params.offset, crit_closure)
  [ accountInstanceList: results, accountInstanceTotal: results.totalCount ]
}

You get a reference to the CriteriaBuilder object using Account.createCriteria(). Note that you get a CriteriaBuilder object not a Criteria object like much of the documentation suggests. You curry this to the closures you want to use, and call them inside a new closure which you pass to the CriteriaBuilder object’s “list” method. And it works! So now you can dynamically build up your criteria queries!

America’s Broken Laws

August 12, 2009 10:26 am

Do you remember Real Player? Huge in the late 1990s. They provided software for streaming audio/video over the Internet before Macromedia blew them out of the water with Flash. They’ve been quiet for a while, but recently they re-emerged with their latest product: RealDVD. It is software that allows you to make a backup of a DVD.

They knew that this was going to get them sued by the MPAA, and they were looking for the fight. As much as I (and millions of others) hated RealPlayer back in the 1990’s I (and millions of others) have been on their side in this battle.

So what is the battle? DVDs are “protected” with the Content Scramble System (CSS). In 1999 DVD Jon gained his name by cracking this system; allowing anyone to access the data stored on DVDs without paying the fee required to get the code to decrypt the content. Yes, that’s right. Any DVD player you’ve ever used; whether in your laptop, desktop, or connected to the tv; was only allowed to be produced after the manufacturer ponied up the cash for the license to legally decrypt the CSS and agreed to the demands that they wouldn’t produce a product which allows the consumer to make a copy of the DVD.

So when DVD Jon reverse-engineered CSS, and let the cat out of the bag, the MPAA was not happy. Luckily for the MPAA they had already managed to pass the Digital Millennium Copyright Act (DMCA). Section 1201 (2) Says:

`(2) No person shall manufacture, import, offer to the public, provide, or otherwise traffic in any technology, product, service, device, component, or part thereof, that–
`(A) is primarily designed or produced for the purpose of circumventing a technological measure that effectively controls access to a work protected under this title;
`(B) has only limited commercially significant purpose or use other than to circumvent a technological measure that effectively controls access to a work protected under this title; or
`(C) is marketed by that person or another acting in concert with that person with that person’s knowledge for use in circumventing a technological measure that effectively controls access to a work protected under this title.

In short: If there is any kind of copy-protection measure on a piece of media then it is illegal to access the content without paying the proper people for access (and since the people authorizing access won’t authorize any use allowing duplication we have our problem).

DVD Jon’s work is illegal under the DMCA. Fortunately for DVD Jon, he lives in Norway and the MPAA can’t really do much about it. The bigger issue for the MPAA is that the code necessary to break the encryption on a DVD is so simple it can be written on a t-shirt. Not exactly a big hurdle for people that want to use it.

So why is this an argument at all in the first place? Something is illegal, so you shouldn’t do it, right? Well, the problem is this little notion of Fair Use (also see the Electronic Frontier Foundation’s FAQ). The Fair Use doctrine says you can make a personal back-up copy of content you own.

Fair Use says you can make a backup copy of any content you own, the DMCA says it’s illegal to make that copy if the content has any type of copy-protection system in place. Take a guess which side wins in these arguments. I’ll give you a hint, it’s not us, the individual citizens of the country. If we want to make a back-up of The Fox and the Hound so that when the DVD gets all scratched and destroyed we don’t have to buy it again, we can’t.

This is the current state of the law in the United States. Absolutely ridiculous and inconsistent. The RealDVD case was decided today by U.S. District Judge Marilyn Hall Patel in San Francisco (ruled against Real). In her remarks she made this absurdity very clear:

So while it may well be fair use for an individual consumer to store a backup copy of a personally owned DVD on that individual’s computer, a federal law has nonetheless made it illegal to manufacture or traffic in a device or tool that permits a consumer to make such copies,

(For a complete article on the matter see Wired’s Judge Rules DVD-Copying Software Is Illegal)

Django – Cronjobs Made Easy!

August 8, 2009 3:04 pm

For those that have no interest in reading about my nerd-ventures, you can stop reading this post now.

If you’re still reading, don’t say I didn’t warn you.

As has been mentioned previously (mainly on my previous blog), I’ve been doing a fair bit of side project work using the Django Framework. Sadly, the out-of-the-box Django doesn’t provide a solution for running cronjobs (for tasks that need to be run within the Django environment).

Since that’s a fairly common requirement I didn’t think it was going to be a big deal, but there wasn’t a really solid solution out there. There are a few different attempts, but they each have some limitation. There’s django-cron but that just skips over the native cron entirely, which I felt was a bit extreme. Cron can already do a good job of waking up and running a command, so duplicating that functionality doesn’t seem necessary. It also self-declares that it is designed for frequent tasks (hourly or more frequently), which doesn’t work for me. Tasks on the Board need to be able to run from a minute scale to a daily scale and beyond.

Then I found this guy’s method, which works, but I’d like a little more integration. That way when developing apps the “go add cron job” isn’t a separate step. I want the job information to right in with the rest of my app information. That way I can see what should be happening and when.

That’s when I came across Django-Chronograph. This solution was 95% of what I wanted. It provides a nice interface to the system to monitor your jobs and view logs. It requires only a single crontab entry. It uses the iCalendar style of task declaration so you have total control of when your jobs run. However, it is limited to running commands through the Django Management system. I wanted something a little more programmatic. Such that I could just point at whatever function I wanted for my jobs.

So I took Django-Chronograph and started my modifications. The result is Django Cron Manager. Setup is very similar to using the Admin system. You call the cron_manager.autodiscover() function from your urls.py file. This goes out and inspect your installed apps and registers any Cron Jobs they declare. Then, using the guts of Django-Chronograph, it keeps track of these jobs in the database and monitors when they need to run.

I’m planning on posting all the code with an example at some point, but I’m going to try to get in touch with Weston (the guy who wrote Django-Chronograph) to see if he just wants to roll my changes into his system permanently. If you stumble upon this post and the changes aren’t in Django-Chronograph, and I haven’t provided any further information. Just leave a comment that you’re interested in the code with a way to contact you and I’ll get something to you.

*** Update ***
I’ve posted the code here: http://code.google.com/p/django-chronograph/issues/detail?id=15