Migrating from Songbird to Rhythmbox

June 4, 2010 4:40 pm

Once upon a time a company created Songbird. A music player designed to supplant all other music players and have Linux compatibility with iPod support. So we set Jess up with it and she started using it to organize and play her music. And it was good.

Mostly.

Turned out that the iPod support was flaky at best, but they were supposedly working on it. Then Songbird announced that they were dropping iPod support. A few months went by and then Songbird announced they were dropping Linux support. So now they're just another music player in a sea of existing and mature players with nothing much to distinguish them from the rest.

So lacking iPod support and getting no further updates for the Linux version we needed a new solution for Jess' music needs. However, she'd already gone through all the hassle of getting her music library setup just right in Songbird with all her ratings and playcounts and playlists. Since she has several thousand songs she didn't want to try to redo all that by hand.

Being the master of computer science that I am I decided to take it upon myself to figure out how to automagically migrate her music out of Songbird and in to Rhythmbox. (When combined with Gtkpod these 2 make an acceptable and working solution for music management and iPod syncing.)

Unfortunately, for unknown reasons neither system used any kind of normal standard for storing their data. Well, that's not entirely true. Most of the data is just fine. But file location data was a mess. Songbird stored it in some weird convoluted ascii encoding of unicode. Rhythmbox stores it in some randomly/partially url-encoded form of unicode. But lacking any clear documentation on the matter I had to reverse-engineer both formats. It was a pain.

But I did it! And now for your benefit I present my automagic music migrator to get you successfully and as painlessly as possible from Songbird to Rhythmbox.

I'm hosting this project on Github: http://github.com/kdickerson/Export-from-Songbird

Download the file songbird_to_rhythmbox.py file and then run it:

#python songbird_to_rhythmbox.py

or make it executable and run it:

#chmod +x songbird_to_rhythmbox.py
#./songbird_to_rhythmbox.py

It will ask you 2 questions. 1. What user you'd like to migrate (the default is the current user, so most people will just hit enter). and 2. If you would like the script to overwrite your existing rhythmbox library or just output the migration to a new file for you.

I wrote and used the script with Python 2.6.5

What this script WON'T do:
It won't migrate playlists. Jess already had an export function for playlists in Songbird and exporting the playlists as M3Us and then importing to the migrated Rhythmbox worked with only a very small percentage of errors for files with special characters.

I thought about spending the time to automate migrating playlists, but I don't feel like it. Rhythmbox stores playlists in another xml file and Songbird stores them in the same database as the other data, so you can use my script as a guide to write your own playlist migrator if you'd like.

Nostalgia for the Future

February 8, 2010 11:07 am

This morning the Endeavour orbiter launched from pad 39A at the Kennedy Space Center carrying people and equipment into space. Only four more shuttle launches will occur and the fleet of orbiters will be retired in mid-September. When that happens, America will lose the ability to put a man in space.

Many will argue that putting people in space or on the moon is/was pointless and a waste of money. Those with this attitude will kindly turn their attention to the list of NASA spin-off technologies. This list includes (among many others) LEDs, in-ear thermometers, and improved water purification systems. There are also countless other improvements in technologies driven by NASA's need for more efficient and more reliable equipment.

President Obama's budget plan for 2011 removes all funding for the Constellation Program, which would have led to the development of a new vehicle for transporting humans in to space. If the cancellation of funds occurs (which is pretty close to a sure thing) then the United States will not be able to put people in space for the foreseeable future. This saddens me. We've given up on exploring space in any serious manner. We haven't been to the moon since 1972. All the astronauts still living that walked on the moon are in their 70s. Another ~20 years and there will be no person alive who walked on the moon. Isn't that kind of sad?

I wasn't alive when any of the astronauts walked on the moon; but I am still filled with wonder at the idea of stepping foot on a different celestial body. While Star Trek is fiction, I think its tag-line embodies the sense of awe several generations have held about being able to put people into space: "...to explore strange new worlds; to seek out new life and new civilizations; to boldly go where no man has gone before". It expresses the desire to seek out knowledge for the sake of learning.

The mission wasn't "to find and commercialize new civilizations and technologies, to open new markets for our products, to boldy make a buck where no man has made a buck before." Unfortunately, that seems to be mantra of our society. NASA operates on a shoestring budget, the National Labs have (essentially) been privatized to focus on making money (goodbye long-term original research projects to discover more about the universe), public education is about rubber stamping every kid's high-school diploma, and being educated is nerdy and undesirable.

I miss the future we were promised as kids. A world where space exploration was common. A world where man's search for knowledge and understanding overcame petty differences of culture and societal status.

------

I've always wanted to watch a shuttle launch and time is running out. Jess and I are hoping to take a trip to see the final flight of the Atlantis orbiter in May.

On September 24, 2010, when the Discovery lands, marking the end of U.S. manned spaceflight, I hope everyone can pause for a moment to reflect on what we've lost.

Grails: Dynamically building criteria queries

October 26, 2009 10:53 am

** Update: Detached Criteria are probably the better way of achieving this type of DRY principle now.  They became available in Grails 2.0; I wrote this post while using Grails 1.x. **

** Update 2: DetachedCriteria have some limitations. I wrote a cleaned up and expanded version of what this post discusses: Criteria Aggregator: Dynamic Criteria Queries in Grails **

Ok, I've been exploring using Groovy and Grails at work to make our team more efficient. Java is great and all, but it's a terrible language for web development. For some quick info: A group of people decided Java could be a good web development language if it weren't so Java-y. So they wrote Groovy. It's fully compatible with Java, but is actually a dynamically-typed scripting language built on top of Java. Great, now we have a good web language with all the support and power of Java. Groovy compiles to the same bytecode as Java, because Java is Groovy is Java.

Well, then Ruby took the web development world by storm with the Rails architecture. Finally web developers could focus on the actual interesting part of the job rather than continually writing frameworks to run their sites. It was such a good idea that it was very quickly mirrored into other languages. Python got Django (my personal favorite and the future of the 100 Hour Board) and Groovy got Grails.

Anyway, if you've found this post by Googling you probably know all that so let's get to the meat of the matter.

In my effort to learn Groovy and Grails I wanted to be able to build up Criteria Queries dynamically. Django lets you do this incredibly easily by passing around queryset objects that have lazy loading. Grails interfaces with Hibernate using some convenient and easy closure work with access to the Hibernate Criteria Builder. So you can easily and quickly define your query but only within that closure that gets passed to the builder. I wanted to be able to piece my query together from other pieces in order to adhere to the Don't Repeat Yourself (DRY) principle.

In my searching I came upon a blog post by Geoff Lane at zorched.net entitled "DRYing Grails Criteria Queries". His method involves extending the metaclass of the HibernateCriteriaBuilder object using Groovy's expando abilities. This is a decent solution, but not incredibly flexible. I needed something better.

For 3 days I relentlessly scoured the Internet searching for answers, I coded all sorts of tests, I digested the source code of the Grails HibernateCriteriaBuilder class, I poked, I prodded, I made wild accusations. All in vain.

I decided that one of the following conclusions must be true:
1. It was so obvious that I just couldn't see it.
2. I was too stupid to come up with a good solution.
3. I was asking the wrong question.
4. I needed more experience with Groovy and working with closures.

Now, I'm not entirely convinced that #3 isn't true, but #4 also happened to be true. I found my solution this morning when I actually paid attention to what the error message was really telling me rather than just "it didn't work".

So, without further ado, here is how you can achieve DRYness with Grails when dynamically building criteria queries, without dropping all the way back into Java.

For the sake of this example let's assume we have an Account class with the properties "accountType", "accountAction", and "owner".

Our automatically generated code to list objects would look something like this (in our AccountController.groovy file):

def list = {
  params.max = Math.min( params.max ? params.max.toInteger() : 20,  100)
  params.offset = params.offset ? params.offset.toInteger() : 0

  def results = Account.createCriteria().list(max:params.max, offset:params.offset) {
    order(params.sort ? params.sort : "id", params.order ? params.order : "asc")
  }
  [ accountInstanceList: results, accountInstanceTotal: results.totalCount ]
}

That's all well and good, but let's add the ability to sort based on the name of the account owner, and display only accounts belonging to a specific user:

def list = {
  params.max = Math.min( params.max ? params.max.toInteger() : 20,  100)
  params.offset = params.offset ? params.offset.toInteger() : 0
  def req_owner = User.get(params.id)

  def results = Account.createCriteria().list(max:params.max, offset:params.offset) {
    if (params.sort == 'userName') {
      owner {
        if (req_owner) {eq("id", req_owner.id)}
     order("lastName", params.order ? params.order : "asc")
     order("firstName", params.order ? params.order : "asc")
      }
    } else {
      if (req_owner) {owner {eq("id", req_owner.id)}}
      order(params.sort ? params.sort : "id", params.order ? params.order : "asc")
    }
  }
  [ accountInstanceList: results, accountInstanceTotal: results.totalCount ]
}

Still fine, but what if we want to build off of this for different controller methods? If I want a page that does the same thing, but limits the display to only accounts of a certain account type I can use the method described in the link above by Geoff Lane. But suppose I want to further provide another page that limits it further or several pages that provide just slightly different listings based on criteria changes. Perhaps there is a great easy way to do this in Grails, and if so, please comment and tell me, because I couldn't find it.

Here's how I do it. First let's pull the code we have to a reusable location. We make a self-standing closure and then curry the "params" variable to it:

def account_list = {params ->
  def req_owner = User.get(params.id)
  if (params.sort == 'userName') {
    owner {
      if (req_owner) {eq("id", req_owner.id)}
      order("lastName", params.order ? params.order : "asc")
      order("firstName", params.order ? params.order : "asc")
    }
  } else {
    if (req_owner) {owner {eq("id", req_owner.id)}}
    order(params.sort ? params.sort : "id", params.order ? params.order : "asc")
  }
}

def list = {
  params.max = Math.min( params.max ? params.max.toInteger() : 20,  100)
  params.offset = params.offset ? params.offset.toInteger() : 0

  def results = Account.createCriteria().list(max:params.max, offset:params.offset, account_list.curry(params))
  [ accountInstanceList: results, accountInstanceTotal: results.totalCount ]
}

The trouble is when we try to combine self-standing closures.

Everytime I tried something that seemed like it should work I'd get errors about "AccountController.isNotNull not applicable for arguments..." or whatever it said. I didn't think hard enough about what it was really saying to me. The additional closure pieces were getting the wrong scope for whatever reason and instead of calling methods on the CriteriaBuilder object they were calling them on the AccountController object.

Here's the solution:

def crit_a = {crit ->
  crit.accountAction { 
    ne("name", "None")
  }
}

def crit_b = {crit ->
  crit.accountAction {
    ne("name", "Renew")
  }
} 

def account_list = {crit, params ->
  def req_owner = User.get(params.id)
  if (params.sort == 'userName') {
    crit.owner {
      if (req_owner) {eq("id", req_owner.id)}
      order("lastName", params.order ? params.order : "asc")
      order("firstName", params.order ? params.order : "asc")
    }
  } else {
    if (req_owner) {crit.owner {eq("id", req_owner.id)}}
    crit.order(params.sort ? params.sort : "id", params.order ? params.order : "asc")
  }
}

def list = {
  params.max = Math.min( params.max ? params.max.toInteger() : 20,  100)
  params.offset = params.offset ? params.offset.toInteger() : 0

  def critBuilder = Account.createCriteria()
  def crit_closure = {
    account_list.curry(critBuilder, params)()
    crit_a.curry(critBuilder)()
    crit_b.curry(critBuilder)()
  }

  def results = critBuilder.list(max:params.max, offset:params.offset, crit_closure)
  [ accountInstanceList: results, accountInstanceTotal: results.totalCount ]
}

You get a reference to the CriteriaBuilder object using Account.createCriteria(). Note that you get a CriteriaBuilder object not a Criteria object like much of the documentation suggests. You curry this to the closures you want to use, and call them inside a new closure which you pass to the CriteriaBuilder object's "list" method. And it works! So now you can dynamically build up your criteria queries!

America's Broken Laws

August 12, 2009 10:26 am

Do you remember Real Player? Huge in the late 1990s. They provided software for streaming audio/video over the Internet before Macromedia blew them out of the water with Flash. They've been quiet for a while, but recently they re-emerged with their latest product: RealDVD. It is software that allows you to make a backup of a DVD.

They knew that this was going to get them sued by the MPAA, and they were looking for the fight. As much as I (and millions of others) hated RealPlayer back in the 1990's I (and millions of others) have been on their side in this battle.

So what is the battle? DVDs are "protected" with the Content Scramble System (CSS). In 1999 DVD Jon gained his name by cracking this system; allowing anyone to access the data stored on DVDs without paying the fee required to get the code to decrypt the content. Yes, that's right. Any DVD player you've ever used; whether in your laptop, desktop, or connected to the tv; was only allowed to be produced after the manufacturer ponied up the cash for the license to legally decrypt the CSS and agreed to the demands that they wouldn't produce a product which allows the consumer to make a copy of the DVD.

So when DVD Jon reverse-engineered CSS, and let the cat out of the bag, the MPAA was not happy. Luckily for the MPAA they had already managed to pass the Digital Millennium Copyright Act (DMCA). Section 1201 (2) Says:

`(2) No person shall manufacture, import, offer to the public, provide, or otherwise traffic in any technology, product, service, device, component, or part thereof, that--
`(A) is primarily designed or produced for the purpose of circumventing a technological measure that effectively controls access to a work protected under this title;
`(B) has only limited commercially significant purpose or use other than to circumvent a technological measure that effectively controls access to a work protected under this title; or
`(C) is marketed by that person or another acting in concert with that person with that person's knowledge for use in circumventing a technological measure that effectively controls access to a work protected under this title.

In short: If there is any kind of copy-protection measure on a piece of media then it is illegal to access the content without paying the proper people for access (and since the people authorizing access won't authorize any use allowing duplication we have our problem).

DVD Jon's work is illegal under the DMCA. Fortunately for DVD Jon, he lives in Norway and the MPAA can't really do much about it. The bigger issue for the MPAA is that the code necessary to break the encryption on a DVD is so simple it can be written on a t-shirt. Not exactly a big hurdle for people that want to use it.

So why is this an argument at all in the first place? Something is illegal, so you shouldn't do it, right? Well, the problem is this little notion of Fair Use (also see the Electronic Frontier Foundation's FAQ). The Fair Use doctrine says you can make a personal back-up copy of content you own.

Fair Use says you can make a backup copy of any content you own, the DMCA says it's illegal to make that copy if the content has any type of copy-protection system in place. Take a guess which side wins in these arguments. I'll give you a hint, it's not us, the individual citizens of the country. If we want to make a back-up of The Fox and the Hound so that when the DVD gets all scratched and destroyed we don't have to buy it again, we can't.

This is the current state of the law in the United States. Absolutely ridiculous and inconsistent. The RealDVD case was decided today by U.S. District Judge Marilyn Hall Patel in San Francisco (ruled against Real). In her remarks she made this absurdity very clear:

So while it may well be fair use for an individual consumer to store a backup copy of a personally owned DVD on that individual’s computer, a federal law has nonetheless made it illegal to manufacture or traffic in a device or tool that permits a consumer to make such copies,

(For a complete article on the matter see Wired's Judge Rules DVD-Copying Software Is Illegal)