Criteria Aggregator: Dynamic Criteria Queries in Grails

February 5, 2014 2:42 pm

A few years ago I wrote about dynamically building criteria queries in Grails.  Well the use case we had disappeared and I didn't do much of anything with dynamic criteria queries until recently.  As I've mentioned in an update on that post, you probably just want to use DetachedCriteria now instead of taking this route, but DetachedCriteria don't support the same range of functionality that normal criteria queries do.

I ran into this limitation when I needed to use subqueries which DetachedCriteria do not support but standard criteria queries do.  After spending hours trying to work around the limitation and still use DetachedCriteria I decided to give up and create a nice way of using the dynamic criteria queries based on my past experience, the comments on that post, and my greatly enlarged knowledge of Grails.

So I created a nice clean class called "CriteriaAggregator":

package org.example.package
import grails.orm.HibernateCriteriaBuilder

// Aggregate query criteria for a Domain Class
// Example: def qa = new CriteriaAggregator(MyDomainClass)
//    qa.addCriteria { idEq(12345L) }
//     def results = qa.get()
public class CriteriaAggregator {
    private Class forClass
    private List<Closure> criteriaClosures

    // forClass should be a Grails DomainClass; but since Grails injects rather than inherits I can't specify the type better than "Class"
    public CriteriaAggregator(Class forClass) {this.forClass = forClass; criteriaClosures = new ArrayList<Closure>(10)}

    // criteriaClosure is the exact same type of closure you'd pass to DomainClass.withCriteria(criteriaClosure)
    public void addCriteria(Closure criteriaClosure) {criteriaClosures << criteriaClosure}

    public long count() {return runQuery('get') {projections {rowCount()}}}
    public def get(Closure additionalCriteria=null) {return runQuery('get', additionalCriteria)} // Query must return only a single row
    public def list(Closure additionalCriteria=null) {return runQuery('list', additionalCriteria)}

    private def runQuery(String method, Closure additionalCriteria=null) {
        HibernateCriteriaBuilder criteriaBuilder = forClass.createCriteria()
        def critClosures = criteriaClosures // Bizarre that criteriaClosures won't evaluate properly inside the "$method" closure, but it won't so this works around that issue
        criteriaBuilder."$method" {
            critClosures.each{closure -> closure.delegate = criteriaBuilder; closure()}
            if (additionalCriteria) {additionalCriteria.delegate = criteriaBuilder; additionalCriteria()}
        }
    }
}

This wraps and defers the creation of the normal criteria builder allowing you to build up the criteria dynamically and execute it when desired.

Suppose you had a Customer domain class with many Orders.  Orders have a date, paymentMethod, and totalPrice.

def customerQueryAggregator = new CriteriaAggregator(Customer)
customerQueryAggregator.addCriteria {
  orders {
    def now = new Date()
    between('date', now-7, now)
  }
}
customerQueryAggregator.addCriteria {
  orders {
    eq('paymentMethod', 'cash')
  }
}

def numCustomersInPastWeekPayingCash = customerQueryAggregator.count()
println numCustomersInPastWeekPayingCash

def highRollersInPastWeek = customerQueryAggregator.list{
  orders {
    gt('totalPrice', 50000)
  }
}
println highRollersInPastWeek

This is obviously a trivial example where you don't need to use the aggregator.  But it's just to illustrate the usage.  I'm using it to build up a query based on a set of options provided by a caller as part of a reusable and flexible service.

Using this I can handle subqueries like so:

import grails.gorm.DetachedCriteria as GrailsDetachedCriteria
import org.hibernate.criterion.Subqueries
import grails.orm.HibernateCriteriaBuilder

def customerQueryAggregator = new CriteriaAggregator(Customer)

GrailsDetachedCriteria largeOrderSubquery = new GrailsDetachedCriteria(Order).build {
  eqProperty 'order.id', 'this.id'
  gt 'totalPrice', 50000
  projections {property 'id'}
}

customerQueryAggregator.addCriteria {
  add(Subqueries.exists(HibernateCriteriaBuilder.getHibernateDetachedCriteria(largeOrderSubquery)))
}

Update April 4, 2014:

I ended up having some trouble with the proper definition of table aliases (it always wanted to call "order" "this" even though we already had a "this") while handling the GrailsDetachedCriteria subqueries.  To work around it I ended up dropping back to straight Hibernate, but it still works with the CriteriaAggregator.

import org.hibernate.criterion.DetachedCriteria as HibernateDetachedCriteria
import org.hibernate.criterion.Restrictions
import org.hibernate.criterion.Disjunction
import org.hibernate.criterion.Subqueries
import org.hibernate.criterion.Projections

def customerQueryAggregator = new CriteriaAggregator(Customer)

HibnerateDetachedCriteria largeOrderSubquery = new HibernateDetachedCriteria.forClass(Order.class, 'order')
largeOrderSubquery.add(Restrictions.eqProperty('order.id', 'this.id'))
largeOrderSubquery.add(Restrictions.gt('order.totalPrice', 50000))
largeOrderSubquery.setProjection(Projections.property('order.id'))

customerQueryAggregator.addCriteria {
  add(Subqueries.exists(largeOrderSubquery))
}

Self-Hosted Blogging Software Options

January 27, 2014 2:56 am

I spent some time this weekend exploring self-hosted blogging software alternatives to Blogger.  I set up a virtual machine and took a look at Ghost, Habari, Chyrp, and WordPress.  I also looked in to, though didn't install or play with, AnchorCMS, Couch CMS, Wolf CMS, Bolt, Monstra, and Pyro CMS.

I was interested in some of the lesser known "next-gen" platforms that seek to be lighter weight than WordPress.  They also have some new approaches to how blogging software should function which I found intriguing.

I didn't get very far with Ghost.  Ghost is built on node.js; which I hadn't used before and it took me an inordinate amount of time to get the thing running just to play with.  And once I finally did get it running, I quickly discovered it does not have multi-user support yet (slated for summer 2014).  Since a Blogger replacement would need to allow both Jess and I to write posts this was a deal breaker.

I tried Habari next and it was less painful to get installed.  Unfortunately, it also is missing some needed functionality.  And this part is about to become a very common theme: video uploading and playing.  I was able to upload a video to Habari's media "silo" (as they call it) quite easily, but I had to write the video player HTML myself.  For just me that's not a big deal, but that's not going to work for Jess.  I probably could try to write a plugin for Habari to do the video player code, but I don't like writing PHP and don't feel like it.  So I kept looking.

Chyrp is similar to Habari in many ways.  I like the simple, clean aesthetics (shared by Habari and Ghost).  But again, video uploading and playing was not a working-out-of-the-box experience and there didn't seem to be any plugins providing the functionality.

The thinking of all of these lightweight blogging platforms is that no one wants to host their own videos so rather than support video uploading and players they provide rich support for easily embedding YouTube videos (and other services).  Which, okay, I understand the bandwidth issues; but it also somewhat defeats the purpose of self-hosting the blog if you're still going to rely on a 3rd party.

For me the point of self-hosting is that it is entirely under my control on my systems doing what I say.  And since I doubt any of our little videos of Heather will ever draw any meaningful crowd I'm not particularly concerned with the bandwidth usage.

Hopefully with a few more development cycles Ghost, Habari, and Chyrp will all reach a feature level that would enable me to use them if I wanted to.

The other ones I listed but didn't install all looked like they'd end up in the same boat with lack of video uploading and playing readily available.  Some had demo installations I played with that made it clear they were less focused on blogging than I was (CMSs, or Content Management Systems, can really cover a lot of territory).

So I finally broke down and installed WordPress.  I had been avoiding WordPress mainly because WordPress has evolved much more into a general purpose CMS than I need.  But it does support video uploading and embedding out-of-the-box and does a decent job of media management.  However, it also comes with a bunch of stuff I don't need and I will need to spend some time cleaning out the stuff I don't want and configuring the stuff I do want.

With that decision made I'll need to install WordPress to my actual server instead of my sandbox environment.  Then I'll need to do the configuration and customization.  After that I should be able to import all of our existing blog posts and then switch over this blog to use that new system.  But I don't currently have a timeline for that.

Reducing Tracking

December 13, 2013 6:52 pm

As a software developer and particularly a web developer it is my responsibility to set a good example and try to be a force for good.

As of today, none of my sites use Google Analytics anymore.  The tipping point, aside from improving general web privacy, was that the NSA is supposedly hijacking the Google tracking data for their own purposes.

While I don't really need any analytics, I do get curious as to what kind of traffic my websites are getting.  So I instead stood up my own instance of Piwik.  This is a self-hosted analytics solution.  Now my websites simply report to another one of my systems when they're accessed (instead of Google).  And I know that my Piwik installation respects the "Do Not Track" setting you can use in your browser (I've tested it myself).  Also, my Piwik analytics won't track you all over the Internet.

I realize this may seem a little silly coming from a blog hosted on Google's servers via Blogger, but I'm also working on that.  I'm looking into using Habari and migrating this blog to be self-hosted as well.  Given that Blogger seems to be a dead product in Google's eyes (little to no updates or changes in many, many months), it's probably better to get off it anyway as it may get shut down at some point.  However, I'll probably wait a few months until the more stabilized Habari 1.0 release is finished.  Such a move will probably coincide with standing up a family cloud using ArkOS and OwnCloud.

It's not that I'm particularly paranoid, but the various pieces of software are reaching a point where for someone like me, it's not particularly difficult or burdensome to self-host things.  So I might as well do it and encourage an Internet model closer to its original design: interacting, decentralized systems.  A model which is harder for any one organization (government or otherwise) to infiltrate, break, or commandeer.

Testing LED Christmas Lights

December 8, 2013 3:49 pm

The icicle lights we have are LEDs which is nice because they use incredibly little power and should last a long time, but has the drawback that they're slightly harder to diagnose when they misbehave.

Once I was recovered from my various maladies I pulled out the 10 strands we have to get them up on the house (starting last weekend).  One of the strands wouldn't light so I got to play the fun game of tracking down the problem.

So I started with the obvious of checking to make sure no "bulbs" (I'll say "bulb" because it conveys my meaning even though there are no actual bulbs) were missing.  And I tested the other end of the strand and was getting a solid 120V connection, so I knew the wiring was still good.

[Edit: Individual bulbs can fail like normal and have a shunt which will allow the rest of the strand to stay lit.  These bulbs are easy to find and replace (they're the only bulb unlit).  The problem I explore in this post is a bulb where the leads are broken so the power didn't make it to the shunt in the first place.]

But once you have to go beyond those steps some of the common troubleshooting techniques won't work on LED strands.  I started with my multimeter that contains an inductive AC voltage detector ("no-touch").  Since the bad bulb breaks the circuit, traditionally you can use a voltage detector to find where the circuit breaks.

Unfortunately, for whatever reason, my multimeter was detecting voltage everywhere along the strand and along each "icicle."  So that did me no good, I'm not entirely sure why.  So then you can take the obvious approach of just switching out bulbs with replacements, but the trouble is, you might have more than one bad bulb.  In which case, you won't know if you're replacing a good bulb or a bad bulb.

Instead you need to be able to test each bulb individually to know whether it is good or not.  The usual approach is a simple continuity tester (check whether a circuit exists from one lead on the bulb to the other).  But, a continuity test won't work because there is no simple circuit to test.  Unless you actually cause the LED to activate (with enough voltage) you won't have a circuit.  A continuity test only uses a very low voltage and so it will report an open circuit (bad bulb).  So, despite having a multimeter and knowing how to use it, it did me no good in finding the bad bulb.

Instead you need an LED tester.  You can make a simple LED tester using the following:  a 9-volt battery, a resistor, and a rubber band.

IMGP1450a
Without the resistor the only thing you'll learn is that the LED used to work because you'll blow out the LED from too high of voltage (which I did twice before finally digging one out).  The resistor I used happened to be 10,000 ohms, but 1,000 ohms should be fine too.

Just use the rubber band to hold the resistor to the negative terminal on the battery (the negative terminal is easier to wrap the rubber band around, otherwise it really doesn't matter).  Then bend the end of the resistor around next to the positive terminal of the battery so you can easily touch the leads of a bulb to the resistor and the positive terminal.

IMGP1452aIf you touch the LED leads backwards nothing will happen (LEDs are one way only).  It won't damage the LED, but it won't light up either.  Because of this your bulb should be keyed to only fit in its socket one way.  Pay attention to that keying so that you can reliably test the correct orientation and save some time.

IMGP1453aSo with my makeshift tester I started popping out bulbs and testing them (the edge of a dinner knife worked great for popping the bulbs out of their base).  I was actually getting pretty quick at it and it was the 21st bulb I pulled that was bad.  Popped in a replacement and everything was working.

Okay, not a particularly fascinating story, but when I was searching the Internet for how to track down a bad LED in a strand of Christmas lights I wasn't finding much useful information (just lots of information about incandescent lights sometimes with a disclaimer that it wouldn't work for LED strands).

(And in case anyone cares, these are 70-LED icicle strands in "warm white" made by Vickerman.)