nerdy – Page 8 – If we had a Marriage, These Would be its Thoughts

Improving OwnCloud Throughput

April 1, 2016 10:22 pm

I have an instance of OwnCloud running from a machine at home that provides file-syncing services for family members. The OwnCloud data is then encrypted and sent on to CrashPlan for backups.

I recently pointed 1.3 TB of data to sync into OwnCloud. These are old home videos in raw format with files sized up to 25 GB. The upload speed was atrocious. The server is connected to my desktop on a gigabit switch and transfer speeds were topping out at 2.0 MB/s.

Most of the issues people have with poor OwnCloud performance are when uploading many small files which is not my scenario. But I followed whatever advice I could find. I modified MariaDB settings and used MySqlTuner to find potential performance gains, which helped a little. I finally found the backported php-apc package I needed for Ubuntu 14.04 to provide php caching, which helped a little. But I was still only up to ~4.5 MB/s.

Then I considered my larger system. The server is on my local gigabit switch, but my desktop is configured using its public domain name, which resolves to my public IP address. This mean every request from my desktop wasn’t just going through the gigabit switch and in to the server. Instead every request was going through the switch to the router, being NAT-translated, back to the switch, and then to the server. Due to an issue with my high-performance EdgeRouter Lite, I’ve been using my old WRT54GL as my router. And that old thing simply can’t handle the load. It’s CPU was maxed out and network throughput was abysmal.

Since I wanted to bypass the router and go directly from desktop, to switch, to server I made an entry in my /etc/hosts file to tell my machine to use the server’s internal IP address instead of the public IP address associated with the domain name. The CPU load on the router is now gone and the OwnCloud throughput increased to ~11 MB/s. Still pretty awful compared to the ~60 MB/s I get using scp, but substantially better than 2 MB/s.

That speed increase was going strong for a while, but after about 20 minutes it slowed back down to ~4.5 MB/s again. The router, however, is no longer in the loop, so at least I’ve removed one potential bottleneck.

I have no idea what the bottleneck is now except for OwnCloud just being abysmally slow. The server is using a fair bit of CPU, but it’s not quite maxed out (usually showing 20% idle overall on a 4-core machine). IO doesn’t seem to be the bottleneck, iotop doesn’t show anything being held up. There’s 2 GB of free RAM available, so that doesn’t seem to be the issue.

I’m running OwnCloud using Apache with mod-php; but I didn’t see anything suggesting that running PHP using fcgi or fastcgi would be better. Using nginx instead of Apache might be better, but I have no experience configuring nginx so it wouldn’t be a short little project to try it.

If anyone has any suggestions on how to get OwnCloud to perform better (particularly when syncing very large files) when connecting over a gigabit local network I’d love to hear them.

Update 4/17/2016

The slowdown from 10MB/s seems to have been the accumulation process running over large files. Files are transferred in small chunks. Once all the chunks have been uploaded, the chunks are accumulated and the original file is reconstituted. During this time the upload speed drops dramatically.

I had a terminal case where a known memory leak in the accumulation process kept causing the reconstitution to fail. Since the chunks are deleted as they’re used during accumulation, the client would then re-upload all the deleted chunks, then the accumulation process would run, fail, and we’d go round and round. This made it look like performance was worse than it truly was (though not uploading files is sort of a bad thing for a file sync tool to do).

In and attempt to get my files finished without waiting until the memory leak fix is released I split some files to be smaller so the accumulation process doesn’t leak as much memory. I also set a rather absurd 4GB memory limit on the PHP process hoping that will be enough to get it through the large files without failing.

Setting aside the accumulation/reconstitution process, I’m getting a consistent 10MB/s transfer on a 100Mbit switch. I had to RMA the gigabit switch because it began misbehaving. I’m hopeful that when the replacement arrives my throughput will increase beyond 10MB/s since that’s about the limit of the 100Mbit switch and the gigabit switch wasn’t working properly.

Update 4/19/2016

My replacement gigabit switch is in and the transfer rate has gone up to ~19MB/s at times (when no accumulation/reconstitution work is occurring). There is still plenty of room for improvement, but at least I’m not stuck at 2MB/s anymore.

Converting Http Session Events into Grails 3 Events

October 19, 2015 1:16 pm

Grails 3 introduced a new Events API based on Reactor. Unfortunately, as far as I can tell, HttpSessionEvents are not natively part of the Grails 3 Events system. Bringing them in to the fold, however, is pretty easy. I based this off of Oliver Wahlen’s immensely helpful blog post about sending the HttpSessionEvents to a Grails service.

First, let’s create our Spring HttpSessionServletListener. Create this file somewhere in the /src/ path where Grails will find it:

File: .../grailsProject/src/main/groovy/com/example/HttpSessionServletListener.groovy

package com.example

import grails.events.*
import javax.servlet.http.HttpSession
import javax.servlet.http.HttpSessionEvent
import javax.servlet.http.HttpSessionListener

class HttpSessionServletListener implements HttpSessionListener, Events {
  
    // called by servlet container upon session creation
    void sessionCreated(HttpSessionEvent event) {
        notify("example:httpSessionCreated", event.session)
    }

    // called by servlet container upon session destruction
    void sessionDestroyed(HttpSessionEvent event) {
        notify("example:httpSessionDestroyed", event.session)
    }
}

Now register the HttpSessionServletListener as a Spring Bean. If you don’t already have a resources.groovy file, create one and add the following.

.../grailsProject/grails-app/conf/spring/resources.groovy

import org.springframework.boot.context.embedded.ServletListenerRegistrationBean
import com.example.HttpSessionServletListener

beans = {
    
    httpSessionServletListener(ServletListenerRegistrationBean) {
        listener = bean(HttpSessionServletListener)
    }
    
}
// Yes this is the entire file

Now you are all set to listen for the “example:httpSessionCreated” and “example:httpSessionDestroyed” events using the Grails 3 Events API. “Example” is the namespace of the event, which in my real code I set to the last part of the package name, so I made it match the package name of “example”. Just use something so you don’t have to worry about naming collisions.

Here’s an example of listening for the events in a standard Grails Controller. Note that the event handlers are attached after construction, and before the Controller bean is made available, by using the PostConstruct annotation.

.../grailsProject/grails-app/controllers/com/example/ExampleController.groovy

package com.example

import grails.events.*
import javax.annotation.PostConstruct

class ExampleController {
    
    @PostConstruct
    void init() {
        
        on("example:httpSessionCreated") { session ->
            println "sessionCreated: ${session.id}"
        }
        
        on("example:httpSessionDestroyed") { session ->
            println "sessionDestroyed: ${session.id}"
        }
    }
}

NASA Apollo Pictures

October 5, 2015 5:40 pm

Last week NASA released a bunch (over 10,000) of original images from the Apollo missions on their Flickr account. They’re all Public Domain images so anyone can download the originals and use them for anything they like. I flipped through and picked out my favorites and cleaned them up. I’ll probably get some nice canvas prints made of some of them when Canvas Press has sales.

Here are my top 10 after cleaning them up. I’ve uploaded my full versions so you can download them yourself if you want to make a poster or canvas print or something. Clicking an image will open the full-size version, which you can then save to your computer using right-click -> Save image…

A Light in the Dark

September 28, 2015 7:34 pm

I wrapped an LED around a button battery with a little material to keep it separated so you push on it to turn the LED on. Then we took turns waving it around while camera took a picture with a long exposure.

Heather’s work was usually very…nuclear:

We encouraged her to move around more:

I tried to write my name, but my spatial awareness is apparently not great:

Jess did better with her contribution:

Heather’s World

September 5, 2015 4:22 pm

Heather has been using her Nabi Jr. tablet for awhile now. It has a camera on it that can spin from front-facing to back-facing. I configured it to automatically sync the pictures to the computer.

With the pictures all in place I created a time-lapse of Heather’s past year. If you don’t care about the nerdy details of how I made it, jump to the bottom to watch the video.

It started with the 925 pictures she took over the last year. I ran them through ImageMagick to pad them to a consistent size:

$ cd /home/heather/Pictures
$ mkdir ~/Desktop/tmp
$ find "./2014/12 December 2014" "./2015" -name "*.jpg" -exec mogrify -gravity center -background black -extent 1600x1600 -path ~/Desktop/tmp {} \;

Then I, again, used ImageMagick to interpolate 3 transition frames between each picture so it’s less jumpy (still pretty jumpy though):

$ cd ~/Desktop/tmp
$ mkdir morphs
$ convert *.jpg -morph 3 ./morphs/out-%04d.jpg

Okay, that’s a little bit of a lie. ImageMagick’s convert tool currently (version 6.x) reads in all input before doing anything. When it tries to load in the 925 images it uses up all the memory in my computer and then dies. So I had to break it up into batches (IM 7 supposedly fixes this problem). I used batches of 100 which still used about 7 GB of RAM:

$ ls *.jpg | head -n 100 | tail -n 100 | convert @- -morph 3 morphs/a-%04d.jpg
$ ls *.jpg | head -n 200 | tail -n 101 | convert @- -morph 3 morphs/b-%04d.jpg
$ ls *.jpg | head -n 300 | tail -n 101 | convert @- -morph 3 morphs/c-%04d.jpg
# .... all the way through head -n 900 .....
$ ls *.jpg | tail -n 26 | convert @- -morph 3 morphs/j-%04d.jpg

Doing it in batches results in a duplicate frame between each batch, so at this point we have to delete b-0000.jpg, c-0000.jpg, d-0000.jpg, etc. (but keep a-0000.jpg).

Now we’re ready to run them through mencoder to produce our video:

$ cd morphs
$ mencoder mf://\*.jpg -mf w=1600:h=1600:fps=16:type=jpg -ovc lavc -lavcopts vcodec=mpeg4 -of lavf -lavfopts format=mp4 -oac copy -o ~/Desktop/output.mp4

I tried doing this with ffmpeg/avconv, but it kept cropping my images and I couldn’t make it work. I also spent hours trying to get all this to work without using intermediate files, but ultimately failed. This process is a little more arduous (I might get around to writing it all into a script at some point) but it does work.

Here’s is the view from Heather’s world from December 2014 through September 2015 (there’s no audio):

Or right-click and “save as…” to download it here: Heather’s View – Dec 2014 – Sep 2015 (small)

If you watch closely you’ll see Christmas, Jess being very pregnant, Corinne appears, Grandma visits, Kyle’s birthday, and a lot of wandering around the house.

It’s not super exciting, but I think it’s kind of an interesting insight into Heather’s world.