Author: Kyle Dickerson

Improving E-books

April 20, 2016 9:23 am

One of the things I really like about eReaders is the built in dictionary.  Unfortunately, they aren’t always comprehensive and, of course, fail when an author is making up new words.

So I came up with an improvement: eBooks should bundle a book-specific dictionary that can be merged with the standard dictionary while reading that book.  This allows books to provide specific definitions of uncommon words and new words.

I’m re-reading The Lord of the Rings right now and Tolkien uses a lot of obscure and archaic words that are real (but which my eReader’s dictionary doesn’t know) and he also has a million made up words in Elvish, Dwarvish, Entish, “common tongue,” etc.  It would be helpful to be able to look those up as well.

So…Amazon, go make it happen.

Improving OwnCloud Throughput

April 1, 2016 10:22 pm

I have an instance of OwnCloud running from a machine at home that provides file-syncing services for family members.  The OwnCloud data is then encrypted and sent on to CrashPlan for backups.

I recently pointed 1.3 TB of data to sync into OwnCloud.  These are old home videos in raw format with files sized up to 25 GB.  The upload speed was atrocious.  The server is connected to my desktop on a gigabit switch and transfer speeds were topping out at 2.0 MB/s.

Most of the issues people have with poor OwnCloud performance are when uploading many small files which is not my scenario.  But I followed whatever advice I could find.  I modified MariaDB settings and used MySqlTuner to find potential performance gains, which helped a little.  I finally found the backported php-apc package I needed for Ubuntu 14.04 to provide php caching, which helped a little.  But I was still only up to ~4.5 MB/s.

Then I considered my larger system.  The server is on my local gigabit switch, but my desktop is configured using its public domain name, which resolves to my public IP address.  This mean every request from my desktop wasn’t just going through the gigabit switch and in to the server.  Instead every request was going through the switch to the router, being NAT-translated, back to the switch, and then to the server.  Due to an issue with my high-performance EdgeRouter Lite, I’ve been using my old WRT54GL as my router.  And that old thing simply can’t handle the load.  It’s CPU was maxed out and network throughput was abysmal.

Since I wanted to bypass the router and go directly from desktop, to switch, to server I made an entry in my /etc/hosts file to tell my machine to use the server’s internal IP address instead of the public IP address associated with the domain name.  The CPU load on the router is now gone and the OwnCloud throughput increased to ~11 MB/s.  Still pretty awful compared to the ~60 MB/s I get using scp, but substantially better than 2 MB/s.

That speed increase was going strong for a while, but after about 20 minutes it slowed back down to ~4.5 MB/s again.  The router, however, is no longer in the loop, so at least I’ve removed one potential bottleneck.

I have no idea what the bottleneck is now except for OwnCloud just being abysmally slow.  The server is using a fair bit of CPU, but it’s not quite maxed out (usually showing 20% idle overall on a 4-core machine).  IO doesn’t seem to be the bottleneck, iotop doesn’t show anything being held up.  There’s 2 GB of free RAM available, so that doesn’t seem to be the issue.

I’m running OwnCloud using Apache with mod-php; but I didn’t see anything suggesting that running PHP using fcgi or fastcgi would be better.  Using nginx instead of Apache might be better, but I have no experience configuring nginx so it wouldn’t be a short little project to try it.

If anyone has any suggestions on how to get OwnCloud to perform better (particularly when syncing very large files) when connecting over a gigabit local network I’d love to hear them.

Update 4/17/2016

The slowdown from 10MB/s seems to have been the accumulation process running over large files.  Files are transferred in small chunks.  Once all the chunks have been uploaded, the chunks are accumulated and the original file is reconstituted.  During this time the upload speed drops dramatically.

I had a terminal case where a known memory leak in the accumulation process kept causing the reconstitution to fail.  Since the chunks are deleted as they’re used during accumulation, the client would then re-upload all the deleted chunks, then the accumulation process would run, fail, and we’d go round and round.  This made it look like performance was worse than it truly was (though not uploading files is sort of a bad thing for a file sync tool to do).

In and attempt to get my files finished without waiting until the memory leak fix is released I split some files to be smaller so the accumulation process doesn’t leak as much memory.  I also set a rather absurd 4GB memory limit on the PHP process hoping that will be enough to get it through the large files without failing.

Setting aside the accumulation/reconstitution process, I’m getting a consistent 10MB/s transfer on a 100Mbit switch.  I had to RMA the gigabit switch because it began misbehaving.  I’m hopeful that when the replacement arrives my throughput will increase beyond 10MB/s since that’s about the limit of the 100Mbit switch and the gigabit switch wasn’t working properly.

Update 4/19/2016

My replacement gigabit switch is in and the transfer rate has gone up to ~19MB/s at times (when no accumulation/reconstitution work is occurring).  There is still plenty of room for improvement, but at least I’m not stuck at 2MB/s anymore.

Empathy

March 29, 2016 2:45 pm

I’ve observed an interesting phenomenon in my life.  I describe it as cognitive versus emotional empathy.  Wikipedia seems to back me up on making a distinction between the two, though the field is apparently rather fuzzy still.

At some point in my life I developed cognitive empathy.  I don’t know when it was which suggests to me it was probably before high school.  Cognitive empathy being the ability to understand someone else’s emotional state and being able to predict someone else’s emotional state in a given scenario.  That is, I could recognize and understand that someone would feel devastated at the loss of a loved one.  But recognizing that reality didn’t cause me to feel any emotion myself on the matter.

I could watch movies and see something tragic happen on screen.  I could understand that the character would feel pretty upset, but it wouldn’t produce any emotional response in me.

During high school the mother of a friend-of-a-friend died.  A bunch of us attended the wake to support the friend.  I understood that it was painful for the friend and their family, but I didn’t have any particular emotional reaction myself (except feeling awkward and out-of-place and not knowing what to do).

Then at some point in my life I developed emotional empathy.  This only happened within the last 7 years.  For the astute readers, this coincides with when I got married.  While that’s a possible factor, I’m guessing the more likely connection is that I went through my mid-to-late twenties and brain development is still occurring in pretty important ways during that time frame.

By the late 20s, “there’s better communication between parts of the brain that process emotions and social information—like what people think of you—and the parts that are important for planning ahead and balancing risk and reward,” says developmental psychologist Laurence Steinberg of Temple University.

Whatever the reason, I developed this new emotional empathy which means I feel emotional responses to the things that happen to other people I observe (read, watch, etc.)  I find this really interesting since I definitively didn’t have this reaction before and noted that lack and then it developed.

For example, today I was reading a blog post from a guy at church about when his first baby was born a few weeks ago.  Some time after the baby was born his wife had a complication and began hemorrhaging.  There was no time to address pain control and he cowered in the corner holding their infant daughter while his wife screamed in agony as the doctors tried to save her life.

It makes me emotional just to write those sentences.  That must have been absolutely terrifying.  8 years ago I could have read and recapped that story recognizing that it would be awful, but not feeling any emotional response.  Today I can internalize being in that room and the emotion is powerful.  It instantly conjures up images of Heather and Corinne when they were born and Jess lying in bed resting.  Having the peace intruded by a puzzled expression from a nurse then shattered by a medical team and chaos and fear.  It makes me think of Jess’ surgeries (back when we were dating).  The urgency.  The uncertainty.  The helplessness.

I’m sure those experiences contribute to my ability to experience emotional empathy more acutely in this case.  This happens to be a particularly strong example, but other less-personal connections still occur.  I’m fascinated by that mental process and that change in how my brain responds to the world around me.

I know you want to know what happened to the woman.  As the medical team began to understand and deal with the situation someone did usher the husband out of the room.  They saved her life and she’s home now.  She needed ~3 liters of blood.  The average human body has ~5 liters (though pregnant women have more).  Again, terrifying.

Corinne’s First Birthday

March 19, 2016 3:07 pm

Corinne turned 1 on Thursday.  She’s almost walking (she’ll try to take a step and then go into a controlled descent).  She’s making some sounds, but not saying anything identifiable in any reproducible manner.  But she seems to have a sense of rhythm, so she has that going for her.  Regardless, she’s still adorable.

Jess wasn’t feeling well so I took Thursday off to stay home and help her get some rest.  Corinne opened presents after Heather got home from preschool and before she had to leave for gymnastics.

IMGP4850as

IMGP4855as

IMGP4864as

IMGP4867as

After gymnastics it was out to dinner at Cravings (pasta/pizza/calzones) and then back home for cake.  A white cake, colored blue, with chocolate ganache filling and vanilla frosting.

IMG_20160317_191525as

IMG_20160317_191915as

Corinne was pretty excited, but didn’t like the frosting on her fingers.  I don’t think she ate more than a few crumbs.  The rest of us are enjoying it though.