Django today (2.2+) still uses the MD5 hash function to generate cache keys and database object names. They’re not security related, so that’s fine. But if you’re running on a FIPS-compliant system then MD5 is disabled and Django blows up.
Red Hat Enterprise Linux provides a patched version of Python which allows you to pass a keyword argument to md5, “usedforsecurity”. If you set that to False the system will allow your call to MD5 to go through.
So, RHEL has a supported method to tell the system that you’re using MD5 for non-security purposes; now we need to make Django pass that parameter without forking Django. We can do this with monkey patching.
I would prefer not to use a global monkey patch. It would work, but then anything using MD5 would be allowed instead of only the things I’ve verified are non-security related. This could be a compliance issue.
Implementing a localized monkey patch isn’t too much harder once you know a little Python voodoo. We load a copy of the hashlib module and then monkey-patch the copy to pass the “usedforsecurity” parameter. Then we inject the monkey-patched version into any modules we need to replace their normal hashlib object with our copy.
import hashlib
import importlib
def _non_security_md5(*args, **kwargs):
kwargs['usedforsecurity'] = False
return hashlib.md5(*args, **kwargs)
def monkey_patch_md5(modules_to_patch):
"""Monkey-patch calls to MD5 that aren't used for security purposes.
Sets RHEL's custom flag `usedforsecurity` to False allowing MD5 in FIPS mode.
`modules_to_patch` must be an iterable of module names (strings).
Modules must use `import hashlib` and not `from hashlib import md5`.
"""
# Manually load a module as a unique instance
# https://stackoverflow.com/questions/11170949/how-to-make-a-copy-of-a-python-module-at-runtime
HASHLIB_SPEC = importlib.util.find_spec('hashlib')
patched_hashlib = importlib.util.module_from_spec(HASHLIB_SPEC)
HASHLIB_SPEC.loader.exec_module(patched_hashlib)
patched_hashlib.md5 = _non_security_md5 # Monkey patch MD5
# Inject our patched_hashlib for all requested modules
for module_name in modules_to_patch:
module = importlib.import_module(module_name)
module.hashlib = patched_hashlib
When our application starts up it detects FIPS mode and runs the monkey patch:
modules_to_patch = [
'django.contrib.staticfiles.storage',
'django.core.cache.backends.filebased',
'django.core.cache.utils',
'django.db.backends.utils',
# 'django.db.backends.sqlite3.base', -- Only if system has sqlite installed
'django.utils.cache',
]
try:
import hashlib
hashlib.md5()
except ValueError:
monkey_patch_md5(modules_to_patch)
This lists the specific modules in Django that use MD5 for non-security purposes (determined by searching through the codebase and reading the code). Each module has its version of hashlib replaced with the monkey-patched version and Django is none the wiser.
The transit started before sunrise in California, but I thought I’d try to get a picture before heading to work.
My first shot overexposed the sun, but is kind of neat anyway. It’s wisteria below and our tree off to the right.
Pentax K-7, 200mm, f/32, 2s. (ND filters)
I took a few other shots and realized my lens has a number of spots that need to be cleaned. I thought they were mercury, but across several shots determined they were moving with the lens and not across the sun.
Looking closer I believe I did manage to get Mercury, smaller than my dirt spots. I believe the spot just above dead-center of the sun is Mercury:
Our tree out front has some pretty good coloring this year. Usually it goes from green to brown without much in between. I tried to get some pictures, but only a few looked like anything other than, yup, a tree.
I’ve been having issues with my 35mm lens ever since I got it. It seems to have an issue focusing in that it focuses behind where it should. So I finally spent some time trying to calibrate it and it seems to be improved but still inconsistent.
This first shot is from the 35mm, then I switched to the 50mm since the distance to the leaves prevented me from filling the frame on the 35mm.
July marked the 10-year anniversary of my starting work at the Lab. I got a certificate and lapel pin (not that I ever have any lapels on which to put it). And also a bump in vacation time and 401k contributions.
I’m still working with the same group I started with on the same general applications. But a lot has changed in that time.
Falling into the Past
When I started, the existing development team had collapsed for unknown reasons. I joined a basically black-box pile of software written in Java with custom, homegrown libraries and support packages. No documentation. Some of the compiled code had no matching source code (that I could find anyway).
It was a bit like being thrown back in time by at least 10 years in how web applications were built. I can admit now that it didn’t take long before I felt a bit worried about whether that job was going to be at all fulfilling for me. The code was a mess of JSP files intermingling core business logic with HTML output (a la the bad old days of PHP). It couldn’t be run locally, so making a code change involved modifying source files, compiling the code, deploying to the test server, restarting the application server (which also hosted the production application, so restarts impacted users), and then testing the change. A round-trip process of 15+ minutes. It was brutal.
I came in to the job with web application experience and I knew that there were much better ways of developing web applications. It wasn’t fusion research–that’s a different department–it was a fairly mature world.
By the Fall of 2009 I had settled in enough that I felt comfortable (or maybe just desperate?) enough to pitch the idea of using an open-source framework for our applications. Essentially saying, “Let’s toss the existing codebase and start over.”
Thankfully, my manager (who I’m grateful is still my manager) was willing to hear me out. But, we had some restrictions. The IT group was run separate from the software team and they dictated our deployment environment. So whatever we did needed to still run on the Solaris operating system, using Oracle Weblogic for the application server, behind Oracle iPlanet web server, talking to an Oracle database.
Even in 2009 that technology stack was unpopular at best and approaching obsolete. Trying to figure out how to configure Weblogic or iPlanet resulted in reading a lot of forum posts like “How do I do this in Weblogic?” answer, “You don’t, use literally anything else.” And that was if you were lucky enough to find an answer at all.
I had experience working with Python/Django, but that as a non-starter. Can’t deploy a Django app on Weblogic. It was a Java world. And our IT staff wasn’t much interested in supporting anything they didn’t already know.
Stepping into the Present
I found the Grails framework which lives in the Java ecosystem (technically it’s Groovy, but it runs on the JVM and as far as the system was concerned it was Java). It looked promising, certainly a huge step forward from what we had. We could go from coding with both arms tied behind our backs to one arm in a sling (which I did literally for a few weeks when I broke my arm, but that was years later).
I prototyped a brand-new type of application for our system. A geospatial view of the data utilizing Google Earth in the browser (back when GE was a browser plugin). Using a real framework I had a functioning prototype in a couple of weeks and it became immediately obvious that this was going to be our future.
That prototype was fleshed out into a full application and has since been overhauled 5 times and remains one of our core applications. It would never have been possible without moving our development work to Grails.
Over the past decade I kept pushing on modernizing our infrastructure as much as we could. Eventually the IT staff were consolidated under the software manager and over time they were replaced with people willing to consider a world outside of Solaris.
We moved our infrastructure to RedHat Linux, migrated from SVN to Git, switched our application server to Tomcat, finally swapped iPlanet for Apache, and even dumped the Oracle database for Postgres. We were finally in a sane development world (not to mention saving, literally, tens of millions of dollars on hardware and licensing). The arm was out of the sling and we got a lot of work done.
We had added additional software team members throughout the years, but never had a real focused approach to creating software. We were making it up as we went along and if you looked at the code, it showed.
In some ways, this was a necessity. As I said, the software team was non-existent when I joined. I had no experience managing a “real” codebase. Certainly not one expected to last for years. But, at the same time, the broader group we supported–the people we were trying to make more effective via software–didn’t have any trust in the software team. We needed to give them wins quickly to show them it was worth their time to interact with us.
And that’s what we did. That first prototype gave them a way of working with their data they never had previously and they wanted more. So, rather than work out a disciplined approach to building software with no experience to guide us, we wrote code. A lot of code.
Building the Future
It’s now 2019 and, for myriad reasons, it’s time to leave Grails behind. This year I became the team lead for the software group so I’ve had a lot of decisions to make. But, this time I have 10 years of experience to draw on.
We’re moving our applications to Python/Django. Something we could have done in 2009 had the IT staff been supportive of the software needs. But it is what it is. The good thing here is that it means the codebase will start from scratch. And, having had 10 years of dealing with a codebase built on the whims of the individual developers, we are doing things differently.
Our legacy applications are functional and stable so we have time to do things right. I spent a huge amount of effort up front picking the tools we’ll use to maintain good code “hygiene.” That is, tools and processes that ensure code is written consistently, that tests have been written, that obvious bugs have been caught, that the tests actually run and pass, etc. This discipline is already making a big difference in code quality.
We also have the luxury of an existing suite of applications–which we have to migrate over to Python–but which means we know what we’re trying to build ahead of time. A lot of our work in the past 10 years was speculative: we think this would be helpful, let’s build it and see what the users think. And a lot of it was trying to meet user demands when the users didn’t really know what they wanted, just a vague idea.
Our first rebuilt-in-Python application went to production in October. It took longer than expected, but I feel really good about the quality of the code in it.
Retrospective
Supposedly, software developers in the Bay Area switch jobs, on average, every 3 years or less. So why have I stayed here for 10?
I like my job. I like the work I’m doing. I like my manager. I like my coworkers. I like that I live 6 miles away and don’t have to get on a highway to commute. I like the impact my work has. I like that I can effect meaningful changes that improve our products. They don’t happen fast or all at once, but I’m quite sure I’d be doing something else if we were still writing everything in house and deploying on Solaris.
The nature of the projects I work on prevents me from saying much of anything about them, but I find the work meaningful. I’m not spending my days trying to figure out how to get more ads in front of more people. Our software has international impact on a regular basis with the goal of improving not just national security, but global security.
I’m not going to get rich, but our health insurance is great, we’re saving money for retirement, we managed to get a house while prices weren’t insane, we’re building up long-term savings and work stays at work. I don’t bring a laptop home, I don’t work on the weekends. I don’t answer emails after dinner.
I like what I’m doing, so I’m going to keep doing it.
On the 30th we worked on carving our pumpkins. Which means I worked on carving pumpkins since everyone else thinks it’s too gross cleaning out the inside of pumpkins.
Heather developed a cat-like design for her pumpkin. And Jess helped Corinne with a friendly-face design for hers. My design comes from the recently released game “Untitled Goose Game” in which you play a goose that terrorizes the neighborhood. Honk!
And costume pictures:
We started trick-or-treating at 6:30. The neighborhood seemed to get busiest around 7:15. Heather was scared of any decoration that looked like it might have the potential to move suddenly. Corinne was scared of any house without any decorations. But we made it down one side of the street, back up the other, and then back to the house. Then the girls were exhausted and Heather helped hand out candy while getting ready for bed (seems to be her favorite part of Halloween).
