Jon Allured

Verify Recurring Job Schedules With a Spec

2026-03-03T09:04:00-06:00

My Ruby on Rails app Monolithium uses Solid Queue for background jobs but also has some recurring jobs. When I migrated to Solid Queue I typoed the schedule strings for these recurring jobs and then had to make a PR to fix them. This felt like a silly mistake and seemed like something that could have easily been prevented. I made a mental note but moved on with my life.

Then this verification topic came up at work this week. I addressed it there and then took the opportunity to apply it to my Rails app as well.

Solid Queue Recurring Job Schedules

The recurring jobs that are managed by Solid Queue are configured with a YAML file that looks something like this:

# config/recurring.yml

production:
  name_of_job:
    class: MyBestJob
    schedule: every 10 minutes

The top-most key is the environment to run so here I'm using production but you could also specify jobs just for development but I never have. From there the next level of key is the name of the recurring job or task that you want to define. Then inside that key you specify the class and schedule and any additional configuration. What I've specified here is that every 10 minutes the MyBestJob class should be enqueued and worked.

For that schedule string here's what the Solid Queue docs have to say:

Each task needs to have also a schedule, which is parsed using Fugit, so it accepts anything that Fugit accepts as a cron.

Ah ha so now we know how these schedule strings are used - they are passed off to Fugit and whatever that gem accepts is allowed here.

Fugit Parsing

The Fugit gem can parse a lot of different types of inputs. It can take something high level like "every 10 minutes" but then it can also take low level cron strings like "5 4 * * sun" which I just randomly generated from crontab.guru. That range of inputs then is inherited by Solid Queue. Neat!

Writing a Spec to Verify Schedule Strings

With all this in mind the idea for this spec is to read in the config file, grab the schedule strings, and then parse them with Fugit. I learned from the Fugit docs that I could use Fugit.parse for this and it will return nil if the input string cannot be parsed.

I made Verify recurring job schedules with a spec and here's what I ended up with:

describe "recurring schedules" do
  it "only has valid schedule strings" do
    config_path = Rails.root.join("config/recurring.yml")
    recurring_config = ActiveSupport::ConfigurationFile.parse(config_path)
    tasks = recurring_config.values.map(&:values).flatten
    schedules = tasks.map { |task| task["schedule"] }
    schedules.each do |schedule|
      expect(Fugit.parse(schedule)).to_not eq nil
    end
  end
end

What this spec does is use the ActiveSupport::ConfigurationFile helper to grab the contents of the recurring jobs config file. That will already have converted it into a hash so then I deconstruct that hash and pluck out the values of the schedule keys and make an array out of that. Then I loop over that array and send each string to the Fugit.parse method with an assertion that the result is not nil. If the config file is valid then the test passes. If I edit the file and use a schedule that is invalid then the test fails. Confidence restored!

Conclusion

This is exactly what I wanted - a safety net that will ensure I do not repeat the mistake of using an incorrect schedule string. I'm not fully satisfied though. There are two things bothering me: the lack of help when the test fails and that Solid Queue should be helping me here.

For that first one what I noticed is that if I intentionally make the test fail then the error message is all about how something is nil. What I think a better test would do is give me which schedule broke. I bet I could write something like a custom matcher for this but who has the time??

Then the other thing is that honestly why doesn't Solid Queue give me something to work with here? If I had the idea for this spec then I'm sure others have as well. In my imagination there are all these teams using Solid Queue and writing the exact same type of test - what a waste! Many Rails related gems have a test helper or test mode or something like that. I guess if I really cared about this what I would do is open up an issue or PR on that project but who has the time??

Week in Review: Week 9, 2026

2026-03-01T15:52:00-06:00

I ended up taking Monday off this week because I hurt my back and needed to work on making it feel better. What I would like to tell you is that I hurt it because I was doing something awesome. Sky-diving or whatever. Nope. I hurt it because I slept funny. That's how old I am.

Highlights

worked 32:00, 8:00 PTO
published Show Timestamps in Local Zone on here
finished The Color of Magic
finished Dress Your Family in Corduroy and Denim
created 14 Pull Requests

Wow can you believe I finished 2 books in a week? Ok I actually did not. I listened to The Color of Magic on Audible and that Sedaris book was short so I flew through it. Both were great!

Oh and The Color of Magic is the first book in a series called Discworld. I kinda regretted listening to it because I think I missed some of the jokes. The next one I read will be in print.

Pic of the Week

Jack made this:

Please note that this snowman has a mohawk.

Good luck falling asleep tonight and if you do then expect this monster will haunt your dreams. I mean nightmares.

Next Week

This week coming up is a short one for me because I'm taking a few days off work to do a short trip to AZ. My Grandma is turning 90 and a bunch of us are going to celebrate with her. I cannot wait to get out of the cold for a few days.

Show Timestamps in Local Zone

2026-02-25T16:44:00-06:00

My Ruby on Rails application Monolithium stores times in UTC and thus far I've been content to show them without considering timezone. As an example, the Boops Page will show the created_at timestamp but it's just in UTC. This annoyed me so I decided to look into what it would take to show them in a user's local timezone.

How Should I Do This?

The first questions I hit were around where I would do the conversion from UTC to another timezone and which other timezone should I use. Should I do the conversion client side or server side? What if I just used my own timezone since this is largely a personal app? Oh maybe I should only do this when I'm signed in? Could I use a cookie to set the timezone and pass that into the server side code?

There's a Gem For This

While pondering these questions I did some searching and found that there was already a gem for this! I found local_time from Basecamp which seemed close enough to official that I should just use it. Plus it answered many of these questions.

How it works is that it provides view helpers that write your timestamps into time elements with data attributes. Then it also includes some Javascript that runs client side and will process these time elements with logic that converts the timestamp into the user's local timezone. Neat!

I decided to try it out and made Localize times with local time gem which demonstrates how to use it for my case. It wasn't hard - I did end up making my own helper method that wraps the helper method that the gem provides:

def in_tz(time)
  local_time(time, Time::DATE_FORMATS[:default])
end

I did this because I always wanted to use that format and it made the view code much more simple:

%p= in_tz something.created_at

Refused Bequest Like Crazy

At this point I noticed that the diff included a Javascript file in the vendor folder and that got me thinking about what this gem was actually doing. I started poking around and reading more of the README. The features of this gem that I was not using started piling up:

helpers for both times and dates
adding a title attribute to the time element
support for arbitrary formats
integration with i18n
timezone edge cases
helpers for relative time that are updated every minute
support for 24-hour vs 12-hour times

So yeah it wasn't long before it dawned on me that I was barely using any of the functionality that this gem provided. Refused Bequest like crazy. Isn't there a better way?

Making My Own Solution

I checked out a new branch and reversed the install of the gem. I left the changes to the view files and set my sights on what it would take to arrive at the same outcome but with much less code.

Here's what I had in mind:

update the helper method to create a time tag kinda like the gem does
add a data attribute to that tag that tracks if it has been converted yet
add a Javascript event listener for when the page is loaded
create a function to find all the time tags and convert them
update the data attribute once done converting

Having the code of the gem to refer to made it actually pretty easy. I did this and made Localize times with intz which is a much more simple and smaller change! By cutting out most of the features of the gem and finding the smallest bit of functionality that I actually needed for my use case I arrived at something I can understand and easily maintain.

It isn't tested though - don't tell anyone!

Formatting Times in Javascript

Back to the code for a second - the function that converts the time elements is pretty small and I think easy to understand:

const intz = () => {
  const timeElements = document.querySelectorAll("time[data-intz='false']")

  for (const timeElement of timeElements) {
    const datetime = timeElement.getAttribute("datetime")
    const localTime = new Date(Date.parse(datetime))
    const localText = formatTime(localTime)
    timeElement.innerText = localText
    timeElement.setAttribute("data-intz", "true")
  }
}

Find all the time elements and loop through them. For each element pull out the timestamp in UTC, convert it to a local time that is formatted nicely, update the text of the element, and then update the data attribute. Pass that function to the event handler for turbo:load and boom, timestamps get converted!

Note that I originally had been using DOMContentLoaded but then noticed Turbo was getting in the way and made Use Turbo event to localize times to switch the event.

What ended up being the worst part of this was that formatTime function. Did you know that formatting a Date object in Javascript sucks? It does if you don't want to use a library! There is nothing like strftime and so I had to build something myself. Pain points included:

month is zero-indexed so you have to add 1
you get the month with a function called getMonth but you get day with getDate - the getDay function returns the number of the day of the week??
hour is 24-hour time so you have to convert it to 12-hour time yourself
there is nothing to get the am/pm out
you have to pad month, day, hour, minute, and second yourself

As someone that avoids Javascript as much as possible it was life-affirming to see how stupid this all was. Feel free to have a look at my formatTime function if you'd like to see how I addressed this problem. I'm pretty sure it works just fine.

Assessing Trade-offs

A fair question might be "why build this yourself - why not just use the gem?" and ultimately this is about trade-offs. In my mind I would rather build this myself because I'm using so little of what the gem does and the code that I added is not complex enough that I foresee problems maintaining it. That's the calculation I'm making. I don't know, maybe you would make a different one!

I do wish I had a way to unit test those functions but oh well. I don't relish the idea of spinning up an entire Javascript test suite just for one file.

Week in Review: Week 8, 2026

2026-02-22T09:34:00-06:00

This was a short week for me with Monday being a holiday. Not much to report other than I did my best swim this week - 850 meters!

Maybe worth mentioning is that I have continued to volunteer to purchase some groceries for families that are too afraid to do it themselves. I get a list and run over to the store to grab what they need. The groceries come home and I get them ready for the next day when I drive them over to the collection point. Then other volunteers deliver them. This is filling my cup and giving me some hope that the world still contains good people.

Highlights

worked 32:00, 8:00 PTO
finished Hero of Ages
created 8 Pull Requests

I continue to work on automating the process of announcing new blog posts on my socials. This week I landed the modeling in order to keep track of blog posts and then updated my Bluesky client gem to support these types of announcements. I learned a bunch about richtext and embeds and how the Bluesky API deals with them. Next up is Mastodon which looks to be a bit more simple.

Some Best Things

I finished the Mistborn series by Brandon Sanderson which I really liked! It's a fun trilogy that does not take itself too seriously and had enough twists and turns to keep me wanting to know what happens next. The magic system is unlike anything else I've encountered.

The other thing I'll mention here is the Wonder Man show which Jess and I finished. I had been hearing some buzz about it so convinced her to give it a shot. She mostly humored me and didn't even stick it out to the final episode but I thought it was cool. I especially liked the Doorman episode - ding dong!

Next Week

Should be a normal, quiet week. No more volunteering while they assess what the community needs going forward. I did upgrade to Ruby 4 on all my projects so maybe this week I'll do a Rails upgrade that I've been putting off.

Upgrading to Ruby 4.0

2026-02-14T11:28:00-06:00

Ruby 4.0 was released a couple months ago on Christmas and I had been meaning to upgrade for a while. I noticed that version 4.0.1 came out too so that's always a good sign that things are stable and ready for prime time.

Reviewing Release Notes

I started by reviewing the release post on the official Ruby blog. Nothing really stood out as all that controversial and so I figured it would be a pretty straightforward upgrade. It was.

But that isn't to say that as a fan of the Ruby language I'm not excited by what was in those release notes - there are some cool things in Ruby 4! You should search that page for "ErrorHighlight" and see what they did to improve the error messages for ArgumentError.

Installing

I still use asdf as my Ruby version manager so next was installing:

$ asdf plugin update --all
$ asdf install ruby 4.0.1
$ asdf list ruby
  3.3.10
 *3.4.8
  4.0.1

So the goal is to upgrade all my projects to Ruby 4.0.1 and then uninstall those other versions.

Setting Default Ruby

My dotfiles repo is where I set the default Ruby for my system so I made Upgrade default Ruby to set this to 4.0.1.

Upgrade All the Projects

Now the toil - go to each project and do a PR that upgrades it to Ruby 4.0.1:

Mostly the process was:

edit the .tool-versions file to switch to 4.0.1
run bundle install to install the project's gems
run bundle update --bundler to get the newest bundler version
run bundle update --all to upgrade the project's gems
run bundle exec rake to see if anything broke

Oh and then if a project has a Ruby version matrix on CI then I'd also drop the 3.2 and add 4.0. I actually forgot this on jay so I opened Update CI matrix as a follow up.

Monolithium was the project that I assumed would be the most challenging to upgrade but it Just Worked. I did notice that it needs a Rails upgrade and I have been ignoring the Tailwind upgrade as well. Those are problems for Future Jon!

Cleaning Up

This was an easy upgrade and the final step was to clean up my older Ruby installs:

$ asdf uninstall ruby 3.3.10
$ asdf uninstall ruby 3.4.8
$ asdf list ruby
 *4.0.1

I love boring software!

Making Time for Boops

2026-02-13T14:44:00-06:00

A couple months back, just before the Christmas holiday break, I received a TIME smart clock as a gift from Shopify. They gave everybody the choice between a few different items from merchants on the platform. I thought it was both a fun way to recognize employee effort and a cool way to dog food Shopify's software. Plus merchants earned a little money off the project. Win-win-win.

Setting it up was easy and fun and I instantly fell in love with the retro look of the display. Those pixels are so chunky!

The default face is the calendar app and it is fine.

Before I headed out for the holidays I messed with the device a little. I learned enough to kinda understand what was possible. Ideas bloomed but I didn't get very far.

Then this was Hackathon week at Shopify and I decided to take one of those ideas and run with it.

The Idea

What I had in mind was something where I'd make a public page on my Rails app where people could click a button and send a message to appear on the device. As I pondered the idea over the couple months, a name for the idea came to me: Boops. Once you have a good name how can you not build it right??

So yeah, people would create a Boop and send it to me. Fun!

LaMetric Developer Account

As I sat down to start really digging into this concept and get something built I first had to create a developer account with LaMetric, the company that manufactures the TIME smart clock. It was just a free sign-up and I was in, no big deal. But the screen you land on when creating your account presents you with 3 choices of apps to create:

Indicator App
Button App
Notification App

I had no clue! Luckily there was a link to the LaMetric Developer Documentation and there I spent quite a bit of time reading and tinkering with things to figure out what I wanted to do.

Frames and Clicks

Ultimately I decided to build an Indicator App. What I needed to do was create 2 API endpoints - one for frames (what to display on the device) and one for clicks (what to do when the button on top is pressed). The device sends GET requests to these endpoints and will error unless it gets a 200 back. This is what I sketched out:

GET /api/v1/time_clock/frames
GET /api/v1/time_clock/clicks

My Rails app Monolithium already had good patterns for adding API endpoints so making Add endpoints for time clock was not hard. At this point the frames endpoint just returned a static default frame and the click endpoint did nothing. But it was enough to complete the form in the developer portal, get verified, publish the Indicator App, and install it on my device.

I tested it and it worked - I tailed my Heroku logs and saw the polling for frame data and when I pushed the button I saw the click request come through.

The Boop Lifecycle

At this point I started thinking about how I would create, show and then be done with a Boop record. I was thinking of this as a queue. There would be a page where a button would be clicked to create the Boop record. Then it would show up in the frames API call and display on the device. Then I would press the top button which would send a click API call which would dismiss the Boop and the next one off the queue would show up. Wash, rinse and repeat.

Look and Feel

This lead me to think about how the frames would look on the device. I was picturing an icon on one side and the text on the other. I wasn't totally sure about the text but I thought that it would be fun to pick some icons. LaMetric provides icons for developers to use and has a Gallery Page so I went hunting for some good options - here's what I picked:

There are a LOT of icons to choose from but these stood out as pretty nice.

The Boop Rails Model

Given the lifecycle I had in mind and the look and feel that was starting to solidify in my mind, it was time to start modeling. What fields should be on a Boop record? I jotted some notes and ended up with this migration:

class CreateBoops < ActiveRecord::Migration[8.0]
  def change
    create_table :boops do |t|
      t.string :display_type, null: false
      t.integer :number, null: false
      t.timestamp :dismissed_at
      t.timestamps
    end
  end
end

Putting it all together I made Add basic boop lifecycle and then Different Boop frame approach quickly after playing with the text. I didn't have a button to click on to create Boop records so I just opened a Rails console and did it manually:

> Boop.create(display_type: "smile", number: 16)

It worked and I started seeing them show up on the device! I pressed the button, got a satisfying success sound, and then the next one showed up. Here's how one looks:

This is a Boop record with display_type of smile and number of 16.

There's an animation where one frame slides down on top of the other. It's Boop time!!

Adding a Public Boop Page

The next milestone was to create a public page where people could pick an icon and send me a Boop. This is a silly project so I had some fun writing up some copy and then adding some flair.

There did end up being an interesting technical challenge - the Boop.display_type field is stored as a string but I wanted to run the icon options on the page and have them be backed by a radio button group.

I tinkered with this for a while and ended up really liking how it came out. What I did was wrap a label element around the input and img elements for each option. Then I hid the input and used some fancy CSS to arrange things nicely in a single rail with a bottom border. Because the icon images were inside the label element when you click on the img then it selects that input element.

Brief aside to mention a trick that I had not considered. Given this type of markup structure:

<label>
  <input type="radio">
  <img>
</label>

How might we highlight the image of the checked option? Turns out it's easy!

:checked + img {
  border-color: pink;
}

That's not exactly what I ended up with but close enough. The trick is knowing that the pseudo selector :checked applies here and if we use a plus sign then the styles will apply to the next element not the checked one. CSS is maddening to me but when you find the perfect way to do something it is so satisfying!!

Ok back to the public Boop page - here's what I ended up building:

There's more there - take a look to find out what's below the fold!

Some Boops are spooky - send me a Boop right now! Please also keep in mind that I made Add redirect to boop so that you will have a memorable shortcut: jon.zone/boop so you can Boop anywhere, anytime.

Quality of Life Improvements

My Rails app has a generator for CRUD pages so it is very easy to add an admin section:

$ bin/rails generator crud:pages Boop

So I made Add Boop CRUD pages and got that going. I want to make a generator command for adding API endpoints but haven't gotten there yet so I did it manually with Add api endpoints for boop. I'm glad I did these 2 PRs because it helped me see a couple areas for improvement around Rails validations for the Boop model.

Boop from the Terminal

With the API endpoints in place it was now possible to Boop right from my terminal using httpie:

$ http https://app.jonallured.com/api/v1/boops display_type=skull

So then I went to my Raspberry Pi and setup a cronjob to Boop me during work hours on the 7s:

7 8-16 * * 1-5 http https://app.jonallured.com/api/v1/boops display_type=robot

Wait did you notice a new display_type?? Yep I made Add more Boop display types because I felt like an automated Boop should have a robot icon. While looking for a good one I also found a fun monster one so I added that too. I didn't add them to the public page so they are sorta Easter eggs for you dear reader.

Boop with a CLI

Don't you think it should be easier to Boop? No one should have to remember a URL to create a Boop. I already had a solution - my CLI Ruby gem that wraps the Monolithium API called mli. What I wanted was this:

$ mli boops create --display-type monster

I made Add Boop command and resource and had that done. Installing that project is done by cloning the repo and then running rake install so there's a bit of friction but boy is it easy to Boop now!

Conclusion

I want to thank Shopify for buying me this fun TIME smart clock and for providing the Hackathon time to play with it. I had a blast stepping away from my normal work to do something completely different. My work is so digital that when I can do something with hardware that manifests itself in the real world I just have to do it.

And I hope this project made you smile. Maybe feel some whimsy? Things are very challenging in the real world right now so having some fun was a wonderful distraction. Keep those Boops coming!

Week in Review: Week 6, 2026

2026-02-08T15:33:00-06:00

There were parent/teacher conferences this week so the boy was off school Thursday and Friday. He's 10 so not too bad but I just found it hard to focus. We went to the gym on Friday and hit the pool which was pretty fun. He likes to put on the flippers they let kids use and race me. He always wins!

Minnesota is still a mess despite what the media or the White House might be saying. As a tiny bit of positive news though, I will tell you that I got hooked up with an organization that helps get food to people that are too scared to leave their house. This was the second week where I got a grocery list, went shopping, and then dropped the food at this organization. Then other volunteers collected it for delivery to these families.

I mean I'm not protesting in the streets but this did feel really good. Very direct help to those in need. But. BUT. This is not normal and we should not rest until ICE is abolished.

Highlights

worked 40:30, no PTO
published New Old
created 9 Pull Requests

I spent most of my time for personal projects tinkering with my CLI tools. This work is both fun and also annoying. My love of Ruby and boring tools has lead me to use Thor as the foundation for these tools and wow is it painful to work with. I'm getting there though - almost have some patterns I don't hate.

Some Best Things

Are you playing Enclose Horse yet? You should really be playing that game every day - I usually play while I'm drinking my coffee in the morning. Watch out for bees! I guess h/t to Erik Krietsch but he knows what he did to me and my free time.

Superb Owl Prediction

My mind tells me that the Patriots will win yet again but my heart hopes both teams lose somehow. I suppose I'll cheer for the Seahawks. Official prediction for bragging rights in case I'm right: SEA 17 / NE 24.

Next Week

You are running out of time to get your Valentine's Day card and flowers arranged. And by "you" I mean "me". Gotta do that tomorrow for sure. Other than that I have a chill week it looks like. I expect to do another grocery store run for some needy families so looking forward to that. Oh and I really do want this to be the week I upgrade my Ruby projects to 4 - it's time.

Week in Review: Week 4, 2026

2026-01-25T16:43:00-06:00

This was a weird week on many levels. Monday was a holiday but then people were talking about a general strike for Friday. No work, no school, no shopping. I wasn't sure what I would do but then at the same time an insane cold front came through. The high for Friday was predicted to be -10 F which was enough for schools to close.

Since Jess and Jack were going to be home and the strike was on I decided to observe it as well. We basically camped out at home and rested up trying to stay warm. We watched some coverage of the strike and talked to Jack about why we were striking and what it meant. We talked about being proud of how the people of our state were responding to the situation. He's 10 so we did our best.

Then the very next day ICE agents murdered another Minnesota man. Crushed, pissed, scared, powerless. Lots of feelings. What do I tell my son now?

Highlights

worked 24:00, 16:00 PTO
created 10 Pull Requests

Most of my time was spent exploring how I will automate the task of announcing on social media when I pushing a blog post. I have mostly got this working locally for Bluesky but nothing I'm ready to actually ship just yet. And I want to get into Mastodon's API too so I can get that automated too.

Some Best Things

I'm trying out a new section - let me know what you think. The idea is to just share some of the best things from the week. Here goes.

Some fun new music came out this week. My new favorite band The Barbarians of California released Bomb To A Knife Fight. In Apple Music it is on a "single" with a couple other songs that I also really like. A couple days later Pelican released an EP called Ascending. They are an instrumental metal band but they did a track with Geoff Rickly which is kinda rad.

I was listening to episode 199 of Working it Out and the guest was Jordan Jensen. They talked about her new special and that reminded me that I had noticed it but not watched it yet. She was so funny and cool on the podcast episode that I paused it and then later that night I watched Take Me With You and it was also very funny and cool.

A bunch of us went to the Timberwolves game on Thursday and they were playing the Bulls. I grew up loving the Bulls and so it was really rewarding to watch them actually pull off an upset and win! This win got them to .500 which…is better than nothing.

Next Week

We have a friend in town so I'm excited to see him for a dinner date. There are also a few other family things including a band concert so you know just a wholesome Midwestern week coming up. Other than that I can't think of anything really going on in my area - you?

Week in Review: Week 3, 2026

2026-01-18T13:20:00-06:00

This was another hard week to be living in Minnesota. Don't listen to what the White House is telling you - this ICE activity has nothing to do with fraud and everything to do with terrorizing a community for political reasons. Hard to stay positive but sometimes channeling my feelings into working on my personal projects provides some relief. I did that a lot this week.

Highlights

worked 40:00, no PTO
published Blog Balm for Pudding Time
published Configure Spec Task Without Verbose Output on here
published Building Analytics Reports in Rails Using Apache Logs on here
created 23 Pull Requests

The biggest thing was landing the work to create an Analytics section on my Rails app for this website. That blog post goes into great detail but I'll just say that it feels really good to have that done and shipped. I did have to fix something today because I wasn't correctly managing the new access log data so I'll see if that fix worked tomorrow.

Pic of the Week

On Friday the Marvel Rivals game started a new season and now you can play as Deadpool. When Jack got home from school he got some time to try him out and he really likes the new character:

You would think jumping like this would mess him up.

Next Week

Monday is a holiday here in the US - Martin Luther King Jr. Day and so I have that day off work. Jess and Jack have it off school too so we will all be home pondering how this country has fallen so low. There's talk of a protest day on Friday and I'm keeping an eye on that.

In terms of projects - maybe on Monday I'll find some motivation to build something. It might be a good time to upgrade to Ruby 4. I'll also be monitoring my fancy new ETL pipeline to make sure my Analytics section is working correctly.

Building Analytics Reports in Rails Using Apache Logs

2026-01-17T16:42:00-06:00

Over on Evaluating Apache Access Log Data I dove deep on the data available to me from my many years of hoarding Apache access log data. By working with the logs in this way I learned what data points existed. I got familiar with the patterns and saw a lot of junk. That made it possible for me to hone in on what exactly I cared about.

The end goal was to create a section on my Rails app for website analytics reports. I broke up this work into 4 PRs:

This post will cover these PRs and detail my approach plus any interesting things I learned.

Import Apache log files with ETL pipeline

The first PR of the set was all about getting the modeling right and then using the concept of an ETL pipeline to import the Apache data from S3. I added rake tasks to work on this locally and then also with my production deploy at Heroku.

Adding a parent model

Something that popped out of this process was splitting up the modeling into a parent/child relationship:

ApacheLogFile: represents a daily file of requests
ApacheLogItem: represents an individual request

An ApacheLogFile record has many ApacheLogItem records. The ApacheLogFile is where I store the content of the access log text files and the ApacheLogItem is where I break that text down into lines and create one for each of them. Well that's not exactly true but more on that later.

Making this modeling choice had a really nice benefit which is that I could create the ApacheLogFile records and download the Apache log data as my extract step but then the transform and loading could happen separately. Prior to this modeling choice my approach was more like Extract-Load-Transform and was clunky.

If you want the juicy details this is the commit to check out: d23758a. It's where I have the database migrations so you can see all the various fields.

Associated Objects for ETL Classes

The next 3 commits on this PR are actually very readable. They each take a letter in ETL and create an Associated Object for it. Side note: you should totally check out this gem if you haven't seen it yet - it's a great way to organize Rails code!

Anyway here are the next 3 commits:

Extractor Class

The Extractor class is simple: use the dateext value to construct the S3 key and then grab the data. Unzip it and then update the given ApacheLogFile record we are working with.

Transformer Class

The Transformer class is bit more complex but mostly because it is where I am mapping the individual access log lines to the database columns where it will ultimately be stored. I organized it as parse, then normalize. So parsing is where the regex happens with a bit of logic about how to handle weird parse results. Then normalization is where I can massage things so that there is consistency.

This class ends up taking an array of strings (the log lines) and transforming each into a hash. This is stored back on the ApacheLogFile in a jsonb column.

Loader Class

We are now ready to take the parsed log lines and turn them into ApacheLogItem records but only if they are worth importing. As I said at the top, I spent a lot of time evaluating this data and pretty quickly I realized that my server receives a LOT of garbage requests. This resulted in me building up a set of rules that I wanted to use to eliminate the irrelevant requests so that I would be left with only requests that were interesting to look at.

In Rails terms I decided to formulate these rules as validation. I started off writing them right in the ApacheLogItem class but soon realized that I'd want something better so I checked the docs for my options and that reminded me that you can validate a Rails model with a class that inherits from ActiveModel::Validator so that's what I did in this commit.

The Loader class then becomes very simple - take the parsed data from the ApacheLogFile record, use it to create an ApacheLogItem record and validate it. Only persist the valid ones and then we are done.

I really enjoyed the process of writing this commit because there was a tight TDD loop where I would take my "rules" and break them down into test cases and then just knock them off one-by-one.

Importing data

Now that I had each step of the ETL pipeline defined I needed just a bit of orchestration. What I wanted was to be able to run a Rake task and have it enqueue background jobs that would call a method to kick off the import process.

That final bit looks like this:

class ApacheLogFile < ApplicationRecord
  def self.import(dateext)
    apache_log_file = create(dateext: dateext, state: "pending")
    apache_log_file.extractor.run
    apache_log_file.transformer.run
    apache_log_file.loader.run
    apache_log_file
  end
end

That's just a snippet of the model file but yeah I really like how that method turned out. The ETL import pipeline is defined as creating an ApacheLogFile record and then calling run on each step of the process - super easy to see how it all fits together.

Bubbling back up to my overall goals then it was clear that I'd want 2 tasks - one to load an individual file and one to backfill. I'd use the former locally to iterate on this process and then check that everything can be imported correctly with the backfill task. Those are invoked like this:

$ bundle exec rake apache_logs:backfill
$ bundle exec rake apache_logs:import[20251201]

The former would slurp up the entire set of Apache log files on S3 but the latter would just grab the one that matched the dateext that I passed in. Tiny aside: dateext is the logrotate term for the way that it puts a datestamp into the filename like access.log-20251201.gz so I followed that terminology for better or worse.

Running the import locally

Prior to the addition of the ApacheLogFile model I would create an ApacheLogItem record for every line in the access log data. That created millions of records that I ended up culling down to less than a hundred thousand as I learned more. This is why it was a better to extract everything into the ApacheLogFile record, transform the data there and only actually load the records that I wanted to keep.

When I first started working with the data it would take 3 or more hours to import everything but with this better modeling it was down to 10 minutes.

Running the import on Heroku

Once the PR was merged and deployed to Heroku I kicked off the backfill and it was quite an adventure! Turns out my Heroku worker dynos were running out of memory and it caused all sorts of problems. I detailed this in a comment if you are interested in the journey but the solution was a combination of changing the thread count to 1 and just restarting the dynos when they were crashed.

I did have to do some data cleanup but it wasn't too bad and ultimately the actual ETL pipeline was all correct - the only issues were more like the constraints of Heroku not the code.

View analytics reports with Apache log data

Major milestone reached! The Apache access log data was now sitting on Heroku and ready to be viewed. All my careful tinkering with the ETL code was correct. PHEW. But this data doesn't do much unless we have a way to look at it.

Start With Sketching

When you aren't sure what to build then a great place to start is with some sketching. I grabbed some paper and a pen and sketched out a few things. I drew in very broad strokes just to get some ideas flowing. As I went I also scribbled some notes about the way the reports might work. Param names and values - things like that.

I quickly landed on a page design that used some text labels to indicate the month and year of the report followed by a table of…something.

This sketch explores setting the metric/mode of the report:

I find that even very rough sketching like this helps me zero in on what I'm building.

This sketch explores navigating through periods:

There are other pages that navigate between periods with these types of links so I hoped I could reuse code from them.

URLs as UI

Before I knew it I was starting to look at my sketches and ponder what the URLs would look like. I moved to writing out URL options and iterated a bunch. I actually filled up a few pages with different approaches to organizing things via URL parts. This was extremely helpful in focusing me on what I wanted to build.

Here's where I ended up:

# Rails route
/analytics/:metric/:mode/:year/:month

I knew I had nailed it when I could look at a given URL and translate it into English. Here are couple examples:

/analytics/browsers/summary/2021/01
=> summary view of browser names during January 2021

/analytics/pages/detail/2025/11
=> detail view of pages requested during November 2025

At this point I put paper and pen down and started coding with some enthusiasm because I had the beginnings of a plan.

Metrics by Modes in Periods

The plan was to build a page where I could pick a metric to view by a mode with matching records in a given period. To start I'm focusing on metrics I either already had or could easily get:

Browser - take the user agent value and parse it with the browser gem
Page - use the page request value straight from the log
Referrer - use the header value straight from the log

For modes I was thinking of a summary that would group and count items plus a detail view in case I wanted to look at individual log items.

I like reports that use a sentence to describe what data they are showing.

At this point I felt like I had explored the idea enough to have some thoughts about what I wanted to do - spike complete. Time to head back to the main branch, start a system spec, and pivot to making this for real. That's what the next PR does.

Reporting Classes

The only maybe interesting thing I'll call out is that I ended up building this out with page objects. Here's the lineup:

Summary Report - uses Analytics::SummaryReport and Analytics::SummaryRow
Detail Report - uses Analytics::DetailReport and Analytics::DetailRow

I extracted an Analytics::BaseReport class as I went and it was nice to see how using POROs made this code easy to land. I deployed this to Heroku and clicked around and the pages loaded great with no performance issues at all. Something was actually easy!

Add CRUD pages for analytics models

Next on my list was to do a little bit of paperwork. A while back I created a Rails generator that will take a model and create the CRUD pages that can be used to admin those records. I don't use these admin pages all that often but I do like making them just in case it's handy to be able to use a UI to tinker with things. I actually did wire up the detail report to link to the show page so if I see something off in the reports then I can jump right to a view of the record.

Here's the list page:

I should have added something like dateext and maybe line number to make this more useful - oh well.

And here's the detail page:

I keep these CRUD pages shallow rather than nesting them because it is more flexible.

To generate these pages I ran these commands:

$ bin/rails generate crud:pages ApacheLogFile
$ bin/rails generate crud:pages ApacheLogItem

From there it's just a matter of filling out some REPLACE_ME details and getting the generated specs to pass. The PR alternates between these two steps.

I do need to write up a post about my generator - so much to do so little time.

Configure recurring import process for analytics data

The last part of this that needed to be done was configuring a nightly job to import whatever new data had been uploaded to S3. I already had a few jobs setup in my config/recurring.yml so adding another would be easy.

What I had in mind was an update to the Extract class that would move files into an "archive" folder. The way that logrotate works is that each day has an access.log and an error.log file so here's what I was thinking:

# before
domino/logs/access.log-20251201.gz
domino/logs/errors.log-20251201.gz

# after
domino/archives/access.log-20251201.gz
domino/archives/errors.log-20251201.gz

Take each day's files that I had already imported and just move them from "logs" to "archives". Then list what remained in the "logs" folder each night and it would only be the new day's files. Archive those too and we have a recurring import process.

The PR is pretty straightforward but I did have to do a bit of trickery when running it over the records that were already imported:

> ApacheLogFile.all.each { it.extractor.send(:archive_files) }

When I attempted this in a Rails console in production the Heroku dyno crashed. I'm not totally sure why - I even tried again with ; nil at the end in case it was trying to return the records. Rather than spend any further time fighting with out of memory dynos I just ran it on my laptop and called that good enough.

I use the Transmit app to view my S3 buckets so here's what it looked like before:

Please pay no attention to the left hand side where you can see my home folder listing.

And then here's what it looked like after I ran the above command and did a little extra random cleanup:

I wonder why I have a folder called javasharedresources and what would happen if I removed it.

And that is correct - those 2 remaining files were not imported yet and should be picked up tonight. Along with whatever lands in there for today.

Conclusion

I've been working on this project since October when I migrated my blog to Jekyll and started posting again more seriously. I had no idea it would take this long! I also did not know that this project would include all these interesting little details and learnings. Now that this is landed I have some other ideas for ways I can use this data but I'm just really happy with how it turned out.

I also feel really validated that I can get website analytics info for my personal site with Apache access logs and not use Google Analytics. Sure, I don't have as much analytic data but I have enough. I know roughly how many requests my site gets, which browsers people are using, and where that traffic is coming from. And I get all this without having to include Javascript in my site nor having to expose myself and my readers to the privacy destroying machine that is Google. Please clap!