Post-Mortem: The Upgrade That Wasn't (Sprint 1)

v1
Mar 14, 2026 · 2.4 KB · 2 min read · by Sarah Blake

Post-Mortem: The Upgrade That Wasn't (Sprint 1) v1

Post-Mortem: The Upgrade That Wasn't (Sprint 1)

Summary

Sprint 1 of the Rails 3 upgrade was scheduled for 2 weeks. It lasted 3 weeks. The Rails version at the end of Sprint 1 is still 3.2.22.

What Happened

Week 1: Discovery

Attempted to run the app locally. Discovered that:

  1. Ruby 1.9.3 does not compile on macOS Ventura
  2. Ruby 1.9.3 does not compile on Ubuntu 22.04
  3. Ruby 1.9.3 compiles on Ubuntu 14.04, which is also EOL
  4. Set up a Docker container with Ubuntu 14.04. App boots. Database connection fails because PostgreSQL 9.2 client libraries are not available.
  5. Installed PostgreSQL 9.2 in the container. App boots. Tests fail. Wait, that's normal.
  6. App throws NoMethodError on every page. Traced to tomltech-utils gem, which monkey-patches String in a way that conflicts with the i18n gem but only on Tuesdays. It was Tuesday.

Week 2: The MysteryWorker

Attempted to understand the MysteryWorker. Findings:

  • It queries every row in the users table
  • It transforms each row using legacy_bridge
  • It sends the result to the SMTP server at 10.0.0.47
  • The SMTP server does not respond to any protocol we've tried
  • The emails are not emails. The content is base64-encoded binary data.
  • We decoded the binary data. It appears to be a serialized Ruby object from 2014. The class it deserializes into (LegacySync::Payload) is defined in the mystery_connector gem, which has no source code.
  • The MysteryWorker has been running every night for 9 years. We don't know what it does. The client doesn't know what it does. Gary might know. Gary is unreachable.

Week 3: The Decision

Met with client. Presented findings. Client asked "what happens if we just turn off the MysteryWorker?" We said we didn't know. Client said "let's find out." We turned it off.

Nothing happened for 3 days. On Day 4, the client received a phone call from a company in Ohio asking why they stopped receiving "the nightly feed." The client did not know they were sending a nightly feed to a company in Ohio.

We turned the MysteryWorker back on.

Action Items

  • Set up Docker environment for local development
  • Document the MysteryWorker behavior
  • Identify the company in Ohio
  • Contact Gary
  • Decide whether to reverse-engineer mystery_connector or rebuild the Ohio integration from scratch
  • Actually upgrade Rails (deferred to Sprint 2)
  • Sprint 2 estimate: "we honestly don't know"