Migration from Rhymes to Pulse: Our Journey in Building a Better ERP System

I'm Yacine Petitprez, the CTO at Ingedata, and I'm excited to kick off a series of articles where we'll dive into the technical decisions that shape our work here at Ingedata.

At Ingedata, the development team is working on one of the most ambitious projects we've tackled in recent years: migrating our old ERP system, Rhymes, to a shiny new platform we're calling Pulse. Rhymes has served us well over the past eight years, handling everything from HR and recruitment to the daily grind of project management. But as the company has evolved, so have our needs, and Rhymes just isn’t cutting it anymore.

Why We Needed a Change

Rhymes was built during a time when our company’s operations were simpler. But eight years is a long time in tech, and what worked back then now feels more like a series of workarounds than a robust system. We need an ERP that not only keeps track of time and metrics for our clients but also adapts to the variety of projects and data topologies we deal with today.

The main challenge? Flexibility. We need a system that's customizable enough to fit any project’s needs without requiring us to build custom modules for every client. And it’s not just about flexibility; it's about ease of use too. We can’t afford to spend weeks configuring each new project. It needs to be quick and painless, something our team can handle with minimal fuss.

Another big issue with Rhymes is its lack of interoperability. The system was never designed to interact with other tools or services, meaning we’ve had to do a lot of manual work to keep things running smoothly. That’s why with Pulse, we’re going all-in on an API-based approach. This will allow us to seamlessly integrate with third-party tools and make our workflows smoother and more efficient.

Why We’re Moving Away from Rails

Our development team has been riding the Ruby on Rails train for years, and for good reason. When we built Rhymes, Rails was the obvious choice. It’s a framework that lets you move fast, thanks to its opinionated design and the vast ecosystem of gems (libraries) that cover just about every use case you can think of. But as we’ve learned the hard way, Rails also comes with some baggage.

One of the biggest issues we've faced is code maintainability. Rails makes it easy to get a project off the ground quickly, but if your team isn’t careful, the codebase can get messy fast. Without strict code reviews and validation processes, you end up with a mountain of tech debt that’s a nightmare to manage.

It's true that a well-architected Rails application can be built without accumulating technical debt, but it often requires a team of highly skilled developers. The flexibility Rails offers, combined with its implicit behaviors, demands a deep understanding of the framework to avoid potential pitfalls. However, assembling a team with that level of expertise can be challenging and resource-intensive, making it less feasible in our situation.

Another challenge with Rails is how it handles business logic. The framework promotes the concept of “fat models and skinny controllers,” but this often results in models that are essentially just bloated extensions of your database records. Developers frequently end up embedding business logic directly into ORM classes instead of developing and maintaining true business models, which introduces a lot of hidden complexities and unintended side effects.

Take callbacks, for example. They might seem like a convenient way to keep your code DRY, but they can also cause a lot of headaches. Here's a quick example to illustrate:

class Order < ApplicationRecord
  belongs_to :user
  has_many :order_items

  # Problematic callback
  after_save :update_user_total_spent

  private

  def update_user_total_spent
    user.total_spent += self.total_price
    user.save
  end
end

This might look harmless at first glance, but it can lead to some serious problems:

• Triggering Multiple Saves: The after_save callback fires off another save on the associated User record, leading to extra database queries and potential performance issues, especially during bulk operations.

• Infinite Loop Risk: If the User model has its own callbacks that interact with Order, you could end up with an infinite loop of save operations.

• Unexpected Side Effects: Callbacks make the model's behavior less predictable, turning what should be straightforward saves into complex operations that are tough to debug.

• Violation of Single Responsibility Principle: The Order model starts taking on responsibilities that don’t really belong to it, leading to tightly coupled, hard-to-maintain code.

• after_save vs after_commit: When a transaction is rolled back, any code in an after_save callback won’t be committed, which is ideal if you want changes to the User object to be reverted as well. However, in some cases, after_commit is the better choice to ensure changes are only applied if the transaction is successful. The decision between after_save and after_commit can easily be overlooked during code reviews, and rollback scenarios are often not covered in test cases. Since transaction rollbacks are more common in highly concurrent environments, there's a good chance that staging won't catch these edge cases, leaving you to discover the issues in production. This can lead to serious problems and data desynchronization when something goes wrong, often only becoming apparent after the code has already been deployed.

These issues, along with a few others, made us realize that while Rails has served us well, it’s time to move on. Our team simply can’t afford to keep fighting these battles, and the truth is, we need something more suited to our current needs.

Enter Pulse: Built with Verse, Our Custom Framework

As we embark on this migration journey, we’ve decided to step away from Rails and build Pulse using our custom framework, Verse. Why Verse? Because we wanted something that gives us more control and flexibility without the pitfalls we’ve encountered with Rails.

We built Verse because, despite our best efforts, we couldn't find a suitable solution within the Ruby ecosystem that effectively supported an event-driven microservice architecture. We needed something that would allow us to separate concerns in a way that Rails simply couldn't handle, particularly in how it deals with modeling layers. With Verse, we've moved to a three-tiered architecture that cleanly separates concerns between services and effects—something we’ll dive deeper into in a future post. We've also made Verse open-source, but we're holding off on any major announcements until it stabilizes and we have solid documentation in place. For now, we're still in the building phase, and things might break, but once we’re confident, we’ll be ready to share it with the world.

Over the next few weeks, we’ll be diving deeper into the technical choices we’ve made, including:

• Event-Driven Microservice Architecture: Why we chose it and how we implemented it.

• ESE (Exposition - Service - Effect) vs MVC (Model-Controller-View): How our three-tiered setup differs from the usual web app approach (and why it’s awesome).

• Data Duplication: Why it’s the right choice and how to do it properly.

• Designing Event Bus using Redis Stream: A risky bet that paid off.

• ActiveRecord vs PassiveRecord: Why we’re moving to PassiveRecord.

• JSON:API: Why we like it and why we hate it.

• Auth Context: Record vs Endpoint security scope.

• Open Telemetry, Sentry, and Performance Monitoring: How we’re keeping tabs on everything.

Stay tuned as we pull back the curtain on our tech stack and share the insights, wins, and lessons learned from our journey in building Pulse!