1. 32

  2. 15

    I’m not a fan of the ActiveRecord pattern as such, but just be aware that current ActiveRecord (the ruby library used in rails) has a pretty solid query builder that is mostly transparent - and can be rather well augmented with views and materialized views as needed.

    I have in fact somewhat recently fixed a typical naive loop over retrieved objects to use idiomatic includes/join.. Going from ~minutes to sub-second execution. Without having to dip down to sql.

    So I’m à little torn on this whole critique - sure, the AR pattern, in its basic naive form is pretty bad, otoh the poster child implementation doesn’t really suffer from most of the outlined issues if used idiomatically.

    1. 3

      I’ve not written code against the original ActiveRecord library that named the pattern so I wouldn’t know. I suspect AR-the-pattern predates AR-the-library but anyway: AR-the-pattern is still in common use today and that is what I’m talking about.

      I discuss in the article how it is not always as easy as taking nested for loops. The biggest problem with AR-the-pattern is that it “frees” the author from thinking at all about what their queries are going to be like and as a result influences table design, service design etc. Discovering that a single logical query is split across three services usually results in giving up on query optimisation and instead putting in a whacking great object cache, buying a big RDS instance and hoping for the best.

      Another time I found somewhere where the team used AR-the-pattern to implement a tree in MySQL - every node as a single row so to get to a leaf you had to make 10s of queries. I can’t remember was done about that but I don’t think it was fixed to anyone’s satisfaction.

      1. 3

        I’ve not written code against the original ActiveRecord library that named the pattern so I wouldn’t know.

        The library is named after the pattern. It’s from Fowler’s Patterns of Enterprise Application Architecture.

        For those that prefer the Data Mapper pattern (where the object has no relationship to the database and is mapped in and mapped out of the database): it’s from the same book.

        I can highly recommend the book, it’s a classic work that collects patterns seen and gives them names.

        1. 1

          Right. I seem to recall dhh having written some opinionated blogs on the AR pattrern in relation to rails - and defending it quite intently - and imnho quite unconvincingly - around rails 1.0 or so.

          Funny thing about graph structures - some colleagues recently did a project with graphs in oracle - but using triggers and built in support (and not rails, but nodejs with some db adapter.. Uncertain if it ended up being AR-like or not).

          I suppose my point is that if you’re using ActiveRecord the library, you end up using the Active Record the pattern… But with a good (if opinionated) query builder that allow you to do “exists” like Author.includes(:books).where( author: {name: name}, book: {language: lang} ).any? for example. And it does allow for reasonable use of relation tables (many-to-many with properties) as well as easy use of (database) views.

          It won’t really help developers without any relational theory avoid shooting themselves in the foot - and (as I can attest) won’t prevent doing too much on the application side (and you highlight very well the two problems: resource use and data safety).

          So I’m not really disagreeing, just pointing out that there is a popular implementation that actually, most of the time, steer clear of the worst limitations.

          (Now, please don’t ask how I feel about built in assumptions about singular/plural forms and automatic pluralization and how that works in a code base written with more than one human language or interacting with legacy schemas in various (natural) language flavours ..)

      2. 6

        However, REST APIs do have one option for mitigation that ORMs don’t: they can vary their endpoints to suit their clients’ needs.

        I’d argue it’s not merely a mitigation technique, it’s how REST is supposed to be done in general. Resources should represent concepts that make sense to client to work with, and not necessarily map 1:1 to the underlying DB schema. Unfortunately architects tend to think of REST as of a semi-automatic way to publish the DB encoded in JSON over HTTP.

        P.S. Also, nobody actually tries to use HATEOAS, so that’s really a non-issue :-)

        1. 11

          The ActiveRecord in this article is a strawman. The real ActiveRecord has offered support for joins, views, and most of what the author is complaining about for years. The troublesome AR pattern, IMO, isn’t its SQL generation but many devs’ tendency to sprinkle AR everywhere – views, controllers, etc.

          1. 8

            Exactly this. I’ve run into active resistance from multiple “architects” who wanted 1:1 correlations between AR models and tables. They didn’t appreciate that AR models are views into databases and that we can create new models to fit the domain as we see fit.

            1. 4

              It’s not a strawman I’m afraid as I’m not referring to that library, which I have never used. If that library has changed it’s tune then so much the better but the pattern is already named - you can address complaints about that to Martin Fowler :)

              1. 5

                I think the issue being taken is that you’re making analysis of ActiveRecord as a pattern without enough experience with its use in a large number of real world codebases and over enough implementations.

                I’m not saying I disagree, but this one is very far from black and white in my opinion. I’ve used ActiveRecord to do magic that wouldn’t have been feasible any other way, and I’ve seen the big footguns too as I tried to steer teams away from them.

                BTW I implemented a tree (well, a forest) for SQL Server without using an ActiveRecord pattern, using a ‘materialised paths’ approach and it worked beautifully. That’s certainly something I can imagine if you weren’t thinking very carefully about design and access patterns for would land you in hot water!

                1. 5

                  I’m afraid there is always someone on the internet who will say “you’re just inexperienced”. Not interesting or a convincing contradiction

                  1. 6

                    By your own admission you never used the library. As mentioned by other in practice pretty much all commonly-used libraries support things like joins so you never have to do that for loop. So yes, I’d argue that some amount of knowledge/experience with actually these libraries would give a more informed opinion. Right now you’re arguing against a theoretical pattern in a theoretical library which doesn’t reflect what people actually do.

                    Fowler’s definition is just a single paragraph, and there is nothing in that which prevents efficient joins or the like.

                    1. 1

                      I haven’t used the library because I don’t know Ruby. There are other languages, there are other libraries, and sadly this is not just a theoretical problem. I’ve seen plenty of code written this way and it is more or less the default - never mind all the people who make “data layers” which “hide the complexity of queries/transactions” as for aesthetic reasons don’t want sesh.commit() in their views.

                      The for-loop example might seem contrived but isn’t uncommon as when it is possible (and easy) to work with objects people naturally gravitate there.

                      1. 4

                        You’re right that people write code like that, I’ve encountered it as well. But I’ve also encountered it with people using regular SQL queries. I’m not so sure there’s a relation between “use a library like this” and “use an inappropriate for loop”.

                        In your post you said that the for loop is a pattern, and this is the part I dispute. I don’t really have strong opinions on the whole SQL queries vs. query builders vs. ORM/ActiveRecord; I’ve used all three extensively and found they all work well but with different pros/cons/tradeoffs (my current app just uses SQL queries btw, and have a very light preference for that as it’s just simpler).

                        1. 1

                          It’s probably worth not getting too hung up on queries in for-loops. That’s just one symptom. As I try to explain in the article the problem is more general: anything that has the main affordance of whole-object basis and key-based access is going to cause major problems. I’m sure there are bad data-access patterns re:SQL but this is the one I see repeatedly.

                          1. 1

                            Okay, fair enough; but it’s quite a substantial part of your post, so IMHO it’s worth addressing.

                  2. 2

                    Everytime someone says “It’s the developers fault for not thinking harder” I’m a little sad. While the statement is true in many ways. It’s also true that when we design API’s and Define Patterns that encourage bad practices it’s also partly our fault. We handed a junior a footgun and then we get all surprised that they shot their foot off.

                    We have the tools at our disposal to create abstractions that “encourage” good behavior from the beginning and enable more advanced usage later. We should stop defending bad abstractions and stop putting all the blame on the people who do the *obvious” thing when given that abstraction for not knowing it was dangerous.

                    1. 1

                      I agree. The problem with making powerful tools is that sometimes it’s not possible to make them easy to use without having problems.

                      I’m not sure there is a huge amount of blame landing on people who don’t know the dangers blowing their feet off with ActiveRecord and similar abstractions. More often I hear developers simply doing the old ‘don’t climb the ladder’ when the memory of the hosepipe is semi-faded.

                      While it seems like we can keep developers from using sharp tools without the required training, unfortunately once word gets out that there are these tools available, and it’s possible to take them off the shelf without paying up front, they’ll get pulled into use quickly and the situation recurs.

                      Seniors in teams can defend against this but can also fuel the fire when they don’t spot or believe the dangers or are overconfident in their teams. If there was such a thing as standardised training for ‘chef’ developers, I’d like dealing with this issue to be a required module.

                  3. 2

                    Then your article makes even less sense. The Active Record pattern specfies precisely nothing about its implementation in SQL or in the host language. All it implies is a 1:1 correspondence between certain database rows and certain domain objects.

                2. 1

                  Nice article! Like (I imagine) many web developers I had to learn all this the hard way, suffering through many problems that didn’t appear until we hit a certain scale (and that generally wasn’t a very high barrier). It’s probably one of the best ways to learn, to have personal experience with the problems these patterns can cause, and discovering how to solve them, but it’s frustrating that most guides completely fail to mention the dangers.

                  1. 3

                    Glad you liked it :)

                    It is depressing that each generation of new developers don’t know the basics of writing database code and has to learn the pitfalls for themselves. Writing database code is a large part of professional work and IMO there should be more b-trees and heaps in coding interviews and fewer hashtables and linked lists

                    1. 1

                      There is a decent amount of educational resources for writing database code available online, but it’s definitely a topic that isn’t very heavily covered in most schools.

                  2. 1

                    I think it would be useful to compile a list of libraries for other languages that are like SQLAlchemy’s core query API and not the active record pattern. For starters, does anyone know if there’s such a library for Java or Kotlin?

                    1. 2

                      I’ve heard good things about jooq but haven’t used it personally. I have used ibatis though and it was basically a thin layer over SQL which worked ok.