1. 18

  2. 5

    An architecture after my own heart.

    One of the most influential works on architecture I have ever encountered is an old talk from Ruby Midwest 2011. Not just in its amount of content (though of course books, being books, have more, e.g. PoEAA), but in the sheer focused force of a few salient points, and quotable quotes, which then drove me to explore further. Chief among them, and which snapped me-circa-2012 back to reality, was the thought “Huh, why are the folders in /app grouped by some pattern name, unrelated to our business domain? No other job I’ve had did that…”

    I gave it a shot twice with my current job’s main monolith. One failed, the other half-succeeded: we gained a more expressive structure than just “MVC”, and we aggressively section off library code into internal gems, but we never truly freed our domain from the tyranny of /app.

    What wound up happening was sequestering sub-sections of the domain that “felt’ like they would one day be broken away into subfolders of /app/services, each representing a self-contained mini-domain. Consequently it means we never fell victim to equally-all-encompassing “service objects” because that folder name was already taken. Our boundary “enforcement” was discipline and code review, and at the cost of a few layers of abstraction. Somewhere along the way, we also stumbled onto what you call an Observer Pattern, but we just call “messaging” (via the guess that it would break off on its own, into RabbitMQ or Kafka, which it has).

    The big difference is that our tests for these are not as disciplined. The domain logic might not be coupled, but the tests wound up being so. Tearing a service’s code out is easy. Getting its tests to pass again once isolated is inappropriately hard.

    It seems like both of us driving at similar goals led to mostly similar decisions! The only difference is we may never break the bondage of /app in that monolith. By the time we gained the engineering headcount and roadmap slack to attempt that, various data science and marketing needs had driven us to at last split the services out, one by one, rather than fight Rails while molding them into a more disciplined structure.

    1. 0

      Is it me, or does a 2x test/code ratio feel like a hilarious waste of time, or not nearly enough?

      1. 2

        I thought we had a normal ratio for a Ruby/Rails application that has thorough test coverage, but I’d love to hear from others on what their ratio looks like. We didn’t aim to have a certain ratio or anything. We write tests as we’re writing code, and that’s what we ended up with. Lines of code isn’t a consistent metric across different projects since style and conventions come into play, but I thought it was something I could share to convey the size of our application.

        1. 2

          The more consistent our architecture became, the more we were able to leverage integration tests, i.e. end to end assertions of the public side effects resulting from a single use case, and have confidence in that use case’s correctness. The average use case simply had far less “exciting” code, relied more on libraries, and far more that could be taken for granted. Unit tests are emphasized in our library code (e.g. internal gems) or in anything “interesting”, which is taken to mean any logic hairy enough that it doesn’t just fall out of air with our bog-standard architecture.

          Certain policy and permission classes, many of which boil down to a comparison operator or two, are also unit tested by default in pursuit of “tests as documentation” rather than assertion of correctness.

          I haven’t run the stats in a while but if you ignore view code (any .erb files, css and our javascript whose tests are nonexistent or in shambles), we are sitting at 1:1 or bit lower in our main monolith. In my conversations with others, I would not say that your 2:1 is out of the ordinary, though anything higher might make me raise an eyebrow.

          At some point if you’re being so thorough that you have excess of a 2:1 test:domain code ratio, you either have a very hairy, naturally complex business domain (and my eyebrow lowers), or you should look into property testing/generative testing.

        2. 1

          I think it depends tremendously on how you write tests.

          EG: If your tests are mostly high-level integration tests it seems quite high; if they’re mostly low-level unit tests it seems low.

          Similarly, If you’re using a terse style (eg via DSL / metaprogramming) it seems high; if you’re using a verbose style (eg the recommended rspec approach) I’d say you’ve mostly tested the ‘happy path’.

          1. 1

            Right, that’s my point. I also was speaking in general, not specifically about a Rails application. I don’t do Rails any longer, but when I did, the codebase was more like 5-8x tests to codes. And of course, in Scala, the type system – for all of my reservations about complexity – allows a much leaner ratio.

          2. 1

            If you’re really serious about testing, especially more end-to-end stuff and interacting with the front-end, it’s pretty easy to end up needing 10 lines in a test to check a feature that only required 2 or 3 to implement.

            There’s some cross-checking too of course, but if writing 3x as much code meant basically no regressions, that’s a decent deal

            1. 1

              I depends a lot on the code. For example, in my current codebase I have something like:

              Maybe(resource).fmap(&:author).fmap( ->(n) { n[:full_name] }).or(Some('Somebody'))

              This is part of a method that gets a hash that, ideally, looks like this:

                author: {
                  full_name: 'John Doe'

              However, there might be some cases where the hash is nil or the hash does not contain the expected keys. So for a line (actually two, since the line is too long for my taste) of code you easily end up with four test cases (hash is nil, hash does not have the :author key, the :author key returns a hash without the :full_name key and the ideal scenario where all the data is present).

              Then again, you most likely have a bunch of lines of code that are just doing simple things, like check if a property is set or something like that, where the test cases are a lot simpler so you might not end up with a 2x test/code ratio.

              1. 2

                I have no idea what your domain/code-culture is, but if you just want something short, maybe plain Ruby is enough? :

                Hash(resource).dig(:author, :full_name) || 'Somebody'

                Your short code example looks like a mix of Ruby with Rust, or Haskell monads. Yet, I wonder what happens when resource is an Array. Does that Maybe function swallow the exception? It’s hard to bolt on types where there were none before! :)

                1. 2

                  The library used is dry-monads and if you pass an array you get an NoMethodError: undefined methodauthor’`.

                  I agree that the dig method is more appropriate for a Ruby codebase and in some places it was used instead of the Maybe monad. The reason why we’re using that (and I was the one pushing it as team lead) was that those constructs are closer to constructs in other languages and one of the side goals that I have is to enable people as much as possible to explore other languages and my feeling is that this kind of code helps.