1. 0

    TL;DR: “uh-oh, a latency-sensible service shouldn’t be written in a garbage-collected language”.

    1. 14

      Perhaps depends on the language though - Erlang is used in what I assume to be latency-sensitive applications like telephone switches, and it’s GC’d.

      1. 1

        Erlang is a special case with per-process tiny heaps that are collected independently of one another. I’m not aware of another system that uses such a model.

        There are fully concurrent GCs in experimental and proprietary JVMs (Azul and Shenandoah) that exhibit remarkably low pause times even with large heaps. They are very much the exception rather than the rule and to my knowledge require a throughput hit compared to standard VMs or manual memory management.

      2. 2

        Oh, that explains why no one writes such services in Java or JavaScript or C# or PHP or Perl or Python or Ruby or Erlang or LISP or Haskell … only C and C++. And Pascal.

      1. 18

        I get where he’s coming from, but I think the really valuable thing about microservices is the people-level organizational scaling that they allow. The benefits from having small teams own well-defined (in terms of their API) pieces of functionality are pretty high. It reduces the communication required between teams, allowing each team to move at something more closely approaching “full speed”. You could, of course, do this with a monolith as well, but that requires much more of the discipline to which the author refers.

        1. 15

          Why do you need network boundaries to introduce this separation? Why not have different teams work on different libraries, and then have a single application that ties those libraries together?

          1. 4

            Network boundaries easily allow teams to release independently, which is key to scaling the number of people working on the application.

            1. 1

              Network boundaries easily allow teams to release independently

              …Are you sure?

              Independent systems are surely easier to release independently, but that’s only because they’re independent.

              I think the whole point of a “microservice architecture” is that it’s one system with its multiple components spread across multiple smaller interdependent systems.

              which is key to scaling the number of people working on the application

              What kind of scale are we talking?

              1. 1

                Scaling by adding more people and teams to create and maintain the application.

                1. 2

                  Sorry, I worded that question ambiguously (although, the passage I quoted already had “number of people” in it). Let me try again.

                  At what number of people writing code should an organisation switch to a microservices architecture?

                  1. 1

                    That’s a great question. There are anecdotes of teams with 100s of people making a monolith work (Etsy for a long time IIRC), so probably more than you’d think.

                    I’ve experienced a few painful symptoms of when monoliths were getting too big: individuals or teams locking large areas of code for some time because they were afraid to make changes in parallel, “big” releases taking days and requiring code freezes on the whole code base, difficulty testing and debugging problems on release.

                2. 1

                  I think the whole point of a “microservice architecture” is that it’s one system with its multiple components spread across multiple smaller interdependent systems.

                  While this is often the reality, it misses the aspirational goal of microservices.

                  The ideal of software design is “small pieces, loosely joined”. This ideal is hard to attain. The value of microservices is that they provide guardrails to help keep sub-systems loosely joined. They aren’t sufficient by themselves, but they seem to nudge us in the right, more independent direction a lot of the time.

              2. 2

                Thinking about this more was interesting.

                A [micro]service is really just an interface with a mutually agreed upon protocol. The advantage is the code is siloed off, which is significant in a political context: All changes have to occur on the table, so to speak. To me, this is the most compelling explanation for their popularity: they support the larger political context that they operate in.

                There may be technical advantages, but I regard those as secondary. Never disregard the context under which most industrial programming is done. It is also weird that orgs have to erect barriers to prevent different teams from messing with each other’s code.

              3. 12

                The most common pattern of failure I’ve seen occurs in two steps:

                a) A team owns two interacting microservices, and the interface is poor. Neither side works well with anyone but the other.

                b) A reorg happens, and the two interacting microservices are distributed to different teams.

                Repeat this enough times, and all microservices will eventually have crappy interfaces designed purely for the needs of their customers at one point in time.

                To avoid this it seems to me microservices have to start out designed for multiple clients. But multiple clients usually won’t exist at the start. How do y’all avoid this? Is it by eliminating one of the pieces above, or some sort of compensatory maneuver?

                1. 4

                  crappy interfaces designed purely for the needs of their customers…

                  I’m not sure by which calculus you would consider these “crappy”. If they are hard to extend towards new customer needs, then I would agree with you. This problem is inherent in all API design, though microservice architecture forces you to do more of that thus you could get it wrong more often.

                  Our clients tend to be auto-generated from API definition files. We can generate various language-specific clients based on these definitions and these are published out to consumers for them to pick up as part of regular updates. This makes API changes somewhat less problematic than at other organizations, though they are by no means all entirely automated.

                  …at one point in time

                  This indicates to me that management has not recognized the need to keep these things up to date as time goes by. Monolith vs. microservices doesn’t really matter if management will not invest in keeping the lights on with respect to operational excellence and speed of delivery. tl;dr if you’re seeing this, you’ve got bigger problems.

                  1. 1

                    Thanks! I’d just like clarification on one point:

                    …management has not recognized the need to keep these things up to date as time goes by…

                    By “keeping up to date” you mean changing APIs in incompatible ways when necessary, and requiring clients to upgrade?


                    crappy interfaces designed purely for the needs of their customers…

                    I’m not sure by which calculus you would consider these “crappy”. If they are hard to extend towards new customer needs, then I would agree with you.

                    Yeah. Most often this is because the first version of an interface is tightly coupled to an implementation. It accidentally leaks implementation details and so on.

                    1. 1

                      By “keeping up to date” you mean changing APIs in incompatible ways when necessary, and requiring clients to upgrade?

                      Both, though the latter is much more frequent in my experience. More on the former below.

                      Yeah. Most often this is because the first version of an interface is tightly coupled to an implementation. It accidentally leaks implementation details and so on.

                      I would tend to agree with this, but I’ve found that this problem mostly solves itself if the APIs get sufficient traction with customers. As you scale the system, you find these kinds of incongruities and you either fix the underlying implementation in an API-compatible way or you introduce new APIs and sunset the old ones. All I was trying to say earlier is, if that’s not happening, then either a) the API hasn’t received sufficient customer interest and probably doesn’t require further investment, or b) management isn’t prioritizing this kind of work. The latter may be reasonable for periods of time, e.g. prioritizing delivery of major new functionality that will generate new customer interest, but can’t be sustained forever if your service is really experiencing customer-driven growth.

                      1. 1

                        Isn’t now the problem moved to the ops team, which had to grow in size in order to support the deployment of all these services, as they need to ensure that compatible versions talk to compatible versions, if that is even possible? What I found the most problematic with any microservices deployment is that ops teams suffer more, new roles are needed just to coordinate all these “independent”, small teams of developers, for the sake of reducing the burden on the programmers. One can implement pretty neat monoliths.

                        1. 2

                          We don’t have dedicated “ops” teams. The developers that write the code also run the service. Thus, the incentives for keeping this stuff working in the field are aligned.

                2. 6

                  It reduces the communication required between teams, allowing each team to move at something more closely approaching “full speed”.

                  Unfortunately, this has not been my experience. Instead, I’ve experienced random parts of the system failing because someone changed something and didn’t tell our team. CI would usually give a false sense of “success” because everyone’s microservice would pass their own CI pipeline.

                  I don’t have a ton of experience with monoliths, but in a past project, I do remember it was nice just being able to call a function and not have to worry about network instability. Deploying just 1 thing, instead of N things and having to worry about service discovery was also nicer. Granted, I’m not sure how this works at super massive scale, but at small to medium scale it seems nice.

                  1. 2

                    Can you give an example where this really worked out this way for you. These are all the benefits one is supposed to have, but the reality often looks different in my experience

                    1. 10

                      I work at AWS, where this has worked quite well.

                      1. 5

                        Y’all’s entire business model is effectively shipping microservices, though, right? So that kinda makes sense.

                        1. 20

                          We ship services, not microservices. Microservices are the individual components that make up a full service. The service is the thing that satisfies a customer need, whereas microservices do not do so on their own. Comprising a single service, there can be anywhere from a handful of microservices up to several hundred, but they all serve to power a coherent unit of customer value.

                          1. 4

                            Thank you for your explanation!

                        2. 4

                          AWS might be the only place I’ve heard of which has really, truly nailed this approach. I have always wondered - do most teams bill each other for use of their services?

                          1. 5

                            They do, yes. I would highly recommend that approach to others, as well. Without that financial pressure, its way too easy to build in some profligate waste into your systems.

                            1. 1

                              I think that might be the biggest difference.

                              I’d almost say ‘we have many products, often small, and we often use our own products’ rather than ‘we use microservices’. The latter, to me, implies that the separation stops at code - but from what I’ve read it runs through the entire business at AMZ.

                            2. 1

                              It’s worked well at Google too for many years but we also have a monorepo that makes it possible to update all client code when we make API changes.

                      1. 2

                        I think ‘services’ in any form are the wrong product of programming. The real purpose of any system is larger computational processes (the ‘business logic’) that span multiple services and systems - frontends, middleware, databases. Currently, each of these contains duplicated and overlapping parts of the system wide model definition. We have to keep each of these definitions in sync, mentally keep track of the correspondence, manually hand hold any changes across the system while ensuring compatibility, etc.

                        I would like to see a topology independent programming model. What I mean is a model where we can define system wide processes without tying it to the implementation topology. This definition would then be mapped to the implementation topology via additional descriptions.

                        Consider how compilers freed our code from being tied to the CPU topology (we don’t reference register names, load and store operations in our code, but make up variable names closer to the business logic, that get mapped to the underlying topology). Similarly we want to be free of writing business logic tied to the system topology. If I’m writing a messaging app, my highest level code should look like room.append(message). Then separately I’d map the room object and the message object to various projections across the services and databases.

                        Does something like this exist?

                        1. 2

                          A topology independent model was part of earlier generations of RPC, like NFS, DCOM, CORBA. Just make all your OO methods accessible over the network and go! Unfortunately this ignores the latency of RPCs, which require API redesign. It also prevents us from making RPC interfaces reliable by hiding state in the server that is deleted on restart or upgrade.

                          Does it help to realise that services can still share client libraries built on interface descriptions like protobuf?

                          1. 1

                            If you have the choice between an object being fully local or fully remote on another machine, with method calls becoming network messages, I don’t think that is topology independent. Your topology will still factor into how you design these object APIs and where you put them. I’m aware of CORBA, DCOM and also GemStone, but I’m talking about something different.

                            Consider the conceptual room object in the messaging app. Where does it live? A part of it is in the client, part of it is in some servers and another part of it is persisted in the database. It’s topologically fragmented but conceptually a single object. The parts of this object are not even identical replicated copies - each one overlaps with another but has more to it. Conceptually the state of the room is the union of the state of the distributed fragments.

                            AFAIK none of the mentioned technologies can let you declare and work with a single object that is implemented in this fragmented fashion. The idea is that you’d treat this as a single object and define various processes and attributes for it. Separately you’d define how the attributes and processes can be decomposed and mapped onto the topology.

                            1. 1

                              Let’s say each of your fragments is one service in a microservices architecture. The advantage is each service can be developed, deployed, debugged independently in parallel by different teams.

                              If you couple all the fragments again in a single codebase (albeit one with projections as you describe) isn’t that no longer possible? Or at best you need to be sure not to edit code outside your team’s fragment because you could break another team’s service. Separate repositories for separate teams is an obvious way to divide responsibilities.

                              1. 1

                                Good questions - these are open questions that would need to be resolved as part of refining the programming and deployment model. I’ll phrase the questions as “how does ownership work?” and “what is a deployable unit?” and speculate on them below.

                                First I want to point out that even in a microservice world, the owner of a shared library may update it at one point in time but it will end up going live at different times in different services based on when they rebuild and push. The library owner may have no control over the deployment here.

                                Secondly I think it’s a fallacy that services are ‘independent’ - a downstream or upstream service push may break a service due to interdependencies, assumptions and sleeping bugs that got activated due to a new usage pattern. Still, in principle the API and SLA encapsulate what a service provides and ownership corresponds to a ‘service’ that satisfies these.

                                In a topo-free-model world, I imagine services will be fluid enough that they would not be a unit of ownership. For instance, there are multiple alternative service topologies that might implement the same topo-free-model. Consider two equivalent topologies, A-B-C-D and A-E-D, where the E service implements the functionality of both services B and C. Now transitioning from ABCD to AED may be done because it has better performance characteristics. Importantly, there would be no change to the topo-free-model definition of the code during this transition. Only the mapping to the topology would change. I imagine this could be rolled out by a single team that owns this subset (B-C) of the topology. Ownership of the topo-free-model itself would be distributed along the business model. I imagine a large object such as User might itself be sliced up and owned by multiple teams - depending on business function. In the end, Conway’s law will still be in effect so the org structure would mirror the topo-free-model to some degree.

                        1. 2

                          TL;DR: Use VirtualBox, then run:

                          VBoxManage guestcontrol $GUEST_NAME $CMD $ARG1 … –username $GUEST_USER –password $GUEST_PASSWD

                          1. 5

                            Perhaps a good fit for this web of documents would be IPFS (or any similar technology), each document is immutable and the referencing is based on the content itself (newer versions would have different hashes or in other words would be a different document).

                            I think it checks all the boxes described on the post, but I might be missing something.

                            1. 2

                              Freenet follows the web of documents idea closely: just immutable HTML and CSS, no JavaScript.

                              1. 2

                                It’s been a while since I looked at Freenet. Can one access content there via a “normal” web browser and URI?

                                1. 2

                                  You start a daemon and then can access pages from a normal browser using URIs like http://localhost:1234/pages/${somehash}.html

                                  Most (all?) writes require a Freenet plugin or separate application to upload content. For example there are apps for mail, a message board, and to upload a site. Of course this seems odd because the web has evolved Turing-complete servers and clients, but it puts Freenet in an interesting niche because it has neither.

                                2. 1

                                  I know next to nothing about freenet, but browsed some of the documentation now.

                                  Is there a starting point for someone who wants something line the old web? Just plain olf websites with all kinds of weird and sometimes useful information?

                              1. 4

                                Problem for me is I don’t wanna have to open access to my home network, and neither do I wanna be responsible for administering a system on the public internet, it’s just too much work to be done securely unless that’s your full time job.

                                1. 2

                                  I agree that it’s a large liability to run a system on the internet. I can imagine running an exposed VM with no private data on it (just public parts of keys for authentication) and then at least you can’t lose any data. Of course someone could hijack it and mine bitcoin or serve something nasty under your name. Maybe this is a good use case for a Unikernel.

                                  I would love to see a pre-hardened linux VM image with some auditing best practices that you could just put online.

                                  1. 2

                                    Totally agree. That’s why I hold the unpopular opinion that managed services where you do have access to all your data written into the contract are just fine and dandy for those who aren’t willing to undertake the part time job of managing infrastructure on the internet.

                                    I use Wordpress.com and love them. They wrote “your data is always yours” right into the agreement you ‘sign’ when you join, it’s inexpensive, and they have all the features I want in a blogging engine.