1. 35

    1. 6

      I enjoyed the post! Genuine question: is keeping all copies of all ingredients around the ‘best strategy’ for playing factorio, as factories get orders of magnitude bigger? What are the ‘best strategies’, and how do these concepts map to the process of programming?

      The largest challenge when first playing Factorio is building a factory without knowing how new technologies will change the game later. Conflicting constraints eventually lead to a required ‘refactoring’ of the factory. In both programming and Factorio, how can one balance exploitation and extensibility such that large systems can expand without becoming ossified due to a lack of foresight? What changes once you’ve already played through the game once?

      1. 16

        Oh man this is a great post, yes. I’m an avid player of Factorio and games like it, when I have the brain-bandwidth and free time, so I can try to answer your question. Long story short, there is no single “best strategy”, which is part of what makes Factorio and similar games fun. You have to experiment and figure out the strategy that fits the bill!

        The two constants of Factorio are that you will never be producing enough of a particular piece of stuff, and you will never have enough raw material. You constantly build new things to expand your capabilities, and then find new bottlenecks and expand your operations to free them up. Which will eventually result in a different bottleneck, but then you expand your operations to solve that, and then you get a new tech that lets you produce something new that either keeps you alive or solves a logistical problem you have, etc etc.

        So with that in mind:

        • No, keeping all ingredients around will not scale well. You can see that in the screenshots of the factories in the article: they end up with like 12 belts of ingredients, as wide as the rest of the factory put together. Belts cannot easily cross: you have to build underground belts, which can only go in straight lines for a limited distance and which take up space at their endpoints. Splitting or merging belts is easier but still takes up space. So if you keep all your intermediate ingredients around, you will end up with lots and lots of time and space devoted to logistics spaghetti, most of which is unnecessary and all of which will be very difficult to change when you suddenly need to, because…
        • Belts also have limited throughput. As the game goes on you will get faster belts (which take more advanced materials to build), but when your automation starts really ramping up around midgame it is pretty easy to produce and consume an entire belt’s throughput of, say, green circuits, which then requires multiple belts worth of input materials to produce. So you can scale in two ways, “vertically” or “horizontally”. To scale “vertically” you have a single belt of inputs “A” with multiple factories taking stuff off of it and putting their products “B” onto a single output belt, which is very easy and efficient and can only go so far until you’re literally consuming everything a single belt can carry. So once you reach that point you have to scale “horizontally” and make multiple parallel production lines, and figure out how to get all their outputs to where they need to be. Which then becomes harder, because…
        • Factories take up physical space. You need to leave yourself room for expansion, often in unexpected directions. You also have to go find your physical inputs which may be far away or in dangerous places, and you have to actually be able to build your factory when there are inconvenient (or useful) lakes or cliffs in the way. You can get rid of lakes and cliffs, or you can put the pieces of your factories further away from each other, but you have to use up stuff to do either of those things, which means building more factories to produce that stuff.
        • Usually you end up with broad two types of products: stuff that you need steady supplies of of but which are difficult to use up faster than you can produce them (at least until late game), such as railroad locomotives, belts, power lines, solar panels, etc, and stuff that you can never have enough of, such as research items, iron plates, electronic circuits, concrete, etc.

        This leads to a lot of lessons that will sound fairly familiar to lots of programmers:

        • You have tradeoffs between efficiency and flexibility. If everything was designed perfectly up front and you had unlimited basic resources to work with, it’s pretty easy to design things to be scaled up to infinity. This is often what the lategame looks like, giant grids of factories that take in fairly fixed inputs and produce fairly fixed outputs. (Often measured in “spaceships per minute” by the people who get really into it.) But trying to do that right from the start will leave you hamstrung, as the game progresses what you need will change so you have to be efficient, but also adapt to circumstances.
        • As /u/kornel says, having a main bus is not the perfect strategy, but is pretty useful for most cases… because it is adaptable. It is quite easy to add more factories branching off of a bus just by making the bus longer, and it is quite easy to add more products to a bus by making it wider – though as I said making it wider has costs too, so you want to make sure you use it for the most useful and general products, perhaps making other things on-site when they are needed. If you know that you will need it, you can also leave yourself some extra space on a bus for improving throughput, such as running multiple belts of the same material or having a “feeder” line coming in from the side that replenishes a depleted resource. So you end up with many loosely coupled components that are joined by this shared interface layer.
        • As your factory grows larger, you will end up with multiple sub-factories in various places, often linked by long belts or more commonly railroads, which do the same basic job but have higher throughputs over long distance (at the cost of higher latency, being more work to set up and manage, and needing more space). These usually end up specialized in certain things, whether it’s as simple as mining iron and producing plates and gears that are shipped back home, or as complex as an entire petroleum plant that results in half a dozen different products in varying amounts, some of which can be partially substituted for others. But because of all the interlocking products and changing supplies and demands, making a truly distributed point-to-point system is very difficult. It’s much easier to keep all your general-purpose continually growing production stuff centralized in one place, and make the offshoot factories specialize, because then you don’t need to get much stuff to your offshoot factories. (Hmm, that reminds me a bit of colonial mercantilism; something to ponder someday.) So you end up with a hierarchical system with a large complicated core interacting with many smaller, simple systems through more special-purpose interfaces.
        • Relatedly, if the transport costs become too inefficient for certain places (usually expressed as them turning into belt-spaghetti), it can be worth it to modularize and specialize. For example if you have a process that consumes lots of steel, you can make an entire sub-factory that takes raw materials (coal, iron ore, power) and turns it directly into steel to feed this one process, and give it its own supply lines of those inputs instead of having it connect to your main bus. Knowing when and where to do this takes some judgement and experience, since for more complicated things it is difficult to wrap things up into a single neat “package” design that produces exactly what you need with no weird hairy edge cases, but if done well you can get a nice reusable, efficient module that is easily slotted into larger designs via a more abstract interface (put in four belts of iron and two belt of coal, get out one belt of steel).
        • Eventually you get flying logistics robots that can move arbitrary stuff from point A to point B, automatically emptying “provider” containers and filling “requester” ones… but until you start reaching the endgame they are slow, expensive and can’t carry very much, so are best used for low-volume, high-value things. Just like the real world, I suppose. True point-to-point mesh networks are great, but have high overhead, and are still outperformed in throughput by more specialized systems.

        You will always have to refactor(y) your setup multiple times, and that’s fine. The state of the game world is too big to hold in your head all at once unless you’re some kind of savant, and there’s plenty of unknowns to trip you up, and many interlocking design decisions that can work out equally well in the end. Experience results in you knowing more about how the different parts of these systems interact, so you have a better feel of what to expect and how things slot together, but no plan is ever perfect unless you’re one of the really white-knuckled players who literally pave over half the game world and do the math to optimize everything perfectly. But I never have the patience for that sort of thing.

        So… Um. Yeah. Factorio is pretty cool.

        1. 1

          Thanks for the comprehensive reply! I should really play some Factorio haha

          I think your proposed tradeoff between flexibility and efficiency is spot on. Ideally, one should start off with maximal flexibility, and ‘gradually cast systems in stone’, optimizing the worn parts of the system for efficiency. See also Functional Core, Imperative Shell type architectures.

          1. 8

            I should really play some Factorio haha

            Sorry, “some” is not an amount of Factorio one can play. The options are “none” and “way too much”.

            1. 1

              Instructions unclear, sat down on a rainy day to play “some” factorio. Now my girl is gone, my kid is grown, and I’m an old man. What year is it?

      2. 5

        What changes once you’ve already played through the game once?

        The 2 biggest changes in my gameplay came when I watched Nefrums do a speed run, and when I got the Lazy Bastard achievement.

        Seeing how someone truly skilled built their factory gave me lots of ideas about what kinds of things were possible or optimal. Using inserters to put coal on a forge belt and using red inserters to take ingredients from 2 parallel belts 2 tiles away from the assembler both seem obvious in retrospect, but I never did those things before. Those are small examples of design minutia, the real value was seeing how Nefrums approached the game. How he built his factory and expanded his resource collection in waves. His priorities and milestones that he targeted deliberately one by one. Studying prior art is not unique to Factorio or programming, but I will say that watching Nefrums, an expert, was vastly more valuable to me than any of the tips or designs from amateurs I found on Reddit or the forums.

        In doing Lazy Bastard (hand craft no more than 111 items) I learned how incredibly ridiculously convenient it is to have a mall (area of your factory where building materials are pre-crafted and stored). I thought a Lazy Bastard run would be a pain, but it really wasn’t at all. Now I never play the game without an extensive mall based on the one I made for Lazy Bastard. This mirrors my feelings about automated tests and inline asserts. The first time I used asserts in a simple storage engine for a DB class in college, over half of them fired at some point or another during development, and I was enlightened.

        In both programming and Factorio, how can one balance exploitation and extensibility such that large systems can expand without becoming ossified due to a lack of foresight?

        My attempt at the “There is no spoon” achievement actually does mirror some of my professional experience, as I had a constraint for myself that I would launch the rocket in 8 hours with a sustainable factory. That is, a factory that I could continue the game with after launching the rocket, rather than throw away after getting the achievement. And I have continued that factory, using it to get all the other achievements. I also required myself to use 100% nuclear power by the time my starting coal ran out. My experiences:

        • Proof of concept: my first attempt was well over the time limit (20+ hours) for either speed run achievement, but I focused on exploring the kind of factory layout I wanted to use for my next attempts once I realized I was not going to make it.
        • Design review: after the first attempt I spent some time in creative mode reviewing my design and tweaking parts of it for my second attempt. I also consulted my brother for his thoughts on some of my designs.
        • Project planning, task breakdown: after my second attempt (~14.5 hours) I spent more time in creative mode making my factory better suited to incremental improvement throughout my next run.
        • Future proofing: my original factories were extremely tight and optimized, until I needed to snake an ingredient through a place I didn’t expect, and there was literally not a single tile of space to do it. From then on I left 4 tiles of empty space between components, even if I spent lots of time designing those components and their neighbors ahead of time and theoretically didn’t need the space.
        • Avoid large rewrites: my first attempt factory was loosely based on the factory I built for my Lazy Bastard run. For my later attempts I avoided redesigning my factory from scratch even though I had plenty of new ideas. My goal was to make the necessary improvements to succeed on my next attempt, not start over completely.
        • Scaling pains: I succeeded on my third attempt (~7.5 hours), but there were still complications in expanding the factory after the rocket launch. I had planned my factory ratios in a way that could be upgraded with modules post-launch, but I had to spend a lot of time revisiting areas of the factory to address unforeseen throughput issues.
        • Monolith vs microservices: I quickly realized that trains were impractical for an 8 hour run due to the up-front cost of building a truly scalable train network. My factory had isolated sections for different components, but was still basically a single integrated entity. Only after the launch and upgrades / maintenance to my original “monolith” factory did I invest in a train network. I kept the original factory running the entire time that I developed external “microservices” for oil refining, smelting ore, mass chip manufacturing, and so on. If I had committed to building a train network for the 8 hour rocket launch, I don’t think it would have scaled well in the long term, and I would have needed to redesign it anyway.

        is keeping all copies of all ingredients around the ‘best strategy’ for playing factorio, as factories get orders of magnitude bigger?

        For my core factory to launch a rocket, I absolutely did not do this. I recreated several intermediates in different places as they were needed. For example, snaking ~1 assembler of gears from my red science all the way to engine units for blue science seemed unnecessary. It was far simpler to just create more gears alongside the pipes next to the engine unit assemblers. I only snaked intermediates that were expensive or complex to make, or highly unrelated to anything else nearby: red chips, blue chips, sulfur, batteries, plastic, and low-density structures. But I did make an effort to place consumers of these things close together where possible. I despise the “main bus” approach to factory building.

        My post-launch “microservices” never completely did this either. I connected complex or high-volume intermediates to the network, but constructed simple, low-volume, or single use things inline. I made trains for the same things I modularized before, as well as including fluids that require water to produce. I never had trains for copper wire, gears, pipes, rods, engine units, robot frames, explosives, and so on. I shipped raw uranium ore to nuclear power plants, and handled all processing on site.

        how do these concepts map to the process of programming?

        I don’t think my approach is necessarily the best way to play Factorio, but it definitely reflects my values in software engineering. I believe that components should be modular with well defined boundaries, but microservices are a heavy-handed tool that should be used sparingly. That hyper-generalization is overrated, and specialization of mundane things can actually make systems simpler and reduce maintenance costs. That systems should be designed up front as much as is practical, while leaving room for incremental improvement or changing requirements down the line.

        1. 2

          ahahaha you posted this right as I finished my own novel-length reply, and I love hearing about how different your design philosophy is! “I despise the main bus approach” made me really stop and think. XD Do you have any good sources besides Nefrums?

          1. 2

            I think the main bus is reasonable-ish if you have no idea what’s coming next. But everything doesn’t need everything else, so it’s not that hard to put similar things together, especially once you’ve launched a rocket before. Chips are the best example, my 8 hour factory is divided by a large vertical mall, chips on the left, and everything else on the right. They don’t have any codependencies, they don’t need to be together for any reason. I don’t even belt green chips to the right half, only red and blue. The entire right half is served by a few green chip factories on that side.

            I don’t actually like videos or streaming as a medium for learning. At first I learned by playing with my brother and a friend, who both had 500+ hours before I even started. Only after launching a rocket multiplayer did I really look into improving my game.

            My brother is into the speed run community so when I asked about Factorio pros he pointed me to Nefrums and AntiElitz, the 2 top Factorio speed runners. I like Nefrums a lot more because he spends more time explaining his decisions and teaching on his stream. But once I got inspiration on common design elements and an understanding of how an expert views the progression of the game, I stopped watching. From there I made and tested my own designs in creative mode. I didn’t copy any of his designs directly, except for forges because there really is just a one true best way to make forge lines and nothing to improve about it.

            For certain things that are more complicated like train loading/unloading, intersections, beaconed factories, and balancers, I looked around online for people who had done exceptional work in those areas.

            Train loading/unloading should be done evenly for each car in the train and there is no one “best” solution. I sought out all the existing approaches so I could synthesize a design I am satisfied with. I don’t have any good resources for this, I mostly looked at a lot mediocre designs to get an understanding of what approaches people have done before. Though for what it’s worth you can get an optimally balanced design with the / -CHESTS circuit approach without much effort. I just found that unsatisfying.

            There is an insanely thorough benchmark of train intersections on the forums that I used for inspiration before designing my own intersections. I made something similar to the super compact Celtic knot, that’s a little more compact and has 4 tiles between the rails instead of 6.

            For beaconed factories, flame_Sla on Reddit has posted saves of 10k SPM designs and beyond. I found those valuable to study, I learned a lot about beacon layout and direct insertion. You can make much more compact designs if you cut belts out of the equation entirely.

            Balancers in particular are an uninteresting problem (to me) solved by The Balancer Book, compiled from a combination of prior art and a SAT solver.

            In summary, I spent a lot of time in creative mode, studying prior art for anything that seemed nuanced enough to warrant it. Every component of my factory has gone through several iterations in creative mode, been put to the test in a real run, and refined from there.

            1. 4

              Thanks a lot! I think this conversation honestly is a pretty good demonstration of the mindset of “mid-tier player” vs “advanced player”. My answer is “design for flexibility and adaptability because you can’t keep everything in your head, you’re going to have to change things no matter what” and yours is “I understand I’m going to have these specific design rules and patterns, so I will build around them and it will be more efficient without needing as much refactoring.”

              It also reflects technical mindset: I love figuring out how to put things together from nothing and exploring new problems, but doubling-down on it and putting lots of work into understanding the details and how to apply them is less interesting to me. It sounds like you’re more dedicated to optimization and refinement than I usually am, and that’s resulted in you having a deeper understanding of how pieces interlock. I’m usually done with a factory after it’s launching rockets semi-consistently, going back to expand and optimize it has limited appeal.

              Didn’t know I had that much to learn, but I have a few things I’m interested in checking out now!

        2. 2

          Now I really need to play Factorio sometime!

          I like how you emphasized the social attribute of systems design. There’s a lot you can learn on your own from your first factory. Building a megafactory, on the other hand, requires goals, community, planning, and coordination. The same is true of programming: no system is designed in a vacuum, and tradeoffs must be discovered, considered, and decided upon. This happens a lot faster if you can learn from others.

          I enjoyed hearing how the design of your factories changed as you the constraints and resulting tradeoffs were determined for a particular run.

      3. 5

        All is probably an overkill, but this strategy in general sounds like https://wiki.factorio.com/Tutorial:Main_bus

      4. 3

        I enjoyed the post! Genuine question: is keeping all copies of all ingredients around the ‘best strategy’ for playing factorio, as factories get orders of magnitude bigger? What are the ‘best strategies’, and how do these concepts map to the process of programming?

        No, because plenty of ingredients aren’t dense enough to ship - for instance, it’s basically never worth putting copper cables on your main belt because 1 copper plate = 2 cables, so you’re better off just making your ‘copper cable belt’ carry plates instead and then crafting cables locally (and it crafts in a second or less, unlike e.g. engines, which are basically just iron+steel but take (20?) seconds each to craft)

      5. 2

        is keeping all copies of all ingredients around the ‘best strategy’ for playing factorio, as factories get orders of magnitude bigger?

        “Megabases” that scale up factories tend to be built on top of train networks. They often use a mod like LTN or Cybersyn to route deliveries, though it can be done with the game’s base mechanics with somewhat more work.

        I’ve often seen “city block” designs described as being analogus to serivce-oriented architecture, though I’m not sure it’s a perfect analogy. Each block tends to be responsible for a single thing, requesting a small number of inputs and providing a single or small number of outputs.

        I’m not sure there’s a single “best” design or set of principles for such designs. How strictly do you enforce blocks doing a single thing — are they restricted to a single recipe, or can they make some precursors? Do you optimise for the lowest number of inputs and outputs, or for speed and scale? Do you scale by improving a city block, or just copying it (like improving performance compared to running more instances of a service)?

        Something that’s kept me coming back to Factorio is that there isn’t a single optimal way to play that makes up a meta that all players follow. While there are some common patterns, there’s a lot of potential tradeoffs players will find they have different priorities for.

        1. 1

          My own factorio experience is to start with spaghetti, then main bus (which are both covered by the article). Then, because the factory must grow, switch to city blocks, which also start introducing interesting concepts of queuing theory.

      6. 1

        If you want to learn more, spend some time watching videos by this guy: https://www.youtube.com/@Nilaus

      7. 1

        Busing is viable but packet transfers trains are even better later on. I usually end up switching certain mass produced items such as iron, copper and green circuits to railway as soon as I need to mass produce modules to boost the bus factories with beacons. Most scalable builds are grid of two to four parallel tracks with mini-factories in the eyes. Similar to multi-rack supercomputers connected with high-bandwidth fabric. At that point I usually give up and get back to regular programming and avoid Factorio for a year or so.

    2. 4

      This post is interesting, I would not have used the same analogy and I would have reached a different conclusion…

      To me the problem here is not that you have destroyed the coal, it is that the coal in unreachable when you need it (out of scope). And that is typically a problem that happens when you use… functions!

      Think about it, if your program is just a huge linear piece of imperative code and you need an intermediate result, you can just introduce a variable and access it later. But if you used fonctions, you may need to modify the signature of several layers of function calls to pass it down.

      There are several ways to solve this, the most trivial one being the nemesis of FP: putting everything in global state. Of course there are more refined solutions! For instance, frontend programmers know this as the “props drilling” problem, and this is why things such as flux/redux stores and context APIs were invented.

      In Machine Learning there is a similar problem: most models are mostly sequential, but you often need a previous intermediate result somewhere (think about skip connections or residual blocks). At work we have a framework to express models more declaratively than typical PyTorch, and we introduced contexts to solve precisely this.