1. 17

    Another disadvantage of services is that you can’t do a database transaction that includes operations from more than one service.

    Unrelated, I worked at a place a few years ago that built microservices because they wanted the various services to work concurrently, and they were using Ruby. Unfortunately, trying to boot all the services, the right order, ensuring a service didn’t come up unless one it depended on came up, was tough. It also forced me to switch from a spinning disk to a solid state one simply because of the number of files being loaded at once.

    That experience was part of what motivated me to learn Elixir. Years later, I recently worked on an Elixir umbrella app. It was like microservices, but running the whole thing was easy. In production, they deployed one app to one server, one to another, and let both of them depend on a third app, which was deployed to both. All the apps that needed a database shared the same one.

    1. 24

      Unfortunately, trying to boot all the services, the right order, ensuring a service didn’t come up unless one it depended on came up, was tough.

      Don’t do that. You need to deal with network issues anyways, so bring them all up and let your error handling take care of it.

      1. 8

        trying to boot all the services, the right order, ensuring a service didn’t come up unless one it depended on came up, was tough.

        I don’t know you usecase, but that seems to be a complex thing to implement that probably shouldn’t be needed.

        1. 2

          systemd has service dependencies and sd_notify built-in. Few people use it but it’s totally possible to start a web of services. Of course this doesn’t help with off-machine resources.

          http://0pointer.de/public/systemd-man/systemd.service.html

        2. 6

          Another disadvantage of services is that you can’t do a database transaction that includes operations from more than one service.

          You absolutely can; I do all the time. It may require fundamentally rethinking what we mean by “database” though.

          Most people think a database is somewhere between “a black box that stores data” and “a black box of hard algorithms for data query” or something like that. To them, they conveniently ignore that the “database” is a service outside of their application, and “can’t transact” across that service boundary either. That’s part of why database migrations and schema changes are so hard: Version 2 and Version 3 of your application are potentially separate services – even in the monolith – and because you “can’t transact” between versions of your application (what does that even mean!?) you’re forced to put logic into your application to deal with both the old schema and the new schema, and you may even often have to do on-the-fly upgrades.

          Or you just have downtime.

          Or you have a whole separate system, and your “transaction switch” is on some kind of network load balancer. Blue/green or whatever you want to call it.

          However there’s a very different way: If your application is the database (and this is easier than you probably think), your crud operations simply need to log their intentions, and then have something read the log which materialises the results into views you can use.

          One way to do this is to go heavy on the stored procedures. I like this approach, but SQL is a really terrible application language, and many programmers are very bad at SQL. Many databases don’t have an audit table for the stored procedures – there isn’t very good version control for them, so that’s another reason people don’t like it. Maybe the tooling could be improved though.

          Most people usually go the other way.

          In Erlang/Elixir, I may (for persistence) use a disk_log to write out arbitrary terms, and have a subscriber pick them up – that is also a gen_server that you can query. At startup, I can read the logs. If the logs are big, my gen_server knows enough to checkpoint – just write out State to another file, and include the offset in the disk log.

          In Go and Smalltalk you can do something similar.

          In C, or lots of other languages that don’t have processes, you can still get this functionality with a little thought, because the operating system has processes and fifos/socketpairs for you to use! It feels very similar to writing microservices, which sucks for different reasons (notably the lack of good IPC, e.g. php has serialize), but it’s not the same as microservices: just mutually cooperating processes. Qmail is a great example of this architecture, that is probably just a bit more complex than it would need to be today since the whole world is Linux and iOS now.

          In q I don’t even need to do that. I can just use the -l and -r options which give me logging/subscription built-in to the runtime. It’s also much more enjoyable because the language doesn’t suck as bad as SQL, and we have great IPC.

          Putting your data responsibilities in your application isn’t popular though. People actually think redis or Postgresql are fast, and many programmers doubt themselves able to make something as fast (let alone faster). This kind of approach however, tends to be done well around 1000x faster than using “a database” (and maybe done poorly or naïvely only 10x faster), gets you 100% uptime even in the face of schema changes, all of the benefits of a distributed/multiservice application with none of the downsides.

          This is, as I see it, a serious barrier: Programmers lack the confidence to build things outside of their specialisation (whether they call it “back end” or “front end”) and even terms like “full stack” seem to be (in their normal usage of the term) limited to “code” – very few “full stack” developers seriously consider rolling a new database for every application. And I think they should.

          1. 1

            What is q ?

            1. 1
          2. 3

            Another disadvantage of services is that you can’t do a database transaction that includes operations from more than one service.

            You can using distributed transactions, but that’s a whole other nightmare to contend with.

            1. 2

              Unfortunately, trying to boot all the services, the right order, ensuring a service didn’t come up unless one it depended on came up, was tough.

              Topological sort?

            1. 5

              Your frontend can’t crash if you don’t use JavaScript.

              That’s what you think, kiddo.

              “dev.to” is garbage - they have a fixed top and bottom bar

              What really pisses me off is that they used to only have the top bar (which, since they have infinite scrolling on the home page, does kind of make sense). They added the bottom bar after I registered, making the reading experience worse… The sign up link didn’t use to animate, either. What the hell happened to TPD?! It was always a bit fancy, but it got worse around the time they OSS’ed it.

              1. 1

                Those links are fascinating! Thanks for sharing. :)

              1. 4

                Related: https://www.destroyallsoftware.com/talks/ideology

                Some people claim that unit tests make type systems unnecessary: “types are just simple unit tests written for you, and simple unit tests aren’t the important ones”. Other people claim that type systems make unit tests unnecessary: “dynamic languages only need unit tests because they don’t have type systems.” What’s going on here? These can’t both be right. We’ll use this example and a couple others to explore the unknown beliefs that structure our understanding of the world.

                1. 1

                  Author here: Ideology by Gary Bernhardt is definitely part of the inspiration of this article, there is no point in hiding it. And I suggest you all go and check the talk, it’s really good, and goes way beyond programming.

                1. 17

                  My main beef with JS front ends is that they often break normal HTML behavior. Eg, an HTML link is easy to open in a new tab, but JS-powered links often aren’t. I’ve seen a banking site where it’s impossible to open a statement in a new tab or to use the back button after you’ve printed one.

                  It’s probably possible not to break normal web behavior with a JS front end, but it’s pretty hard to break it without one. I’d gladly lose a few well-done PWAs if I could lose all the half-baked ones.

                  1. 13

                    I expected the article to reference the whitepaper about the maintainers of npm rewriting a CPU-intensive service in Rust in order to improve performance: https://www.rust-lang.org/static/pdfs/Rust-npm-Whitepaper.pdf

                    Some quotes:

                    The rewrite of the service in Rust did take longer than both the JavaScript version and the Go version: about a week to get up to speed in the language and implement the program.

                    and

                    The process of deploying the new Rust service was straight-forward, and soon they were able to forget about the Rust service because it caused so few operational issues. At npm, the usual experience of deploying a JavaScript service to production was that the service would need extensive monitoring for errors and excessive resource usage necessitating debugging and restarts.

                    Presumably the npm maintainers are among the most experienced JavaScript developers on earth. Yet in one week they were able to learn enough Rust to write a service which turned out to be more performant and more reliable than their typical JavaScript services.

                    I don’t know Rust or Node, but if I were a Node developer, I think that would get my attention.

                    1. 3

                      they were able to forget about the Rust service because it caused so few operational issues

                      There’s a certain orthodoxy in tech’s interest in computation. Computation is cool, but only up to a certain point.

                      You can’t sell this sort of reliability via HN posts. There’ll be the odd blog post here or there, but it’s drowned out by the thousands of posts on why ReactiVueAngular from $MEGA_CORP is the bandwagon you need to be on. Nor is it ever as exciting as those posts. It certainly isn’t new.

                      This is probably the reason that architecture, algorithms, and general software design are viewed as ‘elusive’ skills despite the knowledge being widely available and easy to learn: they require actual time investment.

                    1. 8

                      Another cool thing about the end-to-end principle is that it lets the ends decide whether they even care about dropped packets (eg using UDP). If you’re on a call and a network blip drops two seconds of audio, you’d rather hear what they’re saying right now than wait for their missed audio to be retransmitted, leaving you perpetually two seconds behind. In a sense, you are the end application; if you think you missed something important, you can ask them to repeat it, but the network doesn’t make that decision for you.

                      1. 3

                        This title amuses me because I used to see developers online joking about how users are idiots.

                        People who don’t know the things you know are not automatically idiots. It’s extremely likely that they know a lot of things you don’t know.

                        1. 1

                          A lot of users fail to follow the most basic of instructions. That does, in my view, make them an idiot. They might be really good at their actual job. They might be incredibly effective at some things. They might know a lot. But it doesn’t matter how smart they are. If they fail to follow a basic list of instructions, they are an idiot. The same people, when you lead them through the list, one by one, will do them perfectly and easily. Yet they cannot literally read a list and do things in order. That makes them an idiot, in my opinion.

                        1. 3

                          I love this post and referred to it heavily while building a full-text search feature with Ecto (Elixir) and PostgreSQL, which I wrote up in a blog post. It was really nice to be able to implement full-text search without adding a separate data store and having to keep it synced with the primary one.

                          1. 8

                            Besides excluding people with less bandwidth or CPU and wasting electricity, a slow site wastes a lot of user-hours.

                            1. 3

                              This is a nice feature even in development, so that if you made a mistake in migration code, trying it out is painless; if it fails halfway through, you can tweak and try again.

                              Elixir’s Ecto library takes advantage of this in database migrations, as does Ruby’s ActiveRecord (feel free to chime in with others).

                              1. 3

                                Agreed, it makes life so much easier to atomically run migrations.

                                Django wraps its migrations in a transaction by default too, but it’s not really even a framework-specific thing; you can always wrap a transaction around any piece of code, be it migrations or other queries.

                                1. 1

                                  Yep, all you need is a BEGIN and COMMIT; I mentioned library support because I bet a lot of people are using transactional migrations without realizing it.

                              1. 1

                                I have never used Elixir or Phoenix myself but have used Clojure and Elixir looks like an alternative to it. About two weeks ago I posted a question in here asking how Phoenix.LiveView related to an Elixir library called Texas, unfortunately I pasted in the wrong link in the comment which made it look like a spam made by a bot.

                                Either way I’ll try to ask again now: Have you seen Texas and if so, how would you compare the two?

                                1. 1

                                  Just saw this. I have heard of Texas, but don’t know enough to compare it with LiveView. My understanding is that Texas was announced at the same conference where LiveView was announced. LiveView being the creation of Chris McCord (creator of Phoenix), I think it’s the more likely of the two to see wide adoption. But possibly there will be cross-pollination.

                                  1. 1

                                    Huh, what a coincidence. Thanks!

                                1. 1

                                  One factor to consider is stability. I imagine that my stock of marketable knowledge has some mass. I add to it when I learn. Knowledge erodes as I forget, and rots when it become obsolete.

                                  Some knowledge rots faster than others. Eg, everything I’ve ever learned about relational databases, algorithms, and Unix tools is still applicable. Most of what I learned about frontend development 10 years ago is obsolete.

                                  So you can look at a topic in risk / reward terms. How likely is this knowledge to become useless (risk)? How likely to become lucrative (reward)? Of course, there’s guesswork involved. But having judged that, you can pick based on how much time you have to invest in learning, and how much knowledge you can afford to lose. And you might diversify your investments.

                                  1. 11

                                    How do you figure something like this out, given you would ideally also like to align that with the path you want your career to take?

                                    Learn what’s interesting to you, and your career will end up bending towards that.

                                    1. 2

                                      I think there is a larger question. What if one stops being passionate about programming (though still enjoy doing it as requested at work)?

                                      1. 1

                                        I don’t think this advice requires you code 24/7 for fun, just that you have some preferences.

                                        1. 1

                                          Then learn to knit, or build model trains, or do political organizing, or whatever. You’re going to build skills whatever you learn, and your character will slowly change, too. Maybe those skills will be broadly translatable to your career (planning/estimation, design/architecture, communication/leadership), maybe not.

                                          But being happy in your non-work life will definitely impact your career. And new interests will, if nothing else, lead to new perspectives on both your existing problems and on future possible career paths.

                                        2. 1

                                          That also works in reverse: Avoid learning something where you don’t want your career to go. At least be quiet about it.

                                        1. 4

                                          You don’t have to write JavaScript, you have to write Elixir - which has a much smaller community around it than JavaScript does.

                                          This does look cool though, I just wish there were some live examples I could play with in my browser.

                                          1. 12

                                            On the other hand, the Elixir community is very friendly. :)

                                            Supposedly something like LiveView is coming to .NET - https://codedaze.io/introduction-to-server-side-blazor-aka-razor-components/ - but the post says:

                                            We don’t really know yet how well server-side Blazor (Razor Components) will scale with heavy use applications.

                                            In principle, people could take this approach in other languages as well. But I think Elixir / Erlang are uniquely positioned to do it well, as LiveView is built on Phoenix Channels, which (because they use lightweight BEAM processes) can easily scale to keep server-side state for every visitor on your site: https://phoenixframework.org/blog/the-road-to-2-million-websocket-connections

                                            1. 2

                                              On the other hand, the Elixir community is very friendly. :)

                                              Is that comment supposed to contrast the friendly Elixir community with the JS community? Is the JS community considered unfriendly? It’s way, way bigger than the Elixir community, so there are bound to be some/more unfriendly people. Maybe it’s so big that the concept of a “JS community” doesn’t even make sense. It’s probably more like “Typescript community”, “React community”, “Node community”, etc… But there are a lot of friendly people and helpful resources out there in JS-land, in my experience. I hope others have found the same thing.

                                              1. 11

                                                The Elixir community is still in the “we’re small and must be as nice as possible to new people so they’ll drink the koolaid” phase. The “community” such as it is is also heavily pulled from job shops and the conference circuit, so there’s a big factor too.

                                                Past the hype it’s a good and servicable language, provided you don’t end up on a legacy codebase.

                                                1. 5

                                                  Sounds like Rails, all over again.

                                                  Who hurt you @friendlysock?

                                                  1. 4

                                                    legacy codebase

                                                    How would you define ‘legacy codebase’? I’m assuming it’s something other than ‘code that is being used to turn a profit’..

                                                    1. 3

                                                      Ha, you’re not wrong! I like that definition.

                                                      From bitter experience, I’d say it would be an Elixir codebase, written in the past 4 or 5 years, spanning multiple major releases of Ecto and Phoenix and the core language, having survived multiple attempts at CI and deployment, as well as hosting platforms. Oh, and database drivers of varying quality as Ecto got up to speed. Oh oh, and a data model that grew “organically” (read: wasn’t designed) from both an early attempt at Ecto as well as being made to work with non-Ecto-supported DB backends, resulting it truly delightful idioms and code smells.

                                                      Oh, and because it is turning a profit, features are important and spending time doing things that might break the codebase are somewhat discouraged.

                                                      Elixir for green-field projects is absolutely a joy…brown-field Elixir lets devs just do really terrible heinous shit.

                                                      1. 2

                                                        So you’re saying that Elixir is just another programming language? It’s not the Second Coming or anything?

                                                        1. 1

                                                          I mean, it’s really quite good in a number of ways, and the tooling is really good. That said, there’s nothing by construction that will keep people from doing really unfortunate things.

                                                          So, um, I guess to answer your question: yep. :(

                                                        2. 2

                                                          Elixir for green-field projects is absolutely a joy…brown-field Elixir lets devs just do really terrible heinous shit.

                                                          Totally agree, but I would say that significantly more heinous shit is available to devs in Ruby or another dynamic imperative language. The Elixir compiler is generally stricter and more helpful, and most code is just structured as a series of function calls rather than as an agglomeration of assorted stateful objects.

                                                          The refactoring fear is real though. IMO the only effective salve for that sickness is strong typing (and no, Dialyzer doesn’t count).

                                                    2. 7

                                                      😊 I can see how it sounded that way, but I didn’t mean to imply anything about anyone else. The parent post said the Elixir community is small, so I was responding to that concern.

                                                      1. 4

                                                        Is the JS community considered unfriendly?

                                                        I feel you’re just trying to polemic on the subject… The author of this comment probably didn’t mean harm, don’t make it read like so.

                                                        1. 2

                                                          I’m not what you mean by “trying to polemic”, that doesn’t make sense to me as a phrase, but it was a genuine question about whether the JS community is considered to be unfriendly. I’d be happy to be told that such a question is off-topic for the thread, and I certainly don’t want to start a flame war, but I didn’t bring up the friendliness of the community. I’m sure the author didn’t mean harm, but I read (perhaps incorrectly) that part of their reply as part of an argument for using Elixir over JS to solve a problem.

                                                          1. 6

                                                            What I meant to say was: “If this looks like it could be a good fit for thing you want to do, but you’re daunted by the idea of learning Elixir, don’t worry! We are friendly.”

                                                            1. 3

                                                              I meant starting a controversy, sorry for my poor English! I’m sorry if it felt harsh, that wasn’t what I tried to share. I really thought your goal was to start this flame war.

                                                              Every community has good and bad actors. Some people praise a lot some communities, but I don’t think they mean the others aren’t nice either.

                                                              The only thing that I could think of is that smaller communities have to be very careful with newcomers, because it helps to grow the community. JS people don’t need to be nice with each other, the community and the project are way pas that need. So I guess you would find a colder welcome than with a tiny community.

                                                              1. 0

                                                                Hey there, polemic is a legit English word, so don’t be sorry for someone else’s ignorance! :)

                                                                1. -2

                                                                  I’m not ignorant (well I am, but not about this): polemic is indeed an English word, but it’s not a verb. The phrase “trying to polemic” doesn’t make sense in English, it requires interpretation, which makes the meaning unclear. I can think of two interpretations for “trying to polemic” (there may be others) in the context of the comment:

                                                                  1. My comment was polemic
                                                                  2. I was attempting to start a polemical comment thread, aka a flame war. With the later clarification that seems like what the author was thinking.
                                                                  1. 1

                                                                    The thing is that not everyone is at your level of English proficiency. You’re having a discussion here with people from around the world, you’ll need to make a couple of adjustments for expected quality of English and try to get the rough meaning of what they’re saying, otherwise you’ll be stuck pointing out grammatical errors all day.

                                                                    1. 1

                                                                      I wasn’t really trying to point out an English error, and perhaps I did a poor job of that. I stand by the claim that it is an English error though.

                                                                      I work with non-native English speakers all day, I’m aware of the need to try and understand other people and to make sure we’re on the same page. I’ll give a lot of slack to anyone, native or non-native, who’s trying to express themselves. The problem with the phrase “I feel you’re just trying to polemic on the subject’ is that at least some of the interpretations change the meaning. On the one hand, it could be saying that my comment was polemic, on the other it could be saying that my comment was trying to start a polemical thread. It’s not the same thing. And, for what it’s worth, if you’re going to throw an uncommon (and quite strong) English word like “polemic” out there it’s best if you correctly understand the usage. If the author had accused me of trolling, which is I think what they meant, that would have been both clearer and more accurate (though my intent was not to troll)

                                                    1. 15

                                                      Supercharge your inbox with our AI Gmail filter

                                                      Giving a third party the right to run code against every email I get is too high a price to pay for convenience, IMO.

                                                      I’m already trusting Gmail (foolishly or not). I just use their spam filters.

                                                      1. 1

                                                        Giving a third party the right to run code against every email I get

                                                        says the gmail user

                                                      1. 22

                                                        I gotta say, as annoying as recruiter emails can be, too many of them is the very definition of “a great problem to have.”

                                                        1. 7

                                                          If they represent real job options but you happen to have better ones, sure.

                                                          But if they’re truly irrelevant - requiring skills you don’t have and never claimed to have - they truly have no value to you and waste your time.

                                                          1. 1

                                                            Recruiter emails are to real job opportunities as “You may already have won the Publisher’s Clearinghouse Sweepstakes” letters are to actual lottery winnings.

                                                            1. 2

                                                              Some are good, but random recruiter emails do seem much more likely to be ridiculously bad - like lowly-paid and probably high-stress contractor “opportunities” for stacks I’ve never touched in a random city halfway across the country that I have no interest in moving to or commuting to.

                                                          1. 7

                                                            Often necessity to use deep copy (in any language) signals bad design: when you don’t know for sure that some code to which you pass that object will not mutate it and its deep sub-parts. That’s overly defensive programming. I rarely encounter even shallow copy in practice (almost always it’s right before mutating such objects) and almost never encounter need for deep copy.

                                                            1. 4

                                                              I’m not an FP zealot, but I work with Elixir these days, and “deep copy” is an alien concept there. If you add an item to a list, you get a new copy of the list with the new item added. It’s impossible to affect any other bit of code that had the old list.

                                                              It’s nice not to have to think about that.

                                                              1. 3

                                                                I agree it’s bad practice. My moment of enlightenment was realizing the problem is isomorphic to serializing/deserializing an object graph. Example if you have machinery to completely serialize the state of an object (or an object graph), just use that for a “deep copy” ie deserialize multiple times.

                                                                1. 1

                                                                  I don’t agree that is bad design. Let’s say that you have a method that updates an Entity and at the end, you emit an event with the new Entity and the old one. One solution is to fetch the entity from database clone it to a $previousEntity variable and use the $entity to perform the replaces you need and then emit the event. However does not mean you should have mutable VOs but from my experience at every place I have been there are always exceptions for certain reason and it can be very easy to start having bugs and very difficult to understand where they are coming from ;)

                                                                1. 1

                                                                  Readability is subjective, as the author says. A think a helpful question is “who is the audience?” If I write code and my teammates can’t understand it, that’s a problem. Whether some hypothetical outsider could is not relevant.

                                                                  Maybe my teammates can’t understand it because they need to learn something, or maybe because the code is poorly-written. In either case, I need to help solve the problem, lest I leave them with something they can’t maintain.

                                                                  1. 2

                                                                    I think a faster way to do this is to use a LFSR generator to create a sequence, then select ‘rowid in (…)’. This will be constant time even for deep pages.

                                                                    1. 1

                                                                      Interesting. If I understand correctly, you’re saying I should generate a list of ids in memory on each request, then query the db for rows with those ids?

                                                                      1. 1

                                                                        Yeah. You can find lots of examples by searching for LFSR generator. It has a very small internal state that you can add to the request. Also used for the Doom fizzlefade effect. http://fabiensanglard.net/fizzlefade/index.php

                                                                        You can pick parameters based on your table size. One issue you might have is if you have missing rows, you’ll only get 18 or 19 rows instead of 20 sometimes, but that’s probably not a problem for infinite scroll. Or you select a few extra, etc.

                                                                        1. 1

                                                                          This is a very cool idea. The main downside I see is that it doesn’t play well with other WHERE criteria; if we’ve pre-selected the ids for page 3, but few or none of those records match a WHERE condition we want, we’re out of luck.

                                                                          But I’m definitely going to keep it in mind for future reference. It could be done even without LFSR, by pulling SELECT id FROM table, chopping it into pages in app memory, and caching it.

                                                                    1. 42

                                                                      Reminds me of a quote:

                                                                      Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

                                                                      • Brian W. Kernighan
                                                                      1. 13

                                                                        :) Came here to post that.

                                                                        The blog is good but I’m not convinced by his argument. It seems too worried about what other people think. I agree that we have to be considerate in how we code but forgoing, say, closures because people aren’t familiar with them or because we’re concerned about how we look will just hold back the industry. Higher level constructs that allow us to simplify and clarify our expression are a win in most cases. People can get used to them. It’s learning.

                                                                        1. 8

                                                                          I think he may not disagree with you as much as it sounds like. I don’t think that sentence says “don’t use closures,” just that they’re not for impressing colleagues. (It was: “You might impress your peers with your fancy use of closures… but this no longer works so well on people who have known for a decades what closure are.”)

                                                                          Like, at work we need closures routinely for callbacks–event handlers, functions passed to map/filter/sort, etc. But they’re just the concise/idiomatic/etc. way to get the job done; no one would look at the code and say “wow, clever use of a closure.” If someone does, it might even signal we should refactor it!

                                                                          1. 5

                                                                            It seems too worried about what other people think.

                                                                            I agree with your considerations on learning.
                                                                            Still, just like we all agree that good code must be readable, we should agree that it should be simple too. If nothing else, for security reasons.

                                                                          2. 2

                                                                            On the other hand, sometimes concepts at the limit of my understanding (like advanced formal verification techniques) allow me to write better code than if I had stayed well within my mental comfort zone.