1. 3

    Changing to a BTreeMap instead of a HashMap in the LRU cache to optimize memory usage.

    Why would a BTree map use less memory in rust?

    1. 3

      Hash maps usually try to have a lot of empty slots for performance, while afaik a BTree will usually just have the necessary nodes, right?

      1. 1

        I think it’s more about access patterns than just size. In a hash map the elements are essentially randomly jumbled about, whereas In a BTreeMap they can be sorted by time. The oldest elements will be closer together in memory and O(1) to find, which are both great news for a LRU cache.

        1. 2

          They didn’t say BTreeMap is faster, they said BTreeMap uses less memory. In fact, I very much doubt it was faster.

          1. 0

            I have no trouble believing it was faster for the workload. After all, that was the point of the article.

      1. 2

        I’ll continue hacking on my my ssh-chat bot which will send me notifications when i am mentioned on the chat.

        1. 1

          i have a pi and a meerkat(intel nuc). I run pihole, NFS share and a few long running sessions at home (ssh-chat, irc, etc.)

          Thinking of setting up NextCloud or perkeep for data next.

          1. 3

            I’ve been saying this for years! I remember when it was a small group of really excited people did code in their spare time (it was a larger lift back then) and others had different hobbies.

            Everyone did fine, and often those people who took breaks did better at work because they had time to retain what they learned, and also had ways to relate to non-technical people (a thing you frequently need to do as you level up) or even network internally and externally with other tech. people to learn and improve.

            1. 2

              completely agreed, coding in spare time and being good at writing code are orthogonal properties (you can do one without getting any close to the other).

            1. 3

              I’ve been thinking about this since i found out about zettelkasten method developed by Niklas Luhmann (on a comment on this excellent post You are your tools).

              What I’ve come to realize is data organization in libraries and even by pre-compute dictionary designers could be very insightful on how we create indexes on data.

              Reading more papers about data organization pre-computers is on my todo list, has one else explored this area?

              1. 2

                This particular line of questioning caused me to discover the idea of commonplacing, and the existence of Commonplace Books, which in turn impacted how I take notes and how I collect and organize my own thoughts into a knowledge base.

                The main components are indexing, headings, and purposeful marginalia.

                I have a digital knowledge base (in the form of plain text files that I have recently migrated to a github wiki) that is mostly technical / professional related, and an analog pen-and-paper knowledge base (in the form of a commonplace book) that is more personal interests.

                1. 1

                  A commonplace book is exactly what i was looking for, i’ve started using a mac app NVAlt for note taking and it’s amazing search and reduced the need for creating indexes manually.

                  But i still want to try writing things down on a physical notebook! will definitely give John Loke’s method a try.

              1. 5

                Really liked the article, clear and concise! however i ended disagreeing with the disagreements.

                • A strong stance against exceptions

                Exception are good only for exceptional cases and not any non-favorable behavior. An http request failing is not an exceptional case but most systems treat it as such and create infrequent code paths which can lead to unforeseen failure conditions. (further anecdotal proof/opnion)

                • John is quite in favor of writing comments

                The post author’s stance is not in congruence with their agreement on deep modules. If a class/module has enough depth, it might require a why comment.

                1. 2

                  Thanks for the feedback and appreciate you voicing your disagreement.

                  For exceptions, the book seemed to suggest going out of the way to throw them. I’m not that dogmatic and don’t mind them - as long as there’s good monitoring, and we follow a zero-exceptions policy. So I’m more on the mitigation side, than prevention, especially for backend software.

                  For comments, I’m not a fan of inline comments. The “why” comments, especially at module/class level, I am definitely for. I really like how you put it: for deep classes, comments can probably explain a lot of their depth, helping maintenance greatly. Until I read your comment here, I did not think along this dimension, though. Thanks!

                1. 2

                  I’ve been wanting to write one in Golang backed by BoltDB or Badger for fun purposes. So far I’ve not got time to do so; but I hope I do soon and this article will be helpful.

                  1. 1

                    I’ve been thinking about the same thing. I also wanted to make it multi-tenant (only certain topics are visible depending on client credentials) and possibly distributable among many nodes.

                    The biggest challenge right now is finding the time for it.

                  1. 3

                    A A4 piece of paper folded like a pocketMod to carry anywhere and Evernote to scan and keep. A Xiaomi pen.

                    1. 1

                      Do you scan them every day or at another cadence?

                      1. 2

                        It depends on the note. Some are short lived temporary ones. Others are more important. Those I scan as soon as I can. pocketMods are small and I tend to dispose them easily.

                      2. 1

                        Yes! I always keep one of these in my back pocket.

                      1. 2

                        My main contribution here is not a specific brand of notebook, but a specific practice: every bag I am likely to carry frequently has a notebook and a pen. Whenever I have something to write down, I have the tools.

                        This means that I don’t have a chronological lab book or journal, because I write in whichever notebook I’ve brought with me.

                        Oh also, a product review: I bought the Livescribe pen, and while the practice of digitising notebooks was useful, the pen did not last long. The plastic deteriorated and cracked where the barrel meets the tip, within a couple of months of ownership, rendering the pen useless.

                        1. 1

                          how do you review your notes? i would assume that you end up writing more in some notebooks than others hence finding related notes must be tough.

                          1. 2

                            Two factors that help:

                            1. The bag I take is context dependent so it’s not random where a note ends up.
                            2. The one in my laptop bag (which most often joins me in a work context) is loose leaf so those pages get files somewhere.
                            1. 2

                              interesting, you might find discbound notebooks helpful

                        1. 3

                          Come on, really? The throwaway copycat what’s-your-editor-setup posts were bad enough, but this is plainly offtopic. I hate to be a grinch but if there’s inflection point somewhere where this site just turns into HN or reddit, and I think we should sanction it even though it’s oh-so-tempting to chip in with our personal “setups”

                          1. 5

                            Not all memes are bad. We’ve had a grand total of 4 of these posts, and all of them very easily fall under software development. (Yes, even this notebook one, since I and others have or often do use notebooks to aid software development and programming)

                            I agree that there is a tipping point beyond which they will no longer useful or relevant, and I think we’re close to having tapped out this well in this wave. Sometimes it’s nice to let loose a bit, however, knowing that it will pass.

                            I could see a monthly or quarterly “What is your (work|non-work) development setup” post also being a nice addition to our periodic book recommendations. I don’t think it merits the weekly nature of “what are you doing”, since development setups aren’t changing that often.

                            1. 3

                              It’s on the ask tag; if asking questions is a topic, then it seems on topic. Maybe you want to get rid of ask, or just want to filter it out? I don’t want you to have to see this posts if you don’t want to.

                              1. 5

                                I bet you’re fun at parties. The contrarian gets the upvotes, but think carefully about whether or not you want to reward this kind of behavior on Lobsters.

                                1. -1

                                  Thanks for the personal insult, I appreciate it. I did not actually post this just to be contrarian and score points, as you’ve assumed.

                                  Speaking of rewarding behavior: no, I don’t want to see the rewarding of what are essentially meme posts, and I don’t want this site to gradually veer away to where the front page is meme posts and offtopic discussion. We have reddit for the former and HN for the latter. This thread is mostly a discussion of paper notebooks and favorite pens. I’m sure there’s a subreddit for that.

                                2. 2

                                  but this is plainly offtopic.

                                  Asking lobsters about a tool that they use for/during development in my opinion is not off topic with respect to the tags used for the post. But the mods can clarify that. We’ve discussed this exact topic many times over the years and even development tool generally, why is it bad?

                                  I first learned about Bullet Journals from a comment on lobste.rs and it improved the way i work & write software by a lot.

                                1. 4

                                  What: I switched to zsh 4 years ago and dumped my custom bash config in-favor of oh-my-zsh.

                                  Why: Had to setup multiple new installs (due to new releases, server upgrades, etc.) managing my bashrc was getting tedious and oh-my-zsh was way nicer than what i could get by spending the amount of time i had for bash.

                                  1. 3

                                    In my only experience architecting a large C# application, I never reached a satisfactory conclusion on this topic.

                                    In practice, singleton never seemed to solve any of the issues that led to my exploring it, instead acting as a temporary tonic to my OOP design anxieties while obscuring implicit behavior and making my application harder to reason about.

                                    My final design in said application had me explicitly instantiating at the main entry point a small number of objects representing legitimately global systems (datastore, audio controller, etc.) and explicitly passing those objects as parameters until functions became too yucky, then refactoring to wrap them in something akin to a master application configuration object with a rigid interface.

                                    The dissatisfaction I had and have with this final implementation seems to be aesthetic, like it violates some deep OOP principles (likely), but which aren’t solved with singleton.

                                    Using singleton made me feel clever as a “consumer” of my own singleton each time I wrote a function call, but made the program harder to reason about because the implicit behavior was still implicit, just implicitly implicit instead of explicitly implicit.

                                    1. 4

                                      disclaimer: these are observations from the code bases I’ve worked on, i don’t know your specific situation

                                      explicitly passing those objects as parameters until functions became too yucky

                                      Every time I’ve reached this situation the fault was bad abstractions. The methods were doing too much or needed to be aware of too many things. Later on during development, these also emerged to be the spots where it was difficult to hold the complete logic in my head.

                                      even when said state is legitimately global

                                      There is always global state the point is that the lower level code shouldn’t be aware of it or depend on it. The lower level code should only know what it was provided.

                                      I’ve try very hard to not write any singleton classes but working with a large team with tons of code, i semiregularly stumble upon something that resembles a singleton, it might look like snippit 1 or 2 from the article and I’ll refactor it to snippit 5.

                                      I am not sure if this is useful, it took me a while to realize that patterns are not tools to solve a problem they are categories of solutions. If you end up using a solution that looks like a pattern refactor the code to fix the problems that the pattern frequently has/prevents.

                                      1. 3

                                        every time I’ve reached this situation the fault was bad abstraction

                                        Undoubtedly true in my situation. The most hair-pulling breakdowns in coherent design always appeared in sub-systems I had the least experience with.

                                        Audio is a useful example. If the audio controller was told to play a sound, how and when would it actually play? What if something else was already playing? Did the controller need to understand the caller and why the sound was triggered, and thus depend on every potential caller? Or if the caller passed the details, didn’t that just flip the dependency and unravel the encapsulation of audio logic?

                                        Presumably someone with deeper experience would have started from some sort of informed design, but I bolted on sound after everything else and learned that you had to think about such things only when I stumbled over them for the first time.

                                        1. 2

                                          The most hair-pulling breakdowns in coherent design always appeared in sub-systems I had the least experience with.

                                          Absolutely, this happens all the time.

                                          learned that you had to think about such things only when I stumbled over them for the first time.

                                          i am still pretty bad at this, so i first think, then sketch out what i would do and then do it 2-3 times. it’s a little slow in beginning but keep development faster in the long term.

                                      2. 1

                                        My final design in said application had me explicitly instantiating at the main entry point a small number of objects representing legitimately global systems (datastore, audio controller, etc.) and explicitly passing those objects as parameters…

                                        I do something like this in C# using the Ninject dependency injection framework. It offers various scopes for those to be injected in (e.g., singleton, thread, ASP.NET request, etc.). This way, you don’t really have to write any singleton classes as such, and your calling code doesn’t have to do anything special other than accept dependency injection. It makes testing easier than with an actual singleton, too.

                                      1. 4

                                        Ken, if you’re a lobste.rs user, we’d love to hear you weigh in. :)

                                            1. 3

                                              Hm. If I was Ken, I’d be grumpy at least, having to think of another password at the age of 76 ;) - and I cound not even pick another chess move, because these will be added to wordlists in no time.

                                              1. 15

                                                Well, the he doesn’t need to worry. He can get in without a password at all.

                                                1. 2

                                                  This article was amazing

                                            1. 5

                                              Corporate structure is a replication of feudal structure where the board of directors delegate oversight to lower and lower levels of company “nobility”, and everyone gets to set their own renumeration. From a bottom up perspective they’re not necessary for production at all, but from capital’s perspective executives are the obviously rational way to defend their property interests.

                                              In short, they don’t do anything except for defend profit and extract surplus.

                                              1. 4

                                                At least unlike nobility of centuries past, or even the dynastic capitalists of 100 years ago, executives are required to work, to such an extent that you hear them engaging in the same “Gosh I’m so busy” virtue signaling of working all the time.

                                                The idle rich still exist, but it is not the executive described here. Labor exploitation has moved up the ladder and you have corporate leaders missing their kid’s birthday party to work on the weekends, the same as an aspirational middle class worker picking up an extra shift.

                                                I forget where I read this, or I would share, because it surprised me, but the percentage of people who are in the 1% that get the majority of their wealth from income/salary is higher than it’s ever been. Now that’s percentage of people in that percentile, not percentage of combined raw wealth in that percentile, but fascinating nonetheless.

                                                1. 1

                                                  I agree, but can you offer an alternative? I happen to be involved in a political party and it’s a hard problem just to set up reasonable vetting of newcomers. You get all kinds of applicants ranging from math nerds who love trains to paranoid crackpots who notify police that your WiFi is reading their thoughts. Some are latent Nazis, others radical anarchists. Propose a framework to vet and integrate newcomers, please.

                                                  1. 3

                                                    Not sure what this has to do with company execs, but engaging with it anyway:

                                                    Send them a letter describing how your meetings work and the expectations on members.

                                                    Then invite them to meetings as a prospective member and ratify their membership after N meetings if you want to keep them in the group?

                                                    The political party I am a member of has a simpler system: I think it just accepts members unless a cursory search of public records shows bad behaviour.

                                                    There’s a complaints process for removing members that are problematic. Complaints process overseen by representatives elected by members.

                                                    1. 1

                                                      Complaints process overseen by representatives elected by members.

                                                      Hierarchy. And since they are elected, they have a higher chance to get re-elected.

                                                      They can also throw out people who oppose them and keep their supporters, thus solidifying their position even more.

                                                      Does your party happen to have a rule-making committee (perhaps with a delegate system)? Commies had (or have) one (both Soviet and Chinese). Then they elected a subcommittee (called Politburo) to hold the power in-between the sessions which eventually gained enough support in the central committee to mandate that they will pre-select central commission candidates. That’s how a nice democratic party becomes a top-down evil corporation. (It helps that a member of politburo is a minister of interior.)

                                                      I mean, it’s hard for me to imagine systems that operate in non-hierarchic ways. But the problems with hierarchy are real and are independent of organization type.

                                                      1. 2

                                                        Rules can be changed by assemblies of delegates at different levels, yes.

                                                        If you want to avoid traditional delegate systems then you can use sortition (select executive randomly from membership like jury service).

                                                        If you want to avoid hierarchy entirely then you need anarchist theories, I think.

                                                        I think traditional delegate systems can maybe work OK so long as the electorate is informed and the electoral system is expressive and fair enough. Israel has a good democracy by those measures, apparently, but I don’t know how well the Israeli state is considered to be serving its people.

                                                    2. 2

                                                      I agree, but can you offer an alternative?

                                                      The corporate structure is pretty dictatorial, and there are alternatives to that even in today’s society, like with the government. Suppose company executives were elected by and can be recalled by the population of the company, or perhaps there was some kind of representative oversight council that had to ratify executive decisions (think like the US Senate approving of the president’s appointees), but otherwise did basically the same job they have now.

                                                      Obviously, as we see in government, this is still far from a perfect system.. but it more-or-less works and people are already familiar with the concept so it might be an easy baby step to consider.

                                                    3. 1

                                                      who are the share holders in this analogy?

                                                      1. 2

                                                        The mass of shareholders forms an abstract capitalist which is the monarchy at the top of the feudal hierarchy. The shareholders (of which the board is some portion) have absolute sovereignty over production.

                                                        1. 2

                                                          That’s incorrect, though. Most of the big ones are cartes of cartels with interlocking boards of scheming people backing corrupt laws that benefit their own class, specifically them, at everyone else’s expense, including shareholders. The leaked memos from Citigroup called the new structure a “plutonomy.” Just a new form of plutocracy where they keep their control in executive positions, the boards, paid-for politicians, and infiltrated regulators (esp Goldman).

                                                          If really working for shareholders, the folks in this class would’ve sent them the vast majority of the wealth instead of extracted it for themselves with shareholders holding a mix of real money and IOU’s that can change in an instant. If for stakeholders, the employees and customers would get treated better, too. They haven’t been about either in most cases.

                                                          That’s why there’s a new push for public-benefit companies, foundations, and other structures with incentives that might fix some of this.

                                                          1. 1

                                                            It’s an interesting way to model the corporate structure.

                                                      1. 3

                                                        I wish it was written top-down. I found it makes code easier to read for me.

                                                        1. 3

                                                          top-down

                                                          does that mean high level functions first? like the example below?

                                                          def HighLevel():
                                                              X()
                                                              Y()
                                                          
                                                          def X():
                                                              ...
                                                          def Y():
                                                              ...
                                                          
                                                          1. 2

                                                            In this case yes, that’s what I meant.

                                                        1. 4

                                                          I’m a bit confused about the date thing. My Mac has the date and time in the menu bar on the right.

                                                          1. 1

                                                            Dato can show more information like the calendar and multiple time zones.

                                                          1. 2
                                                            • et-see - initially used to call it ee-tee-see till i heard someone pronounce it et-see
                                                            • lib - like lib[erty]
                                                            • char - like char[acter], called it char[coal] till someone made pointed out it was wrong (it wasn’t wrong but the person was not comfortable with the idea that there can be different pronunciations)
                                                            • f-es-see-k
                                                            • skema

                                                            i taught myself to code and had different pronunciations in my head which changed as i was exposed to other people who also wrote code and used linux.

                                                            1. 2

                                                              It is said that people who pronounce things “wrong” should be respected because they learned it from a book (or now the Internet) rather than being taught in person :)

                                                            1. 7

                                                              Setting up my personal software (daemons and bots) for raspberry pi. They are mostly written in Go so it should in theory be as simple as recompiling. And traveling for the rest of the weekend.

                                                              1. 3

                                                                I am setting up a build environment for AWS CloudFormation/Lambda applications in VS Code. I’ve worked on all the pieces separately. I think I grok them enough to assemble it all together into a code-build-test-deploy project.

                                                                I’m starting off with a triple-language lambda – code that does the same thing in Python, C#, and F#. The cloud I set up should call this code on a regular basis. It will poll a website for changes using a RESTFUL API. If it finds any, it’ll grab the new information and put it in a publicly-accessible S3 bucket.

                                                                I was going to add Route 53 and CloudFront stuff to the build, but I’ve already developed some stuff using that for my first CF stack.

                                                                I want to say it’s tough, but it’s not. However it’s going quite slowly. I think the reason is that while I know a lot about the separate pieces, trying to assemble them into a AWS Cloud Architect-y kinda workflow is quite painful. There are a zillion little loosely coupled pieces all over the place. And aside from basic ATDD poking around, I’m not really sure what it means to test a cloud architecture.

                                                                There’s a lot of stuff to learn and I’m having fun. That’s the important part.

                                                                1. 1

                                                                  I believe CloudFormation will allow you do have CI as part of a deployment for your lambda functions if you change your workflow so the repo’s on AWS. Testing AWS offerings like lambda is a sort of vendor lock in because there’s really no way to do it any other way.

                                                                  1. 1

                                                                    You are correct – and you may be anticipating where I’m going with this.

                                                                    I’ve been writing what I consider to be “true” microservices for many years now: small, pure functional apps that run from the command line. By default they use streams, but any data sync should be fine for them. I don’t use a bus; instead I use various OS features for glue and overall testing.

                                                                    In this environment, lambda really isn’t that big of a deal. I’m getting data via JSON instead of streams. There may be some event/interface work I need to do. That’s part of what I’m figuring out. I know AWS will keep an instance warm for a bit even if you’re not using it, so I need to think about setup vs. processing. Still, I think it’s going to be trivial.

                                                                    I agree with the vendor lock-in concern. I think you can do it without that problem as long as you carefully pack your own picnic box. That is, functions with data streaming in and out that your programmatically join is one thing. Integrating you coding and CI/CD process with your cloud vendor is another thing entirely.

                                                                    There’s the ever-prevalent “video game” danger with AWS: lots of pretty screens with easy to click-and-deploy cool-stuff. I love the ease-of-use part, and I love the offerings. The danger is that you have to be aware of the linkages to Amazon you’re creating. Best keep everything as code, everything in your own repo (GitHub for me), and view where the stuff runs as plug-and-play.

                                                                    This, of course, means honking around with all the little details involved with coding, pipelines, and clouds. That sucks, but I’m okay with that. Great way to actually learn stuff instead of just poking around until it works.

                                                                    1. 2

                                                                      I’ve been working on a similar stack, to avoid any vendor lock it i have started using the LambdaProxy project and now i have a ‘local’ binary which runs a plain http server and is very useful for debugging.

                                                                      I am also going to look into CDK because writing json/yaml for cloudformation isn’t what i want spend my time one.

                                                                      1. 1

                                                                        My belief is that a parameterized CF stack should work for a bunch of stuff. I already have one for static websites. I’m developing this lambda one. There are probably 3 or 4 others and then I’m done. The entire point is not to rely heavily on AWS, so at some point the more I’m doing CF/Lambda stuff the more I’m headed down the wrong road.

                                                                        Once you’ve got deployment and execution in, the rest of it is just replicating continuation stuff, right? Sequencing, map-reduce, orchestration – maybe a couple of others. At that point I think you finish up the work with a bit of containerization and K8 scripting.

                                                                        As an industry, we’re really not that far from true vendor/platform/OS/language-agnostic coding, where we only pay for storage and CPU cycles, ideally at market rates. We’re very, very close.

                                                                1. 8

                                                                  HN comments

                                                                  the author commented on HN about the attempts made previously before settling.

                                                                  1. 3

                                                                    I also recommend reading HN version of articles like this since you’ll see a lot of projects or implementation techniques pop up in the discussions. Especially on interpreters or compilers.

                                                                    1. 1

                                                                      How about we have a threaded story reader that can pull (and rank/rate) comments from multiple sources. If posted to a small enough subreddit, even the reddit comments could have content.

                                                                    2. 3

                                                                      Yep they had done some work with go, Lua before they settled down to rust