1. 6

    I’m currently a Python dev (apparently this is the most recent turn my career has taken), and I’m really bummed out by its web story outside of Django.

    My last gig was Elixir, before that Node, and some Rails and Laravel in there. The tooling in the Python ecosystem, especially around migrations and dependency management, just feels clunky.

    It singlehandedly sold me on Docker just so I didn’t have to mess with virtualenvs and multiple runtimes on my system and all of that. Like, what happened? Everybody groused about 2-to-3 (which is still hilarious) but like even without that I feel like the ecosystem has been vastly outstripped by “worse” technologies (see also, NodeJS).


      It singlehandedly sold me on Docker just so I didn’t have to mess with virtualenvs

      One thing that made virtualenvs almost entirely painless for me was using direnv: in all my python project directories I have a bash script named .envrc that contains source .venv/bin/activate, and now cd-ing in/out of that directory will enter/exit the virtualenv automatically and instantaneously. It’s probably possible to set it up to switch pyenv environments as well.


        One of the reasons why Python packaging still feels so clunky compared to other ecosystems is that the Python ecosystem is a lot more diverse thanks to e.g. the scientific stack that has very different needs than the web peeps so there’s never gonna be an all-encompassing solution like Cargo. Pipenv tried and failed, poetry is carving a niche for itself.

        But the primitives are improving. pip is currently growing a proper resolver and doesn’t e.g. Ruby still need a compiler to install binary packages? As long as long as you don’t use Alpine for your Docker images, Python’s wheels are great (they’re just a bit painful to build).


          How did pipenv fail?


            Short answer: it’s too complex which makes it buggy and there wasn’t a release in over a year. IOW: It’s falling over it’s own weight.

            Long answer: https://hynek.me/articles/python-app-deps-2018/


          The tooling in the Python ecosystem, especially around migrations and dependency management, just feels clunky.

          Currently working on a Rails app, coming from the Flask ecosystem. You have no idea how much I can miss SQLAlchemy and Alembic.

          I agree about dependency management, but certainly not about migrations. Modifying models and auto-generating migrations works much better than the other way around for me.

        1. 5

          But will it support HTML email?

          1. 4

            I really hope not. Standard though it may be, HTML email is a cancer that needs to die out. I always recommend new users on my platform to read useplaintext.email.

            1. 11

              It does support HTML e-mails. However:

              • Sanitized HTML e-mails are displayed in sandboxed <iframe> elements where many features are disabled (e.g. JavaScript). The Content-Security-Policy set by koushin is an additional security.
              • This can be easily disabled: just disable the viewhtml plugin
              1. 2

                I’m curious about this sanitization. Is there a standard algorithm or set of checks that are performed?

                1. 1

                  Sanitization is performed via bluemonday, a widely used Go library.

              2. 5

                HTML is abused, but I don’t want email where I can’t even use bold or italic, or hyperlinks like the one you put in your comment. Why should emails be less expressive than forum comments?

                (Yes, I would totally welcome Markdown email as a standard. I’m sure it won’t happen, though.)

                1. 6

                  Reverting to plain text, Markdown or some other kind of ad-hoc markup because HTML can be abused makes me shudder. I want nice typography and proper design in my emails. I want reading an email to be as good an experience as reading a well-designed website. In particular, I like the current resurgence of high-quality newsletters delivered to my inbox. Instead of throwing up our hands and giving up on this, I would like to see us actually attack and try to solve this problem.

                  1. 4

                    It’s funny, because that’s where Markdown comes from.

                    the single biggest source of inspiration for Markdown’s syntax is the format of plain text email


                    1. 2

                      A lot of good emails clients will display plaintext surrounded by underscores in italics and text surrounded by asterisks in bold. As to hyperlinks, that’s what [1]. Some clients that work particularly well with plaintext will make that [1] a clickable link so you don’t have to scroll down and search for it either.

                      I completely agree about markdown emails. The client that implements that will be my gold standard.

                      [1]: is for

                      1. 5

                        imagine if we had some well-specified plain text system for applying inline formatting to text that “good” email clients could render, and hyperlinks could go inline so you don’t have to use up more screen space and manually re-index them when you add or move them in the text.

                        maybe we could use less than and greater than characters to enclose the inline formatting commands. it would be great!

                1. 1

                  For a project where I intend to be the only programmer, or only grow the team to a handful of people, probably Lua (and some C if necessary).

                  For a larger project, I would choose depending on a lot of factors. Probably one of Python, Erlang or C. TypeScript if it’s in the browser, but I’d probably avoid doing something too large in the browser :)

                  1. 30

                    And after discovering late in the interview process that Apple has a blanket ban on all programming related hobby projects:

                    Ask any potential new employer about their side project policy early on

                    Wow. I cannot even fathom working under such conditions.

                    1. 15

                      Yeah, my view is that my employer owns my time when I’m at work, I own my time when I’m not at work. As long as I don’t use their intellectual property and don’t secretly work for a competitor, they have no influence on what I do off work.

                      Luckily, I’m in the extremely privileged position that I can afford to pick between jobs based on that criteria. I have gotten unreasonable clauses removed from contracts because I have the legitimate option to not take a job I’m offered. Many people aren’t in that situation.

                      1. 10

                        The author explained the rationale in a comment on the HN thread. I, too, couldn’t imagine working in such an environment; the whole point of employment imo is to be able to support myself enough to work on side projects.

                        1. 9

                          Note that clauses like this are illegal in some countries. For instance, in France, exclusivity clauses cannot cover unpaid work (and they become void for a year if you create a company).

                          This does not cancel loyalty clauses and non-competes, but those are strictly regulated as well, and I think Apple’s “we can work on anything without you knowing” would not be applicable.

                          1. 3

                            I don’t go for it personally, but for some people ‘impact’ is one of their biggest motivations.

                            The sheer number of units shipped makes apple an attractive place to work if that’s one of your motivators.

                            1. 1

                              Don’t believe everything you read on the internet. This is category untrue.

                            1. 2

                              It’s the first weekend of February so I am at FOSDEM, for the 10th year in a row.

                              1. 3

                                The comparison is a bit biased in favor of compiled languages because interpreters are always passed a file containing the source code. Passing the code as a command-line argument, for instance, reduces the number of syscalls.

                                For instance, on my machine, Lua (5.3.5) uses 80 syscalls if I pass the code as a command-line argument and 85 if I pass it as a file. Python (3.8.1) uses 944 and 969 respectively, Perl (5.30.1) 227 and 229.

                                Note that Lua still always fewer syscalls than the Rust example (which uses 95 on my machine with 1.40.0)…

                                1. 2

                                  Third week on a new job. The website is French only, but it’s basically a CRM and ticketing solution for property managers - think specialized Zendesk Suite. I keep getting acquainted with the code base, start fixing more complicated bugs, and jot down ideas for how some things could really be improved. Pretty happy so far.

                                  At home, I’m doing Advent of Code for the third year in a row. Inch is mostly Rails, and I haven’t touched Ruby in about 8 years, so I took the opportunity to use it this time. Not seriously laddering or anything (that would mean waking up for 5 AM everyday in my timezone) but I do keep up so far.

                                    1. 2

                                      I did them all almost on time in 2017 and 2018, in Lua both years.

                                      This year I think I’ll try to do it in Ruby, which I just started to use at work, but I probably won’t solve all problems on the day they come out (for lack of time).

                                      1. 4

                                        Have the author ever read the OWASP suggestions? There is explicite suggestion to prevent door knocking and to lock out abusive users, and almost all applications will simply lock-out users that are trying to brute-force login form.

                                        1. 1

                                          A lot of applications will lock them out based on the login, but that doesn’t prevent an attacker from DoSing you using multiple logins. You need to ban IP addresses for instance, and few applications actually do that.

                                          There are other solutions to the issue, for instance make the client compute a challenge (proof of work) to attempt a login.

                                        1. 1

                                          I wrote my own in Lua years ago, but I would recommend that you take a look at Frozen Flask if you don’t know about it yet. It works a bit differently from usual SSGs and gives you a lot of flexibility.

                                          1. 9

                                            It is not just the compiler. It depends on the CPU if 4+4+4 or 3*4 is faster. The instruction set itself does not give any guaranties. So even assembly language is not how the computer works?

                                            1. 22

                                              Nowadays I’d say that no, assembly language is not how the computer works. The assembly language is also running in another abstract machine.

                                              1. 7

                                                Yeah, modern CPUs do all of the following: (micro-)instruction buffering, out-of-order / parallel scheduling, branch prediction, speculative execution…

                                                Assembly language is definitely not how the machine works anymore, and that’s how we end up with Meltdown and Spectre.

                                              2. 4

                                                Hasn’t that been true since microcode was invented? ;)

                                                1. 2

                                                  It’s differently true today… back in the day, reading assembly was informative in a way it isn’t now that a conditional branch can take zero cycles if it goes one way and many dozen cycles if it goes the other way. Assembly is still the most machine-like language we have, but reading it gives a much less complete picture of what the code does.

                                                  It’s so difficult to read assembly and understand that one read’s very likely to be in L1 or L2, while another read is likely to go to main memory, and when it happens the delay will impact the next 50 instructions. Or that when a conditional branch goes one way the CPU will already have prefetched a lot, and the next 25 instructions are already being executed, whereas when it goes the other way the next instruction will take longer to start than those 25 to finish.

                                                  1. 1

                                                    We might need tools that explain it to us based on the CPU from several, optional perspectives. “Click expand or type this to see more.”

                                              1. 1

                                                Awesome, thanks a lot for linking that! It looks like something I am hoping to be able to use since long ago already (so much that I did something myself), probably counting from the day I watched Bret Victor’s “Inventing on Principle” talk. I have huge hopes that https://luna-lang.org will be the killer app in this space, but until they get their performance and stability under control, userspace looks like it may be a great intermediate solution. Now I just have to find out how to boot it…?

                                                1. 2

                                                  I was thinking of Ultimate Plumber when watching this video. Great utility.

                                                  1. 1

                                                    Huh, thanks; I’m a bit moved!… I’m constantly amazed when people from around the world seem to know it and like it :) Userland looks like a definite step up; I would love it if I found out it contains enough functionality of UP that I could sunset it and just redirect people to Userland! :) but first I must get to running Userland through Nix so I can check it out.

                                                  2. 2

                                                    On Arch Linux, this should work:


                                                    On another Linux distribution, replace the first line with whatever is needed to install Love2D.

                                                    Note that in its current state it’s more of an early prototype than a finished product!

                                                    1. 1

                                                      Hmmm; I’m stretching it, I know, but you don’t have a .nix script for it, by chance? :)

                                                      1. 1

                                                        No, sorry. Maybe you can write one ;)

                                                        (But otherwise, I would probably wait until Hisham at least documents this.)

                                                  1. 31

                                                    Missing: “After prototyping it in Python, I re-implemented the same algorithm in C using pre-existing, OSS libraries (eg set manipulation). The resulting executable had no dependencies. It ran faster than the Python version with near-instant startup.”

                                                    1. 8

                                                      Maybe there needs to be a stronger standard library for C tooling in Unix, because the amount of work it would require to pull off something similarly durable in C seems not worth it for somethg like listusers.

                                                      Not to mention that C’s abstraction ceiling is super low, so either you get to mess around in pre-processor soup, or you end up with a huge several-hundred line thing again instead of sub-100 line scripts.

                                                      I think it’s hard to underestimate how much pomp & circumstance is required to write “clean and safe C”. Personally I’d rather these kinds of scripts end up being Python scripts (with some effort made to make Python itself boot up faster for example)

                                                      1. 2

                                                        “Maybe there needs to be a stronger standard library for C tooling in Unix”

                                                        People have said it for a long time. That said, C programmers are already used to putting more work into their tools. Look at what the Python tool replaced.

                                                        “or you end up with a huge several-hundred line thing again instead of sub-100 line scripts.”

                                                        This is getting back to the point. That’s sub-100 line scripts calling a whole interpreter and standard library for it. Now, compare that sentence to a C program calling some C libraries. You’ve added less stuff to your system. You also potentially have libraries to re-use in other areas. I’m not saying they will, though.

                                                        1. 2

                                                          I’m not a heavy C user, but from what I can see, the closest thing to a “stronger standard library for C tooling in Unix” is glib. Not that it’s anything like standard, but it provides a lot of the things you’d need for this kind of program.

                                                          1. 2

                                                            I don’t think there’s a hope for C to get a bigger (cross-platform) standard library. It’s still busy adopting changes from 1999, can’t even agree on project layout or build system, and any C additions not in C++ are veto’ed by MSVC.

                                                            The easy way out is to write these tools in Go (huge stdlib) or Rust (huge crate ecosystem). You get comparable performance and startup time, plus high-level language features that make writing utilities quicker.

                                                          2. 1

                                                            If they published the source code for both versions of the tool along with the article, there would already be plenty of implementations in C, Python, Haskell, Rust, etc competing for performance.

                                                            Since they don’t, it isn’t really interesting.

                                                          1. 4

                                                            I use tarsnap, both on my laptops and my VPS.

                                                              1. 3

                                                                Keep in mind that maybe half the people using Lua are using it inside of some other system that may already have its own package manager (or strong opinions about packaging that don’t apply elsewhere). It’s designed for embedding as well as standalone use.

                                                                1. 1

                                                                  so whats a good way to deal with that problem? I need HTTP client and I dont think its built in - so i need to install lua-http or lua-curl or similar

                                                                  1. 2

                                                                    Lua libraries fall into two categories: single-file, zero-dependency (in which case I just plop it in the repo for the application I’m using) or libraries with really complex builds that interop with C code, in which case I use apt-get for them because they tend to be very mature and stable, and I never need the latest version. In your case luasocket falls into the latter camp, and it ships with an HTTP client.

                                                                    1. 0

                                                                      luasocket doesnt appear to be useful for larger files


                                                                      1. 0

                                                                        I wonder if this is a platform-specific issue or something; I can’t reproduce any problem with the snipped you pasted using the 100MB file.

                                                                        1. 1

                                                                          It doesn’t have progress…

                                                                    2. 1

                                                                      You can run apt install lua-curl.

                                                                      1. 1

                                                                        Cygwin doesnt offer a lua-curl package.

                                                                  2. 1

                                                                    It looks like you are trying to use LuaRocks on Windows with some “non-classical” setting.

                                                                    I haven’t used Windows for about a year, but last time I did https://github.com/Tieske/luawinmulti was the best option. LuaRocks 3 supposedly makes it better but there may still be issues because afaik none of the core devs really uses Windows…

                                                                  1. 2
                                                                    • OS: Arch Linux
                                                                    • DE: Gnome 3
                                                                    • Editors: Sublime Text 3, nano
                                                                    • Browser: Opera
                                                                    • Music: mocp
                                                                    • Terminal: rxvt-unicode
                                                                    • Shell: Bash
                                                                    • Password manager: Pass

                                                                    Web-based software I use daily:

                                                                    • Email: FastMail
                                                                    • Bookmarks: Pinboard
                                                                    • Lots of stuff: GitHub
                                                                    1. 1

                                                                      PS1="[\W \A] "

                                                                      Answers the two crucial questions: where and when?

                                                                      1. 8

                                                                        luarocks isn’t like npm, where you install dependencies via dependencies

                                                                        What does this mean? When you install a rockspec, it installs its dependencies. If you only want the deps (e.g. for local development), you can luarocks install --only-deps. Since version 3, LuaRocks supports local directory development à la npm with luarocks init. What is missing?

                                                                        (I know and use both package managers.)

                                                                        1. 2
                                                                          • Java did interfaces right, unlike a lot of class-based OOP languages.
                                                                          • PHP let a lot of people write dynamic webpages, and its typical request isolation model is still a very good design.
                                                                          • Subsets of C++ can actually be perfect languages for some use cases, and the modern evolution of the standard is impressive.
                                                                          • For JavaScript it is kind of the same: ES6 has made it much better than it was. And of course, it’s everywhere.
                                                                          • Ruby has a great, simple syntax, and its community has really pushed the Web back-end standards forward.