1. 5

    Lobste.rs needs a Julia tag. The language has been around for some time now and it doesn’t seem like it is going to vanish.

    1. 1

      Yes, please. Also an R tag, if I may – there is a lot that the programming-languages-for-building-programs can learn from the programming-languages-for-(interactive)-data-analysis, especially if regarding language and library UI/UX.

    1. 2

      I don’t think Julia is going to beat Python. Python just has so much momentum. Julia seems destined for HPC usage and not much else, imo.

      Multiple dispatch is a concept that I struggled with initially, but the more I learnt about it, the more I found it unwieldy and subpar. Reading julia code is cognitively intense. It doesn’t have the same clarity as python, and it doesn’t have enough guarantees to convince a Typing module user to give up on python. I can’t see myself trying to teach recent graduates how to do things in Julia, but Python has a great track record of being powerfully, succinctly taught to new programmers.

      Additionally, as performance goes, I don’t think Python is in as much trouble as it may seem. Mypyc is a very promising project, and with more work and love, it could lead to extensively enhanced Python performance.

      Anyhow. I’m betting on Python for the next ten years in STEM. Maybe someday, Ocaml.xyz will save us all from dynamism in sci-* code.

      1. 5

        Why does it have to beat it? It is an alternative to, and alternatives are good. I use Python and Julia everyday. The first because of all the packages that exist, and the second because I prefer its Matlab-like syntax for matrix manipulation. Also, I don’t think that class based object oriented syntax (like Python’s) is suited for scientific programing in general but that is more of a personal preference/opinion. Anyway, both languages are FOSS and independent from big corporations so I think people should be grateful for having the possibility of choosing either. Some years ago people almost had to use Matlab because there weren’t any good alternatives to it.

        1. 2

          For lots of users, including Lyndon (the author of that blog), the main benefits of Julia are not related to speed.

          Instead, they value Julia’s expressiveness (the base language is great, and we also have very fancy things like Cassette.jl) and tooling (Pkg.jl is the best package manager I’ve ever used, @code_native is great, etc).

          Obviously it’s fine that you don’t like multiple dispatch, but I think it makes an awful lot of sense for many people. Personally, I’m a big fan of it and of the way it makes all our classes open.

          Re: typing, compile time errors are neat, and they’re something that most Julia developers don’t benefit from at the moment, but the flexibility of Julia means that this can change. You can already use a Julia package to type check your functions (it uses Julia’s compiler) and there is work afoot to let us compile Julia programs ahead of time to on-disk binaries (we can already compile a subset of Julia to binaries for GPUs).

        1. 3

          That was fast. I didn’t understand the point of this at the time anyway, we already have Python and Julia for this area.

          1. 3

            It is always an over-ambitious project with the real possibility to fail. The “innovation” is to have autodiff done within the compiler so every program can be differentiated with the compiler. In that scenario, libraries like TensorFlow won’t need to handle autodiff at all, they just need to provide accelerated computation on tensors.

            That has been said, maintaining a fork of a compiler toolchain with a fast-moving upstream is risky. Considering Swift still in the progress of adding language constructs such as async / await, maintaining S4TF fork must be of very high cost.

            Maybe, just maybe, if Swift supports a stable compiler plugin interface, S4TF can co-evolve with the mainline, and there will be a lot of community participation. Personally, I never wanted to use S4TF toolchain because I had other development in my local environment and would love to keep it in sync with the latest mainline toolchain.

            1. 2

              I won’t pretend to understand it fully, but it looks like a very complex language extension that’s only useful in one specialized area (machine learning, maybe some other parts of mathematical computing.) So it doesn’t seem like a good fit for a general purpose language like Swift.

              1. 2

                Julia is a pretty fledgling language, even at this point. The beginnings of differentiable programming in Swift are a lot earlier.

              1. 10

                This is all very cool, but it is 2020 and we still can’t share mathematics through the web (as well as gopher, and gemini for that matter) without using images, scripts or SVG. Considering that the web was envisioned as a way for academics and scientists to share knowledge, this is saddening. We should already have a lightweight way of sharing maths by now, readable using text only tools such as Lynx or w3m.

                1. 3

                  a lightweight way of sharing maths by now, readable using text only tools such as Lynx or w3m.

                  How would this work in Lynx, because terminals have very limited graphical drawing capabilities, so more complex equations would be difficult or even impossible to render in Lynx, I think?

                  1. 2

                    You can use KaTeX, not requiring Javascript when pre-rendered and using math fonts.

                    1. 2

                      I agree, it’s a sad state of affairs that the best approach for math on the web is to add JavaScript which parses and renders latex.

                      1. 1

                        why not store the rendered product and serve it directly

                        1. 2

                          It’s possible obviously, but it sucks that the web has no native support for math.

                          I can’t, for example, copy/paste parts of an <img> tag. I can’t, easily and safely, let users in a comment form write math expressions. The web’s lack of support for math is, I assume, one of the big reasons why places like lobste.rs and reddit don’t support math expressions in comments. If MathML could’ve been relied on, I’m sure someone would’ve popularized a Markdown-flavor with math support already.

                      2. 1

                        You have multiple options:

                        1. Use PDF/LaTeX. These can have complex formatting, hyperlinks, and equations.

                        2. Use MathML, the standard for equations in HTML.

                        3. Use unicode characters, like tex-conceal.vim does. I don’t recommend using more than a tiny amount of this.

                        1. 6

                          Use PDF/LaTeX. These can have complex formatting, hyperlinks, and equations.

                          This is not web.

                          Use MathML, the standard for equations in HTML

                          Only Firefox renders MathML, and the page size gets huge very fast.

                          Use unicode characters, like tex-conceal.vim does. I don’t recommend using more than a tiny amount of this.

                          Not good enough when we want to write complex formulas with a lot of symbols.

                          1. 1

                            Yeah, I agree that all three options are far from ideal. But those are our options that don’t resort to scripting (that I’m aware of; please LMK if there are others).

                            Websites like Wikipedia seem to use MathML when it’s available, falling back to images. This approach seems to be the best available.

                      1. 2

                        I wonder how pandas’ CSV parser (which is pretty optimized) compares. Whenever I have to parse huge CSV files in Python, I use pandas just for that.

                        1. 3

                          I’ve never benchmarked pandas in particular, but have loosely benchmarked Python’s CSV parser. The inherent problem is measurement. What is your benchmark? Let’s say your benchmark is to count the sum of the lengths of all the fields. Well, that means Python will need to materialize objects for every record and every field. And that is probably what’s going to either dominate or greatly impact the benchmark, even if the underlying parser is written in C and could theoretically go faster.

                          Pandas’ CSV parser is written in C, and if the comment at the top is true, it’s derived from Python’s csv module. Like the csv module, Pandas’ CSV parser is your run of the mill NFA embedded in code. This is about twice as slow as using a DFA, which is what my CSV parser uses. And the DFA approaches are slower than more specialized SIMD approaches. I’m less sure about the OP’s approach.

                          1. 2

                            Thanks! Love Ripgrep! I tried cargo build --release and time ./target/release/xsv index /tmp/DOHUI_NOH_scaled_data.csv, it took about 24 seconds for index to complete (I assume xsv index find all begins / ends of all cells, which approximately is what I am trying to do here for csv parsing).

                            Didn’t do xsv entirely due to my unfamiliarity to Rust ecosystem. Sorry!

                            1. 1

                              Thanks. How do I run an equivalent benchmark using your CSV parser? I don’t think I see any instructions.

                              1. 1

                                It is not packaged separately, and ccv can be built with zero-dependency (meaning you may not have OpenMP enabled) so it is a bit more involved to make sure OpenMP is enabled.

                                You can first install apt install libomp-dev clang, and then checkout https://github.com/liuliu/ccv repo. cd lib && ./configure to configure it with OpenMP (there should be a USE_OPENMP macro enabled, configure script should give you exact output of flags). cd ../bin/nnc && make -j would compile the demo csv program under ./bin/nnc

                          2. 1

                            Recently the guys from Julia started claiming that they have the fastest parser (link).

                            1. 4

                              It kind of looks like Julia’s CSV parser is cheating: https://github.com/JuliaData/CSV.jl/blob/9f6ef108d195f85daa535d23d398253a7ca52e20/src/detection.jl#L304-L309

                              It’s doing parallel parsing, but I’m pretty sure their technique won’t work for all inputs. Namely, they try to hop around the CSV data and chunk it up, and then parse each chunk in a separate thread AIUI. But you can’t do this in general because of quoting. If you read the code around where I linked, you can see they try to be a bit speculative and avoid common failures (“now we read the next 5 rows and see if we get the right # of columns”), but that isn’t going to be universally correct.

                              It might be a fair trade off to make, since CSV data that fails there is probably quite rare. But either I’m misunderstanding their optimization or they aren’t being transparent about it. I don’t see this downside anywhere in the README or the benchmark article.

                          1. 12

                            One of the smaller pain points on the Mac is that there’s no built in paint tool. I want to use it just often enough that I miss it, but not enough that I want to learn what’s out there and/or pay money for a sophisticated tool.

                            I realize there’s some subjectivity to this, but I think a simple paint tool ought to be part of what’s comes included with every “normal” machine.

                            1. 5

                              I’m using Acorn for MacOS, after seeing a recommendation or two. It’s light, fast and easy (for me) to understand. It was also cheap. I don’t do much - just cropping, copying, pasting, drawing an arrow, etc.

                              1. 3

                                I also endorse Acorn, it’s a great tool. One under appreciated aspect of it- you can open an image, modify it, command + s and it saves it over the original file. No “export as” or “save for web” nonsense.

                                1. 1

                                  Boxy SVG is really nice and free.

                                  1. 1

                                    This one? https://boxy-svg.com/

                                    Looks great, but not free to use (15-day free trial is available though). Restricted version is available for free on Linux, but only snap (which I don’t want to use)

                                    1. 2

                                      That’s the one. I guess I’ve just been so used to using it that I completely forgot it wasn’t free.

                                      I’m using the Mac version. I guess since I got it from the MAS and didn’t have to go through the license activation rigamarole, it was easy to forget.

                                      Sorry about that.

                                      1. 1

                                        sure no problem, the suggestion is still a good one :)

                                        1. 2

                                          No worries. I’ll definitely be more careful about double checking things from a while back or qualifying them.

                                  2. 1

                                    I second Acorn recommendation, although it’s not really a Paint clone. It’s more like a “subset of Photoshop that a Paint user would recognize”.

                                  3. 4

                                    there’s definitely money in it! after trying a bunch of paint alternatives I settled on (not even joking ) renting a $5/mo Windows compute server from Azure for the sole purpose of using MS Paint…

                                    1. 1

                                      You can run MS Paint using Wine. Super easy to install with WineBottler too.

                                    2. 3

                                      If your goal is to draw over a screenshot or an existing picture, you can use Preview for this. It’s not exactly a “dumb” image editor like MS Paint but it has a couple of additional features like recognizing the shapes you draw by hand. It’s very convenient for annotating screenshots if you use “File > New from Clipboard”. Otherwise you can use Pages or Keynote as a vector graphics editor.

                                      1. 2

                                        I’ve used Preview for existing screenshots, but I’d also like to be able to start from scratch and scribble.

                                        I didn’t realize that Pages and Keynote were free until just now, but they’re not built-in, so they still fail one test. I’d also suggest that using a document editor to do scribbling/painting is a little surprising and not the thing that comes to mind so easily.

                                        1. 1

                                          Mmm, I don’t think they have a mac flavor, but the best thing for screenshots is flameshot.

                                      2. 2

                                        I’m with you on this. Both macOS and Linux need a good Paint clone. The other way I battled for hours with Inkscape to do a basic image manipulation with cutting and pasting, something that would take seconds on Paint. Not because Inkscape is a bad application, but rather because I haven’t learned it and don’t have the need to do it for my daily tasks.

                                        1. 4
                                          1. 2

                                            kolourpaint.

                                            1. 1

                                              I wouldn’t use inkscape, it’s more of a vector thing, I use GIMP for images. It doesn’t matter though because I think your point still stands, the learning curve is steep.

                                              The closest I have come to something simpler out of the box is Krita.

                                            2. 1

                                              I had the same complaint, and apparently I installed this: https://paintbrush.sourceforge.io/. Although I can’t remember when I last used it.

                                            1. 1

                                              I would like to find the energy to do something like this, but between all the devices and different platforms it feels like the biggest shore ever and I give up every single time.

                                              One thing that bothers me with the approach discussed in the blog is that the writer is collecting URLs disregarding that most of the stuff in the web can disappear in the future. Sometimes I save whole web pages (when I’m on my computer) that I think are really important, but it would be nice to have a solution that tackles this issue. On any device, for every platform.

                                              There are simply too many information sources today, it feels overwhelming.

                                              1. 7

                                                A bit offtopic, but the choice of name is unfortunate. There is a project since 2012 that uses that name.

                                                https://github.com/stan-dev/stan

                                                1. 4

                                                  That Stan is AFAIK the most mature and ‘production-grade’ probabilistic programming language and general-purpose Bayesian modeling framework. (PyMC 3 is another contender… most of the rest are academic toys.) So, yeah, it’s a not a great name to be shadowed by.

                                                1. 1

                                                  Haven’t read it, yet, but I have to say that your website is beautiful.

                                                  1. 1

                                                    Thank you, I spent a lot of time thinking about how best to present the material. My inspiration is the PDF format, but for the modern web.

                                                  1. 14

                                                    It’s missing a crucial point:

                                                    • Be ready to give up good HiDPI screen support

                                                    I have a 4K monitor because it’s much more enjoyable to use for development, but it becomes a pure nightmare on Linux. I ended up installing an Hackintosh.

                                                    1. 8

                                                      Not necessarily. You can look at this as purely anecdotal, but my Dell Precision 5510, which I’ve had for about four years, has a HiDPI display on par with what I have on the MBP I use for work. Maybe this is down to the fact that I run Ubuntu on it, but I’m just using stock Intel and Nvidia drivers on it, depending on my use case.

                                                      1. 3

                                                        Just curious, what makes it a nightmare?

                                                        1. 11

                                                          My most recent encounter with this is fractional scaling:

                                                          I have a 27’’ 4k monitor. I find that at this resolution, 1.5x scaling works best as 2x is too big and 1x is too small. With MacOS or even Windows, this is not a problem at all, with Linux it’s a can of worms:

                                                          • Xorg doesn’t natively support fractional scaling, Instead you have to rely on hacks or only scaling the fonts (which quite frankly, looks like shit). I never managed to make any of those hacks work reliably and consistently in all apps, eventually you’ll open that one app that uses Qt and you require yet another hack.

                                                          • Wayland does support fractional scaling, too bad that not all apps support Wayland, most notably Firefox. I tried running Firefox with the experimental Wayland backend with fractional scaling on, and everything looked blurry, it’s just not there yet.

                                                          Now, I’m sure for all of those issues there are 10 different workarounds to try and things to tweak to make things better, but I can’t be bothered to do any of that when in macOS (or even Windows) it Just Works™.

                                                          1. 8

                                                            Just the fact that the ArchWiki page on HiDPI has a comprehensive list of required hacks is another example of what I’m referring to.

                                                            1. 3

                                                              Have you tried with KDE? I’m curious because I’m planning on using HiDPI with 1.25x scaling.

                                                              1. 2

                                                                Yes, with KDE it’s slightly better but still not as good or smooth as Windows and macOS in my opinion.

                                                              2. 2

                                                                Latest versions of Gnome in Ubuntu 20 have fractional scaling and it’s been set and forget for me.

                                                                1. 2

                                                                  Ubuntu ’s fractional scaling is using significant more CPU power compare to 2x scaling.

                                                            2. 3

                                                              I tried switching to Ubuntu recently (not for the first time) and ended up going the Hackintosh route, too. For me the breaking straw was not being able to adjust mouse wheel scroll settings in a way that would work everywhere and didn’t seem to come with caveats or be labelled as a hack.

                                                              Setting up a Hackintosh certainly wasn’t without its hassles, but having got there I’m very happy with it. I also have a Macbook and an iPhone, so that is another motivation to stay in the Apple camp (vendor lock-in, I guess?)

                                                              1. 1

                                                                I’m surprised you had mouse wheel woes. Do you have a special or fancy mouse or something? Or want very specific wheel behaviour?

                                                                1. 1

                                                                  I just have a normal mouse, and TBH I’m a little fuzzy on what happened now. It could very well have been that if I has been using Gnome (or KDE, whichever one I wasn’t using) then it wouldn’t have been a problem at all, but what I really wanted was MacOS anyway, so I just did the Hackintosh thing instead. I kind of took “well I can’t get this basic thing to work right” as an omen.

                                                              2. 3

                                                                I use Xubuntu 20.04 on my X1 carbon and desktop with a 27” 4K monitor. Both work totally fine with HiDPI. The main issue I’ve had was when I plugged my laptop into non-HiDPI monitors, I had to lower the DPI for the monitors and that made everything on the laptop small.

                                                                1. 1

                                                                  The distros the article mentions are ElementaryOS, Pop!_OS, and Fedora Workstation. I believe all of those have had great native HiDPI support “out of the box” for a few years now.

                                                                  I totally get not loving the aesthetic of GNOME or whatever, especially bumped up at x2, but I think it’s cool that Linux lets us customize everything end to end to our heart’s desire. If we’re going for a more custom setup, totally agree that it can require doing some custom tweaks to get a good HiDPI experience and it won’t be completely automagic.

                                                                  I don’t think it’s fair to say it’s all a pure nightmare on Linux as a whole. Beginners can have a good experience, experts can have a good experience, crossing that valley can be painful.

                                                                  Also worth noting that most of the ArchWiki page on HiDPI refers to outdated workarounds that are no longer required, though not all.

                                                                  1. 0

                                                                    Honestly, yeah. I purposely bought a normal DPI display just to avoid this pain. It’s never gonna be perfect. Apps aren’t gonna scale right, even with fractional scaling. Other apps might scale right, but be blurry. It’s gonna drive you crazy.

                                                                    Just get over the loss aversion, buy a good normal DPI monitor, and get back to work. I’m 100% satisfied with normal DPI.

                                                                  1. -1

                                                                    It’s Dendrite based, so I have doubts about the sustainability due to New Vector’s constant changes on Synapse breaking compat with everyone else; another sad manifestation of the perverse incentives preventing Matrix from being a stable and performant alternative.

                                                                    1. 22

                                                                      Wow, this is some quality FUD. We haven’t broken backwards compatibility ever on the Client Server API - you can spin up a Matrix client written pre-launch in 2014 and it should work fine with a 2020 Matrix server. On the Federation (Server-Server) API, we upgrade the room synchronisation protocol on a semi-regular basis using room versions (https://matrix.org/docs/spec/#id5)… and we’ve still kept backwards compatibility with the earlier room versions.

                                                                      For wider context: New Vector (NV) is the company set up by the original Matrix team which sponsors much of the development of the core Matrix project. The accusation here is that New Vector is somehow incentivised to sabotage development of Synapse (the reference Matrix server) to prioritise its own commercial interests at the expense of the wider Matrix network. This is categorically untrue. All work that NV people do on Synapse is donated unilaterally to the Matrix.org Foundation, which is governed incredibly strictly (including by the UK non-profit regulator) to ensure the protocol and reference implementations advance for the benefit of everyone - without ever prioritising any commercial interests, particularly NV (now or in the future). https://matrix.org/foundation spells this out.

                                                                      So: a) Synapse doesn’t break compat; b) NV doesn’t prioritise itself when doing Matrix.org development, and if it did, the Foundation would course-correct; c) Dendrite work is mostly funded by NV anyway :/

                                                                      1. 4

                                                                        This is the impression I got from observing the Matrix project, Dendrite’s progress specifically over Synapse, and third-party implementations. The biggest pain point is how checking off features seems to be prioritized over actual polish and optimizations to the servers themselves.

                                                                        1. 6

                                                                          I think you are extrapolating incorrectly. Almost all work in Synapse over the last year or so has been around polish & optimisations - and Dendrite dev was on hold in order to focus on Synapse stability & perf. In the last few months we’ve been able to afford to spin up Dendrite dev again, and meanwhile Synapse is (for once) stable. We’re almost at the point of going back to feature dev - but other than E2EE-by-default, there’s been very little feature dev since 2018; it’s all been about stabilising for Matrix 1.0 (in June 2019) and then subsequent polish.

                                                                          I think you might be basing your conclusions on where things were at in 2018 or so (when we had indeed been rushing features out the door in order to try to secure long term funding for the project, and to try to keep up with the centralised competition).

                                                                        2. 2

                                                                          What % of Matrix development is funded/driven by New Vector? It’s cool that they are pushing for this kind of software.

                                                                          1. 3

                                                                            NV is the startup which the original Matrix team formed in 2017 in order to fund themselves to continue working on Matrix fulltime. Of the core Matrix.org codebase, probably ~90% of it is written by NV employees, all of which is donated to the Matrix.org Foundation, which was set up as an independent custodian of the protocol itself (protecting it from NV and any other commercial players). The wider ecosystem building on Matrix spans hundreds (thousands?) of various projects, projects and contributors.

                                                                            So it’s not coincidence that NV drives a lot of Matrix development, given all but one of the team who created Matrix work there :)

                                                                        3. 6

                                                                          I can’t follow the reason for your criticism without further explanation. What breaking changes did Synapse introduce in the past? Who is “everyone else”? What kind of “perverse incentives” do you mean? Why should this be a problem for Dendrite, which is also developed by New Vector? Any sources?

                                                                          1. 0

                                                                            New Vector is basically paid to develop features over stabilizing and polishing the low-level details. This made federation somewhat of a moving target, and it means things like performance are moved to the wayside.

                                                                          2. 6

                                                                            It constantly amazes me that the only thing that a lot of people like to complain more than proprietary software is free and open source software that is done by people other than themselves.

                                                                          1. 3

                                                                            For someone starting with probabilistic programming, would you recomend Turing.jl or Stan.jl?

                                                                            1. 1

                                                                              Check out the #probprog channel on the Julia Slack and ask there :)

                                                                            1. 8

                                                                              It is funny how we went from:

                                                                              • mainframes + terminals

                                                                              to

                                                                              • personal computers

                                                                              to

                                                                              • personal computers + internet

                                                                              and are finally getting to

                                                                              • cloud™ + terminals

                                                                              I’m not sure I like it…

                                                                              1. 6

                                                                                u/ddevault I’m not particularly fond of using a circle as sourcehut’s icon, but even ignoring that, the fact that the text is not centered with the circle is triggering the OCD in me really hard. In this image, we can see that top partition is larger than the bottom partition.

                                                                                1. 5

                                                                                  I have a really bad eye for this… would you mind terribly if I imposed on you for a patch? The code is here:

                                                                                  https://git.sr.ht/~sircmpwn/sourcehut.org

                                                                                  hugo serve -w will get it running. The stylesheet is in assets/main.scss, and the HTML is in layouts/partials/nav.html. I can backport the fix to *.sr.ht after that. I spent a few minutes messing with it but I can’t really tell when it’s right.

                                                                                  1. 3

                                                                                    Would you mind if I would also give it a go?

                                                                                    1. 4

                                                                                      Scyphozoa already sent a patch - and it made it upstream. Maybe you’d like to try it for *.sr.ht, rather than sourcehut.org?

                                                                                      https://git.sr.ht/~sircmpwn/core.sr.ht

                                                                                      See srht/templates/nav.html

                                                                                      1. 3

                                                                                        Sure. Would you mind if I maybe tried something with the logo?

                                                                                        1. 4

                                                                                          By all means.

                                                                                    2. 1

                                                                                      Thanks for replying. I might try, but I have never written a single line of either html or css in my life :(

                                                                                    3. 1

                                                                                      fwiw, I liked it… encompassing

                                                                                      1. 4

                                                                                        I think it would be cool if he did some pun because of the word “hut”, maybe just using a caret symbol on top of the circle. I don’t know, just a circle seems bland, personally. And because it is a circle, at a glance it might trick you into reading “Osourcehut”

                                                                                        1. 3

                                                                                          sôurcehut (:

                                                                                    1. 13

                                                                                      What I find really interesting about this update is the public index. Before there was no way to explore sourcehut. It’s a feature I really enjoyed about github, until they completely crippled it.

                                                                                      1. 6

                                                                                        yeah, it even has a “Featured” column. It is a really clean design.

                                                                                        1. 3

                                                                                          Ooh, that’s nice! I love it. Add filtering and sorting, and this thing will have GitHub beat by a mile.

                                                                                          1. 5

                                                                                            Indeed; advanced search and the network effect complement each other as GitHub’s two killer features. I wrote about how effective these can be in a comment on Orange Website:


                                                                                            This is excellent. The great thing about Sourcehut is that if you have extra git remotes, you can keep working like nothing ever happened if the site goes down. Issues and patches are decentralized over email.

                                                                                            The Project Hub looks like an excellent way to tie together all the separate Sourcehut services to better compete with other “complete” VCS-based collaboration suites like GitHub and GitLab. In the future, it would be really cool to expose an API to allow adding “custom” services that aren’t part of Sourcehut.

                                                                                            It’s good to see Sourcehut focusing on project discovery, since this is the area where GitHub excels at the most. When I search for a small CLI/TUI utility, I often run these filters:

                                                                                            • filter out weblangs, frontend-oriented languages, and languages with heavy runtimes (JS, TypeScript, CoffeeScript, Vue, CSS, HTML, Dart, Purescript, Livescript, Elm, Swift, JVM languages, .NET languages, Vala, QML, etc.). I have several shortcuts for many combinations of languages so I don’t have to type them out every time.

                                                                                            • filter repos below a certain size (a repo above 10mb is probably full of bloat).

                                                                                            • If applicable, filter out repos whose last commit was before a certain date.

                                                                                            • If applicable, filter by topic

                                                                                            • If it concerns a recent technology, I can filter repositories created after a certain date.

                                                                                            • If I want to try a smaller project that isn’t cursed with mainstream success, I filter repositories below a certain number of stars.

                                                                                            For instance, if I feel like my MPD setup is missing something, I might search:

                                                                                            mpd stars:<150 pushed:>2019-01-01 size:<8000 -language:purescript -language:livescript -language:vue -language:javascript -language:typescript -language:coffeescript -language:elm -language:dart -language:java -language:scala -language:kotlin -language:clojure -language:groovy -language:php -language:objective-c -language:objective-c++ -language:swift -language:css -language:HTML -language:haxe -language:csharp -language:fsharp -language:"jupyter notebook" -language:powershell -language:cuda -language:assembly -language:tex -language:batchfile -language:erlang -language:elixir -language:emacs -language:vim -language:plpgsq -language:julia -language:xslt -language:systemverilog -language:verilog -language:hcl -language:tsql -language:jsonnnet -language:gdscript -language:r -language:smarty -language:freemarker -language:nix -language:saltstack -language:"visual basic" -language:"visual basic .net" -language:plsql -language:"rich text format" -language:dockerfile -language:vala -language:QML -language:actionscript -language:matlab -language:alloy -language:cobol -language:graphql -language:m4 -language:qmake -language:fish -language:opencl -language:json -language:rmarkdown -language:xml -language:markdown -language:applescript -language:puppet
                                                                                            

                                                                                            The result shows quite a few nice utilities. If I want to go even more minimal, I could filter out Ruby and even Python projects.

                                                                                            It would be great to have a FOSS implementation of an advanced project search utility that isn’t limited to (or even part of) any particular hosting provider. Maybe ActivityPub could help facilitate connecting and indexing project metadata from different hosting providers.

                                                                                        1. 35

                                                                                          This allows you to use them independently and compose them however you please, which maps well onto the reality of how many projects are organized. Compare this to platforms like GitHub, GitLab, Gitea, and so on, resources like bug trackers and pull requests map 1:1 to a git repository.

                                                                                          I think this is a great leg up on the competition. Having repositories be 1:1 with issue trackers always annoyed me on GitHub and results in hacky workarounds like sending issues between repositories or having one repo act as “the issue tracker repo”.

                                                                                          1. 4

                                                                                            This is awesome, specially for mirror repositories and stuff like that.

                                                                                          1. 2

                                                                                            As an aside, I often hear good things about Julia’s performance, but it’s unclear to me if that performance extends beyond numerical computing. Most applications don’t deal in crunching large matrices but rather in creating and manipulating many small objects. Garbage collection, allocation, and dispatch (especially in dynamic languages) tend to be the bottleneck; however, it has been hard to find information on how Julia performs relative to more mainstream languages.

                                                                                            1. 1

                                                                                              Is that relevant though? I don’t think the language offers tools to develop “normal” apps, so even before thinking about performance, using Julia for those tasks would make the programmers life really difficult.

                                                                                              1. 3

                                                                                                There are actually all kinds of “normal” libraries for Julia, [1] and [2] for example. That being said, I’ve always found it kind of silly that people want to write things like web apps in Julia. Learning a new language and a couple libraries is, IMHO, much easier than fighting with a language to make it do something it isn’t optimized for.

                                                                                                [1] https://github.com/JuliaGizmos/WebIO.jl [2] https://github.com/JuliaWeb/HTTP.jl

                                                                                                1. 1

                                                                                                  I think so. First of all, I think a package manager and a test framework go a long way with respect to tooling. Everything else is somewhat “nice-to-have” (formatter, debugger, profiler, linter, etc) and/or can be written from scratch for a modest amount of work. Libraries might not exist, but those can come with time (and many such important libraries probably already exist since data science still involves a lot of moving data over the network, compressing/decompressing, changing serialization formats, etc). Performance is the more limiting factor in my opinion.

                                                                                                  1. 1

                                                                                                    Julia is a pleasant, fast “lisp”-y language. It has nice support for computation, but unlike languages like R there’s not really anything that binds it to that use case. If you imagine a world where “time-to-first-plot” is solved, I don’t see why it wouldn’t be used for general purpose coding.

                                                                                                    That said, if you try to use it general purpose right now, you are trailblazing.

                                                                                                  2. 1

                                                                                                    Most applications don’t deal in crunching large matrices but rather in creating and manipulating many small objects.

                                                                                                    Mmm, I agree that most applications deal in creating and manipulating many small records; but whether those are represented as objects, or structs, or data frame rows is going to depend both on the language’s paradigm and the program’s design.

                                                                                                    Garbage collection, allocation, and dispatch (especially in dynamic languages) tend to be the bottleneck;

                                                                                                    Julia’s design has a few properties that allow its users to program in a dynamic style while Julia still creates machine code that run fast:

                                                                                                    • JIT compilation to native machine code
                                                                                                    • Just-ahead-of-time static analysis that will e.g. devirtualize function calls, specialize code for argument types at function boundaries, and even precompute constant expressions.
                                                                                                    • Users can specify types on function arguments to help the compiler specialize/de-dynamicize – one can specify memory types like uint64, but also user-defined types and struct types, etc.
                                                                                                    • Direct calling of C and Fortran libraries without glue code.

                                                                                                    Among the overheads you mentioned, that should take care of most of the function dispatch and value unboxing overhead.

                                                                                                    As for GC and allocation overhead: no idea. Julia has a non-compacting, generational, mark-and-sweep, tracing collector, and I can’t tell you what that means for performance.

                                                                                                    An interesting discussion of the tricks the language+compiler use: https://juliacomputing.com/blog/2019/02/19/growing-a-compiler.html . It rather like how the presentation doesn’t use benchmarks to demonstrate compiler speedups, but instead uses Julia’s built-in commands to render functions’ intermediate representations and compiled assembly.

                                                                                                    beyond numerical computing.

                                                                                                    Certainly possible, but also, I think you’d be pioneering? Julia’s current community mostly comes from the scientific computing and analysis field that is currently served by R, Fortran, and Python. On the other hand, web frameworks and Gtk.jl exist, and I found a pure-Julia OAuth module, and being able to directly call C libraries might also help you, so you could get probably get as far as you want to get.

                                                                                                  1. 1

                                                                                                    There is another one being developed https://github.com/denoland/deno