I have a Mac mini and my Mac OS setup is that it is my server of sorts. Trying to get as good as possible in understanding SSH and docker for that reason.
I have a very specific setup in Windows which I wasn’t able to replicate in Mac. My windows setup is Ditto (clipboard manager), AutoHotkey, Razer Synapse 3 (for setting macros) and the driver software for my 15 key MMO mouse.
I worked as a web scraper for more than 6 years. I did my fair amount of pattern matching to extract clean data.
For pattern matching don’t start with regex outright. Start with string slicing, stripping characters and splitting strings to manageable chunks. Then try the golden regex .*? [0]. If you are not writing system critical code, ignore what people say about this being greedy. In my opinion, if you are writing custom regex, you will find that .*? does what you need most of the times.
Then learn what the blog post is mentioning. It is a VERY good post, and absolutely echoes what I feel is true. The section [Ignore this stuff] is extremely true and I really like the fact they included regex groups here.
I am trying to “memefully” learn Rust. The endgoal is to be competent enough to create a Tauri based GUI software.
For the “memefully” part the goal is to just enjoy the learning process as much as possible. Rust is a hard language, we all know that. So, I plan to write something in Python, then in Nim and then in Rust. Then write a blog about it. I plan to follow the Rust CLI book [0], however I want to do random projects. This weekend’s plan is to write a CLI that takes makes async get requests to a bunch of input URLs.
async Rust feels a bit more like C or C++, where performant and versatile take top rank, and one has to have a lot of experience to know how to avoid sharp edges.
I recommend considering using ureq instead, especially (but not only) if you still are learning Rust and you see Rust as hard.
The costs of async are worth paying, if you’re writing an HTTP server that must serve many many clients with minimal overhead. However, for HTTP clients, we believe that the cost is usually not worth paying. The low-cost alternative to async I/O is blocking I/O, which has a different price: it requires an OS thread per concurrent request. However, that price is usually not high: most HTTP clients make requests sequentially, or with low concurrency.
For several years I thought that Rust was too hard and didn’t even attempt it seriously. Then last year I started the Advent of Code 2022 in Python (that I use professionally), and found this masterpiece from Amos: https://fasterthanli.me/series/advent-of-code-2022/part-1
It was super refreshing as it was showing me how close to Python basic Rust can be, but also with many additional features that make it amazing.
I was able to do about the first 13 days in Rust thanks to that series (I was solving it before looking at the solution from Amos). I must say that it was an eye opener to me. I really recommend this series.
Just curious— how did you end up with 159 KB with Rust? I have tried cargo build --release and rustc -C debuginfo=0 -C opt-level=3 main.rs and they both result in 508 KB. If I strip it, it becomes 333 KB.
I’m on rustc 1.72.0-nightly (d59363ad0 2023-06-01), if that’s relevant.
As a self proclaimed “Python Developer” my rule of thumb is that, if the codebase is not super complex, was written by you and you are strapped for time just use Pypy. If it was written by someone else, you will probably find a fair few optimizations by going through the codebase first.
That’s it really. If you want more performance out of a Python, just rewrite that in a Go, Rust or even Nim. I feel like any kind of syntactical changes to a Python codebase to adopt a python flavor optimization framework is not worth it.
Yeah, it’s unfair that frontend development keeps getting criticized for being fad oriented. I do think it’s accurate. But I don’t think it’s a skill issue or worthy of shame. The improvements are superficial and there’s alot of churn simply because the problems haven’t been solved yet. In comparison to many backend tasks:
Browser and device/resolution compatibility is too complex to be addressed simply. Big strides have been made, but it’s hard to see tooling as anything but frustrating complexity when you haven’t struggled with the problem it solves. Most people’s personal projects don’t reach the point where they can’t ignore those problems anymore.
Human factors like accessibility, design, ux, language/internationalization, and “the-user-is-a-drunken-toddler” are omnipresent. Not that they don’t exist elsewhere too, but they aren’t an implicit constraint in everything. The elegant solution you have always falls over when there are that many competing constraints.
The realm is still largely driven by enthusiasts compared to backend web dev. This means fashions are natural as we experiment with new solutions. It also means many projects are on more equal footing. Open source tooling for backend may start non-commercial, but because large frameworks almost always solve the problems of scale or enterprises they inevitably become commercial. This means there’s a vested interest in slowing down experimentation (and therefore fashion) for stability or to keep the money flowing.
Being sandboxed into a browser, being reliant on backend apis, and having non-opt-in security means the list of “default” features you need is massive. We don’t even think to call them out anymore…they’re included in bundles or defaults in generated boilerplates. You can’t really roll your own for everything or opt out of certain things like you can when you control your system’s universe/environment. You can’t really even isolate things in the same way, though it’s worth trying.
People are largely unsatisfied with it. Open problems lead to more attempts at a solution. Not that there aren’t open problems on the backend, but the everyday humdrum widely-applicable problems at least have a solution that generally works.
That’s not to say things shouldn’t improve. The writer of The Grug Brained Developer is also the developer of HTMX which is an important attempt at improvement. But I hate that it’s always framed from the viewpoint of “we’re not like those other solutions” when they exactly are: it’s an experiment to tame complexity and make frontend dev either easier or produce actually functional software. I support this fashion because I like it, but there’s nothing more legitimate about this particular fashion.
This was quite insightful. So, with frontend programming, there isn’t much in terms of massive problems, and they have limited areas to innovate.
Are there any other fields like that in programming? Suppose business software programming language developers (for example - VBA) don’t have much to innovate because most of the development ecosystem is out of bounds for them.
It sounds pretty weird to think that frontend development is widely popular yet they have some odd artificial restrictions that are imposed on browser design and “good SEO practices”. So, quite oddly innovation here is usually just about personal preferences and not about universal utility. Which is literally what fashion means.
“we’re not like those other solutions” when they exactly are: it’s an experiment to tame complexity and make frontend dev either easier or produce actually functional software.
That’s a bit reductive, no? htmx’s purported uniqueness is in its technical approach: instead of taking inspiration from native GUI toolkits and having client-side JavaScript build UI (with SSR added on top), htmx builds the plain “links-and-forms” model of the web and has the server drive the app by sending HTML responses that encode the possible interactions.
We’re all trying to make web dev better. What else would we do? Make it deliberately worse?
I don’t think so? But the question makes me think I was unclear. I fully support and even prefer htmx’s approach. I like the drive towards simplicity. Moving functionality to an area with fewer limitations and centralizing logic to minimize moving pieces is a great way to do that. I have 2 live projects using it and it does what it promises.
My point wasn’t that htmx isn’t valid. It’s that it’s equally valid as an experiment in how to build frontend software. Since there hasn’t been a clear pattern that has emerged as the “best” way to tackle things, I don’t like the chastising of other approaches when they are simply making different tradeoffs. Many problems haven’t been solved yet. It’s reductive to those problems that people are trying to solve. We can talk about the pros and cons of approaches and make tradeoffs without shaming the industry, and we can encourage more experimentation.
We’re all trying to make web dev better. We should be deliberately experimenting. Not writing off entire ideas as “fad based” in an attempt to frame one as separate from that social constraint. My gripe isn’t towards htmx at all, it’s at the general treatment of frontend development as broken rather than “in progress.”
I’ll just repeat what I was saying on IRC channel about that: in my experience that line between “straightforward” to “clever” varies greatly not just between projects, but between teams and areas and moon phase too probably.
The judgmenent “will a junior dev understand it?” is somewhat questionable; like normal people they all come in different shapes and forms and to grow up they have to see some clever code and algorithms and research it and improve. Say you implement some relatively tricky part of your codebase and just describe it as “We use observer pattern here” or “that’s your run-off-the-mill Dijkstra” and said junior might not know what all that is. That is still not a reason to avoid solutions like that;
Then again I don’t approve of “clever code” in shape of “clever one-liners”, when stuff is not essentially more effective but just shaped to be crammed in very little symbols amount. There is ioccc fot that kind of elaborate insanely cool stuff :D
This is not “production-ready” code, though. This is recreational code.
The author explicitly mentions this however they connect recreational code to clever code. In programming quiz sites, there is a sense of joy in coming up with solutions that only focuses on performance and briefness of characters. Moreover, when you see other solutions and you try to figure out how it works, you feel same joy. Hence “recreational programming”.
Production code doesn’t demand that kind of concentration or adventures from employees. Code maintenance is prioritized above all. However, coming up with one liner is kinda fun. As long as you provide enough context, comment and that stackoverflow link, I think it is fair to write oneliners.
As long as you provide enough context, comment and that stackoverflow link, I think it is fair to write oneliners.
Only if it brings a benefit over the more mundane approach. Otherwise, why require the user to follow that stackoverflow link and load up on all that context?
Even the “run of the mill Dijkstra” can be implemented in “clever” ways and straightforward ways. Also, it would help to name the algorithm (perhaps with a reference) in the comments, so the junior dev can look it up and understand it that way.
Yep, that’s what I meant! Even tiny comments can help with understanding seemingly “tricky” code parts.
Overall my thought is that the baseline for the code compexity is vastly different for diff. people and teams, and what is considered excessively “clever” for some people/teams/projects might be just an average for others. Otherwise we can lower the plank to the “will any random non-programmer understand this?” and just do everything in BASIC on Commodore64 :D
In Japanese sumo robot fights [0] the arena is covered in a sticky substance to improve traction. I wonder if the rules allow for a substance like this to be used on the tires.
Aside from work and work, trying to figure out whether I should stick with the tried and true, threading-based approach for running background processes or just learn celery at this point. It is a personal project hosted on RPI, accessible on my local network and built using the Flask framework. Celery does sound nice on my resume, but for this application it is like “using a cannon to kill a mosquito.”
I tried out Sourcehut a bit and I liked it mainly because of their “unlisted” repo feature.
However, in this case considering OP’s strong opinion about “a” platform, I believe hosting their own git server is the right thing to do. If longevity and absolute autonomy is the goal, hosting your own git server achieves that purpose with very little friction.
I did actually try running my own git server. It did not work out. It kept going down, and I needed something more reliable than self-hosting. Keep in mind that my server consists of a decade-old laptop in my dorm room.
You can rent a VPS for cheap and get very workable availability. If you need redundancy in case the VPS host dies, you can mirror to your less-available laptop-at-home server, and use that to set up shop elsewhere when the need arises.
It would be a large time-sink to both set it up and maintain it. I do enjoy having CI, and that’s difficult to set up on a free-tier VPS.
I’m a student, so I don’t have very much money, so paying for a VPS is out of the question. I’m already using the free-tier VPSs for more essential things (like forwarding a static IPv4 address to my server). There is also the bit about trust. I don’t trust the free VPSs to protect my data and not be scummy. In the event that they are, I don’t want what I use for schoolwork to go down.
Perhaps you can pay for a VPS with some classmates/buddies that feel the same way about hosting? Then all of you can host all your projects there. It doesn’t have to be just for yourself!
Edit: Regarding CI - you could still use something like the free tier of TravisCI or CircleCI if setting all that up is too difficult/expensive.
SourceHut offer a few extra things that have appeal aside uptime. Having a CI set up, mailing lists, issue tracker, and IRC bouncer are a small suite of things that you’d probably also want high uptime + reliability on as well like OP mentioned. At least the goal isn’t engagement or loss-leader with VC cash (and it’s all open source, and open for contributions). I’d be curious what the git send-email web GUI will look like because it’s certainly a barrier to many–and I also hope FOSS tools like himalaya can help bring modern usability features to e-mail that it seems many clients have lost.
I really found their openness about financial position [0] incredibly commendable. I studied accounting, I have a soft spot for financial reporting but in accounting we also learned about “signaling theory” [1] (in relation to corporate ethics). Open-Source technology-based businesses that are so transparent about their finances without being forced by regulation will always have my absolute respect.
Recently I noticed if you try to search a project’s “code” on Microsoft GitHub while unauthenticated, you’ll get ? results and be prompted to log in to see anything.
I have a Mac mini and my Mac OS setup is that it is my server of sorts. Trying to get as good as possible in understanding SSH and docker for that reason.
I have a very specific setup in Windows which I wasn’t able to replicate in Mac. My windows setup is Ditto (clipboard manager), AutoHotkey, Razer Synapse 3 (for setting macros) and the driver software for my 15 key MMO mouse.
“Optimized” a ChatGPT powered grammar fixer automation: https://anyfactor.github.io/posts/upgrading_my_grammar_automation_script/
Taking a look at Cloudflare as a webstack with cloudflare workers. Will be looking into CF Pages and R2 next.
If possible, would love to look into Chrome extension building as well.
I made a grammar checker with ChatGPT, Powershell and Autohotkey.
Here is the blog: https://anyfactor.github.io/posts/chatgpt_grammar_checker/
I worked as a web scraper for more than 6 years. I did my fair amount of pattern matching to extract clean data.
For pattern matching don’t start with regex outright. Start with string slicing, stripping characters and splitting strings to manageable chunks. Then try the golden regex
.*?[0]. If you are not writing system critical code, ignore what people say about this being greedy. In my opinion, if you are writing custom regex, you will find that.*?does what you need most of the times.Then learn what the blog post is mentioning. It is a VERY good post, and absolutely echoes what I feel is true. The section
[Ignore this stuff]is extremely true and I really like the fact they included regex groups here.[0] https://stackoverflow.com/questions/3075130/what-is-the-difference-between-and-regular-expressions
I am trying to “memefully” learn Rust. The endgoal is to be competent enough to create a Tauri based GUI software.
For the “memefully” part the goal is to just enjoy the learning process as much as possible. Rust is a hard language, we all know that. So, I plan to write something in Python, then in Nim and then in Rust. Then write a blog about it. I plan to follow the Rust CLI book [0], however I want to do random projects. This weekend’s plan is to write a CLI that takes makes async get requests to a bunch of input URLs.
[0] https://rust-cli.github.io/book/index.html
Async causes a significant fraction of the difficulty of Rust. Nicholas Matsakis, long-time leader of Rust’s language design team, admitted last year that
I recommend considering using
ureqinstead, especially (but not only) if you still are learning Rust and you see Rust as hard.I concede that this depends on how large your “bunch of input URLs” is.
Thank you very much for the recommendation. Trying it out now. It looks easier than reqwest.
Or reqwest with the blocking feature flag
Which just launches a
current_threadTokio runtime, uses it to send a request, then drops it. As far as I know, creating a new Tokio runtime per request isn’t that expensive, but it’s definitely more overhead than ureq.I did not know that. Thanks!
For several years I thought that Rust was too hard and didn’t even attempt it seriously. Then last year I started the Advent of Code 2022 in Python (that I use professionally), and found this masterpiece from Amos: https://fasterthanli.me/series/advent-of-code-2022/part-1
It was super refreshing as it was showing me how close to Python basic Rust can be, but also with many additional features that make it amazing. I was able to do about the first 13 days in Rust thanks to that series (I was solving it before looking at the solution from Amos). I must say that it was an eye opener to me. I really recommend this series.
Fasterthanlime and No Boilerplate is the reason I am trying to learn Rust. Thank you very much. Checking the blog posts out.
Oh hey, I did something very similar to your batch URL requests recently! Here was what I came up with: https://github.com/adamhammes/rust_web_serial/blob/master/src/main.rs#L29-L48
That is nice. Thank you for sharing. Bookmarked. Now, I have to expand my feature list a bit more and throw JSON deserialization in the mix.
Checking out Deno. Learned about Deno Compile and it’s support for NPM [0].
[0] https://deno.com/blog/v1.34
Edit: The experiment didn’t go very well: https://anyfactor.github.io/posts/from_zero_to_deno_compile/
Just curious— how did you end up with 159 KB with Rust? I have tried
cargo build --releaseandrustc -C debuginfo=0 -C opt-level=3 main.rsand they both result in 508 KB. If Istripit, it becomes 333 KB.I’m on
rustc 1.72.0-nightly (d59363ad0 2023-06-01), if that’s relevant.No idea. Added the followup section: https://anyfactor.github.io/posts/from_zero_to_deno_compile/#followup-about-the-rust-file-size
Oh, I’m on OSX (on an M1) so it must just be a difference due to the platform. Thanks for sharing the details!
Okay, I posted this issue on r/rust. Quite an interesting issue.
https://old.reddit.com/r/rust/comments/14qrdh3/why_the_file_sizes_of_executable_binaries_be/?
As a self proclaimed “Python Developer” my rule of thumb is that, if the codebase is not super complex, was written by you and you are strapped for time just use Pypy. If it was written by someone else, you will probably find a fair few optimizations by going through the codebase first.
That’s it really. If you want more performance out of a Python, just rewrite that in a Go, Rust or even Nim. I feel like any kind of syntactical changes to a Python codebase to adopt a python flavor optimization framework is not worth it.
I think with frontend development, most “inventions” in the last five years have been largely superficial.
Yeah, it’s unfair that frontend development keeps getting criticized for being fad oriented. I do think it’s accurate. But I don’t think it’s a skill issue or worthy of shame. The improvements are superficial and there’s alot of churn simply because the problems haven’t been solved yet. In comparison to many backend tasks:
That’s not to say things shouldn’t improve. The writer of The Grug Brained Developer is also the developer of HTMX which is an important attempt at improvement. But I hate that it’s always framed from the viewpoint of “we’re not like those other solutions” when they exactly are: it’s an experiment to tame complexity and make frontend dev either easier or produce actually functional software. I support this fashion because I like it, but there’s nothing more legitimate about this particular fashion.
This was quite insightful. So, with frontend programming, there isn’t much in terms of massive problems, and they have limited areas to innovate.
Are there any other fields like that in programming? Suppose business software programming language developers (for example - VBA) don’t have much to innovate because most of the development ecosystem is out of bounds for them.
It sounds pretty weird to think that frontend development is widely popular yet they have some odd artificial restrictions that are imposed on browser design and “good SEO practices”. So, quite oddly innovation here is usually just about personal preferences and not about universal utility. Which is literally what fashion means.
That’s a bit reductive, no? htmx’s purported uniqueness is in its technical approach: instead of taking inspiration from native GUI toolkits and having client-side JavaScript build UI (with SSR added on top), htmx builds the plain “links-and-forms” model of the web and has the server drive the app by sending HTML responses that encode the possible interactions.
We’re all trying to make web dev better. What else would we do? Make it deliberately worse?
I don’t think so? But the question makes me think I was unclear. I fully support and even prefer htmx’s approach. I like the drive towards simplicity. Moving functionality to an area with fewer limitations and centralizing logic to minimize moving pieces is a great way to do that. I have 2 live projects using it and it does what it promises.
My point wasn’t that htmx isn’t valid. It’s that it’s equally valid as an experiment in how to build frontend software. Since there hasn’t been a clear pattern that has emerged as the “best” way to tackle things, I don’t like the chastising of other approaches when they are simply making different tradeoffs. Many problems haven’t been solved yet. It’s reductive to those problems that people are trying to solve. We can talk about the pros and cons of approaches and make tradeoffs without shaming the industry, and we can encourage more experimentation.
We’re all trying to make web dev better. We should be deliberately experimenting. Not writing off entire ideas as “fad based” in an attempt to frame one as separate from that social constraint. My gripe isn’t towards htmx at all, it’s at the general treatment of frontend development as broken rather than “in progress.”
I understand now, and I agree with your view. Thanks for clarifying.
If you were to extend that to the last 30 years I would agree with you.
Reducing things down to for example desktop OS design, I will submit without being two (2) important innovations in this time.
Both originated within Apple’s Mac OS X project.
Experimenting with ProxyBroker [0].
https://github.com/constverum/ProxyBroker
I’ll just repeat what I was saying on IRC channel about that: in my experience that line between “straightforward” to “clever” varies greatly not just between projects, but between teams and areas and moon phase too probably.
The judgmenent “will a junior dev understand it?” is somewhat questionable; like normal people they all come in different shapes and forms and to grow up they have to see some clever code and algorithms and research it and improve. Say you implement some relatively tricky part of your codebase and just describe it as “We use observer pattern here” or “that’s your run-off-the-mill Dijkstra” and said junior might not know what all that is. That is still not a reason to avoid solutions like that;
Then again I don’t approve of “clever code” in shape of “clever one-liners”, when stuff is not essentially more effective but just shaped to be crammed in very little symbols amount. There is ioccc fot that kind of elaborate insanely cool stuff :D
The author explicitly mentions this however they connect recreational code to clever code. In programming quiz sites, there is a sense of joy in coming up with solutions that only focuses on performance and briefness of characters. Moreover, when you see other solutions and you try to figure out how it works, you feel same joy. Hence “recreational programming”.
Production code doesn’t demand that kind of concentration or adventures from employees. Code maintenance is prioritized above all. However, coming up with one liner is kinda fun. As long as you provide enough context, comment and that stackoverflow link, I think it is fair to write oneliners.
Only if it brings a benefit over the more mundane approach. Otherwise, why require the user to follow that stackoverflow link and load up on all that context?
Even the “run of the mill Dijkstra” can be implemented in “clever” ways and straightforward ways. Also, it would help to name the algorithm (perhaps with a reference) in the comments, so the junior dev can look it up and understand it that way.
Yep, that’s what I meant! Even tiny comments can help with understanding seemingly “tricky” code parts.
Overall my thought is that the baseline for the code compexity is vastly different for diff. people and teams, and what is considered excessively “clever” for some people/teams/projects might be just an average for others. Otherwise we can lower the plank to the “will any random non-programmer understand this?” and just do everything in BASIC on Commodore64 :D
The project that he uses:
Github: https://github.com/sjvasquez/handwriting-synthesis
Web demo: https://www.calligrapher.ai/
In Japanese sumo robot fights [0] the arena is covered in a sticky substance to improve traction. I wonder if the rules allow for a substance like this to be used on the tires.
[0] https://www.youtube.com/watch?v=QCqxOzKNFks
Aside from work and work, trying to figure out whether I should stick with the tried and true, threading-based approach for running background processes or just learn celery at this point. It is a personal project hosted on RPI, accessible on my local network and built using the Flask framework. Celery does sound nice on my resume, but for this application it is like “using a cannon to kill a mosquito.”
I tried out Sourcehut a bit and I liked it mainly because of their “unlisted” repo feature.
However, in this case considering OP’s strong opinion about “a” platform, I believe hosting their own git server is the right thing to do. If longevity and absolute autonomy is the goal, hosting your own git server achieves that purpose with very little friction.
I did actually try running my own git server. It did not work out. It kept going down, and I needed something more reliable than self-hosting. Keep in mind that my server consists of a decade-old laptop in my dorm room.
You can rent a VPS for cheap and get very workable availability. If you need redundancy in case the VPS host dies, you can mirror to your less-available laptop-at-home server, and use that to set up shop elsewhere when the need arises.
It would be a large time-sink to both set it up and maintain it. I do enjoy having CI, and that’s difficult to set up on a free-tier VPS.
I’m a student, so I don’t have very much money, so paying for a VPS is out of the question. I’m already using the free-tier VPSs for more essential things (like forwarding a static IPv4 address to my server). There is also the bit about trust. I don’t trust the free VPSs to protect my data and not be scummy. In the event that they are, I don’t want what I use for schoolwork to go down.
Perhaps you can pay for a VPS with some classmates/buddies that feel the same way about hosting? Then all of you can host all your projects there. It doesn’t have to be just for yourself!
Edit: Regarding CI - you could still use something like the free tier of TravisCI or CircleCI if setting all that up is too difficult/expensive.
Student is a good excuse.
gitea on free cloud server and backups at home? https://paul.totterman.name/posts/free-clouds/
I’m already using the free Oracle VMs. Backups are also going to rsync.net.
SourceHut offer a few extra things that have appeal aside uptime. Having a CI set up, mailing lists, issue tracker, and IRC bouncer are a small suite of things that you’d probably also want high uptime + reliability on as well like OP mentioned. At least the goal isn’t engagement or loss-leader with VC cash (and it’s all open source, and open for contributions). I’d be curious what the
git send-emailweb GUI will look like because it’s certainly a barrier to many–and I also hope FOSS tools likehimalayacan help bring modern usability features to e-mail that it seems many clients have lost.I really found their openness about financial position [0] incredibly commendable. I studied accounting, I have a soft spot for financial reporting but in accounting we also learned about “signaling theory” [1] (in relation to corporate ethics). Open-Source technology-based businesses that are so transparent about their finances without being forced by regulation will always have my absolute respect.
[0] https://sourcehut.org/blog/2023-03-27-2022-financial-report/
[1] https://en.wikipedia.org/wiki/Signalling_theory
Recently I noticed if you try to search a project’s “code” on Microsoft GitHub while unauthenticated, you’ll get
?results and be prompted to log in to see anything.It’s been like that for a couple of years at least
I vaguely remember this was because of people running token scraping operations from unsecured public repositories.