-
I’m working on getting more interesting / bigger scope / more architecture-focused projects at work. I’m new to the company and the team, so I have been primarily doing smaller stuff. Hopefully soon I can take ownership of larger components of the project.
For personal projects, I want to start working on my personal library management system.
-
I will be moving into my new house, which will be exhausting. Then, I would like to either continue work on one of my two slow-moving projects: 1) a distributed mesh messaging network 2) a website for my personal library, so that others may see what books I have and borrow books from me. Ideally this will turn into a thing that anybody can load their libraries into.
-
Personal: I’m adding to my bytecode interpreter blog post – virtual frames, name resolution, etc.
Work: Adding some more features to the runtime at work so we have feature parity with upstream.
-
I wrote this to understand bytecode compilers and interpreters. Please let me know what you think!
-
Thanks for the mention at the end :) I haven’t touched that README in awhile, and reading over it reminded me to get cracking on that work!
-
Of course – I am fascinated by your project and would love to find some application for OPy in my life. This was hopefully the first step in doing something related.
-
Cool, feel free to send me any questions about it! There is a public messaging system at https://oilshell.zulipchat.com.
Right now I don’t think OPy is that interesting, aside from being small and hackable. But if I actually manage to compile Oil down to something more efficient and compact with it, then it will be interesting!
Basically “OPy” will be defined as “whatever collection of compiler techniques work for the Oil codebase”. That makes it very concrete and I think a pretty interesting open question. It’s also possible it will fail, but we’ll see :-/
-
Huh, looks like I meant OVM2. I am interested particularly in having a tiny Python bytecode interpreter. Is there anything I could do to help with that?
-
It’s hard for me to think of something specific, since there’s a lot of background knowledge required to answer that question :)
But to be brief, I started the VM, and I would basically accept any patch that makes it run more real Python programs (and which has tests showing that). I mentioned in a blog post that it runs a toy Fibonacci. It doesn’t even implement functions or dictionaries yet, so it’s very basic, and there’s a lot to do.
https://github.com/oilshell/oil/blob/master/ovm2/main.cc
I definitely need help because finishing up OSH and implementing OVM2 is easily a year’s worth of work, and that leaves aside the Oil language. It’s not clear if there’s a good place to jump in, but we can chat about it or feel free to hack around.
I maintain a list of issues for the broader Oil project: https://github.com/oilshell/oil/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22
There are some small issues, as well as some larger ones that don’t require much background on Oil itself but require some knowledge in another area (e.g. fuzzing).
-
-
-
-
-
-
I’ve enjoyed working with libtap. It’s small and effective.
-
I read Neal Stephenson’s Anathem and loved it. Blew my mind. It did take a while to get going, though. But totally worth it.
-
Anathem is really wonderful, I would suggest it too.
-
-
I have a Lenovo T480S running Ubuntu and love it. It’s a beast and it’s pretty light, too!
-
I’ll be starting my second week on my new team. I’m excited to have joined! We’re building a runtime. Can’t wait to learn more about that. As far as technical specifics go, I’ll be adding some metaprogramming builtin functions and some new types.
-
I’m going to make be cooking for a potluck and seeing a good friend who is visiting while interviewing in the area.
-
Why is this tagged
release? It looks like it was last worked on last year.-
Or
php, orvimfor that matter…-
I killed the other tags but left in vim because it’s a vi-style editor.
-
Oh, I missed that. Thought it was pure acme. Neat!
-
-
-
-
I don’t think articles with paywalls should be posted. I cannot read this article without purchasing a subscription.
-
-
-
Hi, sorry – didn’t see that it had already been posted across the comments. I appreciate you posting the article. It’s good content.
-
-
-
-
-
This is the one tool that is making me seriously consider flipping my workflow upside down and switching from vim + git cli to emacs.
That said, if anyone knows of something similar to magit that doesn’t require emacs, I’m interested in hearing about it! I’ve played with tig[0] a bit, but always go back to using git from the cli..
-
I recently started using spacemacs in vim mode (which is its default) and it’s lovely. It’s been a much better experience getting a decent environment going than I’ve ever had with vim. Haven’t dug into Magit yet but I’m eager to. The docs for spacemacs are a bit scattered but this was the most useful for me to get going:
https://github.com/syl20bnr/spacemacs/blob/master/doc/BEGINNERS_TUTORIAL.org
-
How long did you use vim workflows before switching? I’ve been a fulltime vim user for about 10 years, and I’m afraid that my productivity will take a serious spill as I try to unlearn muscle memory and build new ones. If you used vim for a while, I’m curious how that experience was for you.
-
I switched mostly to emacs + evil (also used for Spacemacs) after a decade and a half or so on various vi implementations. Evil is very complete, evil often feels more like a complete vi implementation in Emacs Lisp rather than ‘vi emulation’. So far I didn’t have to unlearn any muscle memory.
I used Spacemacs for the initial switch, but I found it to be very buggy and slow. So, at some point I just switched to Emacs + hand-picked packages. I use
use-packageto load/install packages that I install with Nix and manage withhome-manager. I use general to set up spacemaps-like key bindings (e.g. SPC-p-f to find a file in a project SPC-b-b to search buffers, etc.).However, the Emacs ecosystem can be a bit overwhelming, so Spacemacs is a good entry point.
-
I never spent longer than six months or so using vim full time, so I’m in a very different situation. You’ll be able to put much of your muscle memory to good use in spacemacs, using the same keystrokes to navigate and edit within a buffer. However you’ll need to re-learn many other common tasks.
You will absolutely take a big productivity hit at first, but if you stick with it then you’ll start feeling comfortable pretty quickly. Learning is its own reward! :)
-
-
-
I have used vimagit and fugitive (both Vim/Neovim plugins) together for a while now. Vimagit is far from equal to magit in terms of power, but provides my Vim setup with the main features I missed from magit (visual range staging, easy amends, etc.). Fugitive is also useful on its own, but I currently mostly use it to asychronously push/pull from within Neovim (as vimagit does not yet provide these features itself).
-
-
One thing I’m wondering about trying is a TUI-only Emacs configuration with a virtually completely empty
.emacsfile, flset up for just magit and nothing else. I’m wondering if the load time with a maximally stripped-down configuration would be short enough to make it feasible to use magit in a non-Emacs-oriented workflow. So, edit in something else, launch Emacs+magic in terminal, stage changes, close Emacs.The Emacs daemon mode might be an option too but it adds a bit of complexity to the setup. :/
-
-
Fugitive is amazing. I use it extensively at work.
-
-
Yeah! It’s got good diffing, committing, etc.
-
-
-
What does it do differently/better?
-
-
-
-
I’m working on a PL project in Rust at work, which will be fun. This week I’ll be adding some niceties such as strings to the VM.
-
-
One of the things I’m not thrilled about with this project is that I can’t say much. :/
-
-
-
My clang patch was finally committed so I’m stepping back from that. This week I’ll be working on more PL stuff at work as I try out a different team.
-
I find the “What are you working on this week?” threads depressing.
Sometimes I do, sometimes I don’t work on things that might be interesting, but seeing the thread gives me a bit of anxiety that there is an expectation that people should be working on something every week.
(Just to add something that I’m working on: migrating the last few dozen accounts that I have on various sites over to my unified password manager (1password), so that it uses a centrally managed list of service-unique and strongly random passwords.)
-
Don’t sweat it too much.
I don’t participate every week. This week, for instance, I’m probably not going to do much tech work outside of my day job. I’ve been letting a lot of household things linger, and I need to spend a little while catching up. At work, I’m basically doing what I did the week before, with the wrinkle that I’m team lead for a new team, and it’s a lot more work than my old team (at least for now).
-
+1! There are plenty of weeks where I have nothing to say or feel like saying nothing, too. Depending on what I’m up to, it’s even been like that for months at a time. Those times, it’s hard not to feel left behind compared to the selection you’ll find here – which frequently comprises things people are really excited to share, and not so much the quotidian “spoon against the wall” effort that’s also part of most any work. I guess my consolation is feeling happy seeing other people up to so much, even if I’m just shoveling metaphysical snow.
So, I’m not sure if I’m up to much or not! This week is 80% Airflow operationalization for some research-on-research work at the office (Wellcome Trust Data Labs), and 20% back on a
dlnmpowered time series analysis associating civil unrest with rate of healthcare visits per person. The latter finally started making sense this weekend. A real relief there.
-
-
Sometimes I use it to talk about what I’m doing at work. Sometimes I’m doing interesting things in my free time. Sometimes I like to browse what others are doing and use it for project discovery. I think for the most part the community recognizes that not everyone will be doing “interesting” stuff on their own time all the time.
-
I can appreciate that, but the way I see these posts is more “Hey who’s doing something interesting?”.
It’s an opportunity for people to offer help, encouragement, and experience they might have and for the community to appreciate all the amazing things people here build. People are human, even the most prolific coders aren’t producing gems for the ages every single week :)
-
-
It’s worth taking a look at @rain1’s https://gist.github.com/rain-1/a253e47b939fc0769524d8716541c96e .
-
-
As far as I can see, it doesn’t have any notion of “byte”. But for normal strings, it’s a string of length 0.
-
It’s length-prefixed, like the canonical S-expressions which inspired it, and mentions byte equivalency, so I think it has to be byte-oriented.
It’s a notation for lists represented as dotted pairs, but its pair notation can’t be used directly because there’s no way to distinguish
("a")from('a" . ""). With normal NIL-terminated lists that’s okay, because NIL is special in a way that I think the empty string is not.But maybe I’m wrong.
-
Unless I am completely missing something, this is what I have come up with:
("a"): .S1:aZ0: ("a" . ""): .S1:aS0:-
-
The extended one is definitely the interesting one :)
-
-
-
-
-
-
-
It’s posts like this where I’m reminded that it’s only a matter of time before something goes wrong on my computer and I’m too dumb or burnt out to try and fix it.
-
-
Does running Void Linux make audio easier? How is it controlled on Void?
-
-
But are there different tools or does it still require mucking about with PulseAudio and Jack and …?
-
-
-
-
-
-
I’ve started writing a post called “The case for linearly bootstrapping compilers”. I don’t quite know where I’ll go with it, but hopefully I can argue the need for compilers which are not 100-percent circularly bootstrapped.
-
Three come to mind against 100% circular:
-
You get the compiler done faster if you use a language better suited for building compilers/interpreters. Many argue for LISP, ML’s, and rewriting/meta languages. Intended language might not be among them.
-
Your compiler will run fastest if you use a language easy to translate to GCC, LLVM or C. It doesn’t have to be normally but maybe one has reasons for the speed. Some are much easier to translate than others, esp if keeping advantages of high-level form.
-
If wanting formal verification now or later on (“verifiable” compiler) with low effort, then you’re limited in building blocks to what languages, algorithms, and data structures people already verified. You don’t even have to use the verified versions: just using unverified pieces with same functionality lets someone swap them out for real thing later.
Interested to see what yours says when you’re done with it. So far, I think circularly bootstrapped just hits a button in people’s heads that says, “Wow, what’s wild to think about.” Field evidence shows other options are probably more practical depending on what goal you’re optimizing for. Except for static Scheme’s closer to the machine (eg PreScheme) where doing 1-3 simultaneously might be feasible in same language. Especially if you bypass C’s ecosystem with profile-guided superoptimization on fast paths. I don’t know if that exists but it probably should. ;)
-
I want to talk about auditability in a trusting trust scenario, formal verification, and something I’m calling the “verified implementation graph” for lack of a better term. Basically I want to bootstrap the world from a small core language and build production compilers from there.
-
That makes sense. It’s what quite a few on this page were doing. My favorite is still nineties’ project. For more mainstream support, my concept was a C-like, While language with metaprogramming support. An interpreter written in assembly to start with. Tcl or LISP-like syntax to make hand-made parsing a breeze. Build up a few primitives and module support for programming in the large. Hand or tool-assisted conversion of something like TCC to it. Bootstrap. Then, TCC to early version of GCC. Various GCC’s from there.
Designing the solution at a high-level was fun. Implementation looked like a lot of tedium for me. So, I moved on once a solid plan was visible.
-
Yeah, I’ve seen miraheze.org. I’m trying to make this kind of bootstrapping plan more accessible to people who would like to join the effort. Perhaps part of that encourages people to write small runtimes with large libraries – like a mini Python that’s mostly written in itself. Just enough Python to bootstrap another programming language. That way there is more than one way into the graph – multiple edges can be made, and then it’s possible to compare the results of two seemingly equivalent compiler implementations compiling themselves…
-
Sound interesting. Especially an accessible project that allows comparisons. Look forward to seeing what comes of it.
-
-
-
It’s specifically designed to build his language. He probably didnt work to make it faster, more portable, have a great FFI, etc. The part I like is his staging and mix of native/interpreted to get benefits of each. He goes from ASM to bit better language to bit better and powerful VM. Then, it’s so much easier to express the tooling for his actual, intended language.
-
I’d love to have it in a printed book form, to be able to easily study and annotate… ideally with some reasonable comments, like the ones in the slides… This kind of bootstrapping is totally an idea that zaps through my head from time to time, and from the slides, his approach looks like it might be approachable enough to just take and read, and certainly immensely educative…
-
That would be pretty awesome. I think we can get some docs and source at best. The rest could be a nice project for someone else. The project links are here.
-
-
-
-
-
-
-
So far, I think circularly bootstrapped just hits a button in people’s heads that says, “Wow, what’s wild to think about.”
I suspect one other major motivation here is that you impose on your fledgling language a BigRealWorldish endeavour that will leave you no choice but to continuously validate, refine, evolve and/or revamp most of its initial design choices as well as its stdlib (this latter facet again feeding back into the former along the way), which will also finally need to grow to approximately the minimal size and scope that the potential target audience might judge “barely-sufficient to justify playing with it instead of discarding outright”.
-
That’s what they say. Many often write a non-optimizing, simple interpreter or compiler for the circular part, though. It doesn’t really stress the language in a lot of directions. It just says it’s good at expressing an over-simplified interpreter or compiler.
I think a better target for that goal would be porting a set of libraries that were really different in purpose and how their solution is best expressed. One might do some string manipulation, number crunching, complex structures, layers of control flow, high-error situations, concurrency, parallel processing, low-level interfacing, and so on. You prove it can handle multiple types of apps by implementing multiple types of apps. Each can be as small as needed for proof of concept.
-
-
-
-
Would it make sense to implement this as a hook in fzf? Then fzf’s UI (including “preview” feature, which makes sense here) and keybindings code can be reused.
-
Wow, that would make this truly excellent.
-
Hmm! I know about fzf’s existence, but never actually used it yet, so I’m not very familiar with how it works. Could you point me out to where the hooks mechanism is described/implemented? I don’t seem to be able to find such keyword in the readme…?
To tell the truth, I actually considered trying to build up in fzf at an early point after my vision materialized! I even tried to see whether fzf doesn’t already have such functionality :) I think I became overwhelmed with the fzf’s guide however, such that I wasn’t sure if my would be able to fit in, and even if it did, whether I wouldn’t get lost in the fzf codebase… I would be interested in getting some help here with someone knowledgeable enough in fzf’s inner workings. I might try to reach to fzf’s author, maybe by creating an issue with a proposal at some later time {note to self: TODO} — but please do chime in earlier if you can help.
That said, I do also have some ideas where I would like to try to take up further, and I’m also not really sure if they are compatible with fzf in any way… meaning, I’m afraid they could tear fzf apart anyway if I tried to apply them there… Hmm! — but maybe I could leave the up repo to myself for prototyping, and just try porting what’s possible to fzf! Ok, that sounds like a reasonable plan to me at this point. Thanks!
-
So, not seeing the entry for Lua yet, I’ll allow myself to hereby add it! By the way, I first learnt of the language exactly as part of an app where it was used as an application scripting language. I’m amazed by Lua as a language ever since.
Other than that, I’m not sure if Red is embeddable as of today; but if not, I believe it could become a really interesting option in this area eventually.
Finally, I’m recently learning Nim, and I find it very interesting. As subset of Nim called NimScript can be run in a VM, and can be made embedded in other programs (though it seems it’s not exactly a turn-key solution as of today).
I have been using Lua as well, for about 12 years now. My main reasons are:
The simplicity of the (reference) implementation. Lua is about 16000 lines of C, any experienced programmer can read them and understand how the whole interpreter works. The same cannot be said of most other mainstream languages.
The wonderful language of the design. Nothing in Lua feels like a kludge to me. That is probably because it is designed and implemented by a very restricted core team, and not by committee.
The syntax is not S-expressions. As much as I want to like Lisp, I cannot bring myself to find that syntax pleasant to use in practice.
The ease of embedding it in C and sandboxing it. Some people do not like the Lua - C stack-based API, but to me it is one of the main selling points of the language. I have written bindings for other dynamic languages such as Python and Ruby, and compared to that experience doing the same for Lua is wonderful.
And over the years, I have found an extra reason, which is the community. The Lua community is small but great, maybe in part because it is one of the few mainstream languages that does not originate from the Western block (Europe / USA). Because of that, its community is a mix from people from all over the world, from very different industries, who use the language in very different ways. Going to a Lua-related conference such as Lua Workshop is always a great time.
I think that the fact that Lua feels as well designed as it is is because it has had 5+ iterations with few considerations given to backwards compatability. Which isn’t to dog on it, but to more say that it’s had a lot more time to be refined in ways that would have not flown if an early version of Lua had been shipped in web browsers, like JS was.
Completely agreed.
All great points; but your third point reminded me that the unrelenting simplicity of the language does make it a fantastic compilation target. So Lua doesn’t support a s-expression-based syntax out of the box, but it can be added in roughly 1kloc of code without changing the semantics. Prefer syntactically-significant whitespace for some reason? You can do that in a compiler too! (Moonscript unfortunately does add a bunch of new semantics like the class system, which is a huge misfeature IMO, but it’s readily within the grasp of one person to create a moonscript-lite that’s just syntactic.)
The first two links in “can be made” are the same. Was that intentional?
Nope, sorry! The second one should have been: https://github.com/komerdoor/nim-embedded-nimscript. Thanks for asking!
I too use Lua, but at work, it’s more for LPEG (Lua Parser Expression Grammar). It takes a while to get used to it, but I find it much more expressive and powerful than the Lua patterns (or even regex). Not only can an expression be reused as part of another expression, but the resulting code is its own VM (distinct from Lua) that is geared directly for parsing.