I’m doing reading that I think will be helpful for my grad school application (master’s in Scottish Ethnology) and have just discovered the magic of org-roam for Zettelkasten-style note-taking in Emacs. This week I produced 3300 words of notes spanning 93 files, but I’m a little behind on my reading plan and want to catch up this weekend. There are also a few org-roam features that I’m aware of but need to learn how to use, such as tagging and launching a server that dishes up an interactive mind map
Would love to talk to anyone who has experience with humanities grad school, org-roam, etc!
org-roam is fantastic! Especially when paired with org-protocol, you can add a reffed entry straight from your browser. If you are in school and writing lots of papers, org-ref plays nicely with it as well which is a game changer if you re the sort to write lots of papers. Its one of those things where I wished I had a use case because the workflow was so nice.
Good to get the encouragement about org-ref. I’ve been handling that very manually right now and it’s not great.
On the one hand it’s a little overwhelming to figure out how to use all the relevant bits of the org ecosystem, but on the other it’s nice that I can be productive now and still have room to become more efficient by doing some toolsmithing. I’m excited for my setup to evolve. And, for that matter, am excited for when I have 100k words of notes spanning 500 files. A lot of papers is the goal!
It’s an interesting place to be: there’s still a lot of low-hanging fruit in the field (18th-century Scottish music history) since not many people are working on it and the internet has made so many more primary sources easily available. A lot of the top scholarship was written in the ’80s and was an admirable effort, but of course they were limited by having to physically travel to many far-flung libraries and use incomplete physical card catalogs to access materials.
I’m making up hours missed at work during the week. I’m almost done with my Transcranial Magnetic Stimulation (TMS) appointments (supposed to help with clinical depression, but I’m not seeing much of a change (it has around a 65% success rate and I guess I’m one of the 35%)). TMS has been kicking my butt. I get migraines and it has increased the number of headaches that turn into full-blown migraines.
But, I’m having a lot of fun at work. I’m steering the entire company into a whole new direction and it’s really fun. I hack on PHP, C, and C++ code all day. So it’s not like working over the weekend is a drain for me. I’m passionate about what I do, sometimes to a fault.
Writing an IRC bot in Rust that broadcasts messages from a channel to HTTP clients using HTTP streaming, so I can later use it to implement danmaku chat (chat messages overlaid on top of the video flying from left to right, as seen on Niconico and Bilibili) on a small stream that me and a few friends host every January. It’s my first time writing anything using concurrency in Rust, I predictably ran into some issues with borrowing but I somehow managed to solve them using an Arc.
AoC. I managed to stay on track so far so I’m gonna keep it up. I’m doing all problems in F# more out of convenience than anything and I’m finding it very nice; the .NET standard library is huge so it has lots of stuff built in that I’d have to write myself if I were using OCaml, and its strong typing makes it easier to work with complex data structures while still being terse enough thanks to type inference. And you can always use mutation as an escape hatch when you need it.
Playing some more Cyberpunk, I’m enjoying it a lot so far.
Yeah I’m using the same crates, I’m guessing the code’s probably similar too. I already got the basics down, just have to remove all the .unwrap()s from the code and maybe add some logging and all that. I’ll try to finish it by myself but I’d be interested in seeing your version!
Hopefully putting the cherry on a rework/reorg/rename of the flags and envs for a CLI tool I’ve been working on this year (roughly: it rewrites command invocations in shell scripts to absolute paths).
It has been a little tiring and hard to focus on for a few reasons:
It’s in some of my personal weak/blind spots:
I’m prone to analysis-paralysis
I have a hard time categorizing things (maybe “separating” things?)
My background is writing/editing (poetry, specifically); the part of me that ~needs to find The Right Word can be hard to disengage once it’s activated.
Because shell, there are a fair number of patterns it has to treat as blocking errors until the user disambiguates how it should handle them. Most of these don’t have obvious names. Some were obvious early on (such as using a variable as a command, like $GIT_COMMAND) but a fair number just emerge from use. To avoid some of my weak/blind spots I just gave these hasty ad-hoc names during initial dev.
There’s some pent-up demand, so I’m putting more pressure on myself than I usually would to rip the bandaid off, rather than iterate in public and repeatedly breaking existing uses.
Initially I was trying to iterate directly on the arg-parsing code, but I eventually settled on writing the manpage to stay focused on the concepts. I think I dragged my feet for long enough that the major patterns have all emerged.
Now, it’s ~time to actually implement the argument processing and do any related refactoring. :)
I’lll be continuing to race Advent of Code at midnight!
I’m also working on pushing my music server to a stable release. This weekend, I’m planning on:
Switching the React frontend styles to twin.macro over raw TailwindCSS.
(Attempting to) migrate the database to an immutable event log-like design. The database is SQLite, which doesn’t support materialized views, so I’m wondering how to best go about this. I’m thinking of using triggers to manually emulate materialized views, but this feels rather error-prone.
I thought of a way to combine a nix like package manager, with a less obtrusive packaging model. You might imagine it like half way between nix and docker.
When I have time, playing around with some more AOC fun. I’ve found that I need to be careful diving into the problems because I enjoy them so much I’ll end up coding for hours, even rewriting code that works to make it better. Much fun. Too much fun
Working some more on adding content to my professional wiki. I am quite enamored with using wikimedia
How has the performance been and graphics in general? I was thinking of waiting until they release a version optimized for PS5 (figured some of the bugs would be ironed out by then also).
Performance has been great for me, graphics are decent and I’ve experienced pop-in and random t-poses. Crashes are a regular occurence but not annoyingly so.
Aside from all this, there has been no lag. Unless you are a Deus Ex/RPG fan, I’d wait for the optimized version. I absolutely love the last two DX games and I’m enjoying exploring the world of 2077 via side quests.
I’ve been working on my java vm/aot compiler this week, and will spend some time this weekend. I have a problem that someone among you might be able to help with.
Consider the class Hashtable.Enumerator, which is an inner template of a template. Hashtable has two parameters, K and V, Enumerator adds a third, T. You cannot use that class directly, you have to ask either Hashtable.getIterator() or Hashtable.getEnumeration(), both of which are non-static template methods that also add a T.
K and V are obviously tied to Hashtable. But how about T? It seems obvious to me that the T in Hashtable.Enumerator is intended to be the same as in getEnumeration() and in getIterator(), but I don’t see any rule that ties the three Ts together. Where’s the rule?
I’m not sure I understand the question. The Enumerator‘s generic parameter is not explicitly tied at the language level to the generic parameter of the outer class, because you can use it for key and value enumerators. The type parameter in the constructor tells it whether it’s a key or value enumerator and so T is equal to either K or V, depending on the value of that. Given that generic parameters are type-erasing in Java, this doesn’t actually matter for the generated code: in the .class file it’s an Object.
For someone writing an AoT compiler; however, this is interesting because the final field makes this class a great candidate for compile-time reification. You could eliminate the type field and provide two hidden subclasses of Enumerator specialised on the two possible values of the field. This requires a little bit of reachability analysis (what are the types passed to the constructor for the final field?) and a bit of logic in your VM to ensure that your reflection code can handle the case that two objects have the same Java type but have isa pointers that refer to different concrete instantiations.
How my compiler treats the Enumerator class depends on how much it knows about T.
When I look at the code it seems as if the T in newIterator() is meant to be the same as the T in Enumerator, but I cannot find a rule that lets the compiler assume anything at all, which puzzles me. Since I am frequently within a hair’s breadth of being the world’s worst programmer, I consider it possible that I might have missed something, and asked. But I don’t think I’ve missed anything.
The definition of T comes at construction time. With type-erasing generics, it doesn’t flow through the static type system at all (as I recall, it isn’t even present in the bytecode in the .class files). You have to infer it. Looking at the code, it doesn’t actually matter for code generation of Enumerator what T is, but it does matter to the callers because they’re going to do a redundant down-cast from Object to T. You may want to do some dataflow analysis to figure out whether, for each Hashtable a non-K key or non-V value can ever be inserted (I can’t remember the semantics of Java generics. If the compiler inserts a cast before the insert, then you just need to find out whether the Hashtable<T> is ever cast to a Hashtable). If it can’t then you can skip the down-cast (that said, with a good representation, down-casts in Java are three loads and two compares, so it may not be worth it.
In a language with generics that doesn’t just erase them, it’s useful to do compile-time reification. C++ does this for templates, C# for generics, and so on. Here, you’re generating a specialised version of each function based on the values of the generic parameter. In Java, the type in generics doesn’t typically give you anything useful here but the values of final fields do. In a JIT’d version, you’ll likely see the methods being specialised on the value of the final field if they’re hot. In an AoT compiler, you can see if there are branches based on the value of the final field and look at the construction sites to determine whether you want to reify based on those values.
The really fun thing for an AoT Java compiler is figuring out where to do devirtualization. Do you support dynamic class loading?
I had another think about the original question, and it was a little mistaken. The compiler complained that it couldn’t compute some blah blah, but nothing actually depended on the particular value that couldn’t be computed, so there was no reason for it to complain to me.
Yes, I do support some class loading, not 100% but enough to support all the java programs at all of the places I’ve worked.
Class loading really is unavoidable. Even printing hello world requires that; openjdk wakes up in the morning, yawns, looks at the environment variables, constructs a class name and loads that to use as foo-to-unicode converter, then it starts parsing command-line arguments using the new converter. Several of the widely-used libraries do similar things, e.g. Class.forName(prefix + “/handlers/” + tagName).newInstance().handle(parsedThing). I personally feel that that’s not in the java spirit. Java is a t-crossing i-dotting language. But it’s done and it has to be handled.
Devirtualisation is fun, particularly when it interacts with reification and inlining it can make the output look so slick.
I imagine in an AoT environment you can handle class loading in three ways:
Fake it entirely, compile all things that might be loaded, analyse the whole program as if they might be loaded (only adding them to class tables and so on for reflection once they exist). You may be able to do some speculative devirtualization here that you turn off after ‘loading’ a class.
Partially fake it, be really conservative about the devirtualization that you do (or, at least, make it cheap to turn any of it off, so guard everything with checks), AoT compile any classes you may want to load and just dlopen them. I think this is more or less what gcj did.
Add a JIT or similar that invokes the compiler before each load. A couple of implementations I’ve seen use an interpreter for loaded classes, which gives an exciting performance cliff.
I’d be curious about which approach(s) you’re using. Is your code online somewhere?
I interpret static initialisation and allow class loading there. Just before the first line of main(), I stop and generate native code, and after that point I (partly have done, partly will) look at the program and classify each class as either loadable or somewhat more devirtualisable. List and AbstractList are examples of the former, ArrayList of the latter.
After that decision, an LTO pass can devirtualise method calls to methods such as ArrayList.contains(), which isn’t marked final but may be known at link time to be effectively final because no classes in the VM extend ArrayList and reimplement that method. And of course, classes can be loaded but loading one that implements/extends an unapproved class will throw an exception. There are some very tricky issues to solve involving static initialisation.
My reasoning is that
loaded classes tend to extend/inherit a few special classes, generally abstract classes or interfaces.
it’s done less anyway, rebuilding and redeploying is the 2020 fashion, so perfection isn’t necessary.
It’s a bit fake and will OSGI fans will not like it, but there’s no performance cliff.
Source isn’t available and won’t be for a while. Among other things, I worry about time-draining discussions. There’s an attention-span cliff: People ask questions about this that are simple to ask, but answering the questions or understanding the answers are beyond the cliff. (I’ll answer you, of course. Your attention span is excellent.)
That sounds pretty close to what the .NET Native team did and it seems like a good direction to me. Good luck!
I’m somewhat curious because I was looking for a decent Java AoT compiler when I did the CHERI JNI work. A modern JIT on an (in-order, single-issue) FPGA softcore at 100MHz is painful, but cross-compiling from a nice fast machine is very easy. I ended up using JamVM, but because it’s a pure interpreter I couldn’t run anything other than microbenchmarks and get meaningful results. I’d love to do something a bit more real on the Morello boards when we get them.
Now I wonder how those implementations handle lambdas. Do you know?
Lambdas use class loading and much else. My own compiler reaches 120 stack frames. Doing all that at runtime… maybe their performance cliffs are as steep as my stack is deep.
I haven’t looked at lambdas in detail in Java, but my understanding was that they didn’t modify the JVM, a lambda is just a class that has a single method in its vtable to invoke it (plus a constructor to copy in all of the bound variables). The disassembly that I looked at from a class that contained a lambda just had an inner class representing the lambda with a mangled name for the invoke. I’m not sure why it would need class loading for this, it’s no different than the Java 1.0-style action-listener classes (inner classes with a single method conforming to a single-method action interface). I’d be curious to see examples that required class loading.
It’s a little more complicated, but you have the gist of it. The single-method class you mention is generated by javac using the .class format, but that data isn’t written to a .class file in the file system or a .jar, but rather output as a constant byte array inside the .class file that contains the lambda, and when the lambda is first executed, a compiler-generated method calls ClassLoader.defineClass() with that byte array and a javac-generated name. That’s also a bit of a simplification, IIRC it’s about right for OpenJDK+J9 but OpenJDK+Hotspot has some more moving parts, so that it can execute some lambdas without needing to load the .class. Don’t even look at a class called InvokerBytecodeGenerator, it’s bad for your soul.
As I recall, I first ran into this for java.util.time.format.DateTimeFormatterBuilder.QUERY_REGION_ONLY.
I’m also not sure why it was done this way. I did see something about code reviews… there is a possibility that code reviews in parts of the java source code may be stricter than in others, and that when the authors of the lambda system wrote it, they tried to add extra flexibility so that future changes would not need to change the source code with the highest level of scrutiny. I’m not sure. I’m just speculating.
That’s impressively insane. Do these byte arrays at least have a plausibly sensible name mangling so that you can AoT compile them speculatively and just expose the new classes when someone calls defineClass with that byte array?
The naming scheme plain enough, but I don’t need to, I just follow about a half-dozen levels of indirection from the invokedynamic opcode. AoT compilation works for lambdas when I use the J9 variant of OpenJDK. No luck with the Hotspot variant.
I think the Hotspot developers must have a very high opinion of their future selves.
Relax and mess around with making video games. Probably work on game engine stuff as well.
Probably play more Wing Commander Privateer too. Never played it when I was a kid and I’m starting to think I missed out. Just need an engine upgrade and an afterburner and I’ll show those pirates who’s boss…
Improved the purchase flow for Graph Galaxy to avoid the misleading, and another cool feature just supported is that we can zoom in and out the graph by pinching now. So this weekend I am going to do nothing and have a rest ;)
Received some ZnSe lenses sooner than expected, so I might play around with focusing a thermal camera.
Otherwise, messing with OpenSCAD to see if I can’t figure out a way to generate multiple different structures I’ve had in my mead for a while. If I’m lucky, I’ll even have time to try fabricating them too.
The wife is working all weekend, so other than working on Advent of Code not a whole lot. Maybe I’ll finish my Metaprogramming Ruby book.
<mini book review>
don’t buy if you already know what metaprogramming is and/or you already know the Ruby object model. Having read Ruby Under a Microscope before, most of this book was already covered there. And I HIGHLY recommend the latter, it teaches much more and is just a better book (imo).
</mini book review>
Having a lot of fun with it, although its a ton of refactoring since Im learning so much.
I made the switch over to Linux this week, away from Mac and am really liking it. I decided to also give VIM (Neovim) a try as a main development editor, and while the learning curve seems super high, I can already see how good it can be.
So basically, im spending my weekend writing code in a language I dont know that well, on an OS which im still getting familiar with, in an editor where I have to look at a cheat sheet every time I want to copy some text or look at another file, and I couldn’t be more excited!
Btw, if any people with Rust experience have any comments on the project above (I am sure I am doing a ton of stuff wrong), I am open to any feedback.
I have been teaching Common Lisp at a small online mathematics and computer science literature club (#spxy:matrix.org). One of the biggest hurdles while beginning to learn Common Lisp is that the popular books on the subject recommend Emacs and SLIME as the development environment. For someone unfamiliar with both of them, this can be quite a significant learning curve.
While there is Portacle to ease the initial effort to an extent, I am working on my own take on this problem: Emacs4CL. This takes a DIY approach to setting up one’s Emacs + SLIME development environment. It provides a tiny ~/.emacs file that shows how to customize Emacs and automate package installation. The accompanying README provides a line-by-line walkthrough of the entire ~/.emacs file to help a beginner to Emacs make an informed decision about which lines from the file to keep and which ones to discard in order to better suit their taste and preference. I did a Show Lobsters post too about it a while ago and the feedback has been good so far.
Well a day into the weekend, so far I’ve made more progress on the longest ever workbench build in the history of workbenches. It was meant to start a month ago, but got delayed due to delayed materials delivery. With luck it’ll be finished tomorrow.
Who knew only a few hours after I posted the above, I’d be spending 4 1/2 hours to recover a client’s database from backups + recreate missing records from logs, because of a malicious attack.
I’m figuring out how to market SaaS stuff. Spent lots of time building the service, applying it to developer’s repositories, convincing myself there’s value here, submitting patches, PRs, making UX changes etc.
It certainly isn’t an “if you build it they will come” landscape. Directly reaching out to projects that are good fits works but is very manual. Conferences and trade shows are usually not attended by in-the-trenches developers. I’ve heard Coverity spent years just showing their results on projects before gaining traction - it is quite a long timeline.
Flying back to Canada and getting a job permit at the border so I can finally start my new job.
Nice, congrats!
Awesome, congrats!
Congrats!
Yay, congrats!
I’m doing reading that I think will be helpful for my grad school application (master’s in Scottish Ethnology) and have just discovered the magic of org-roam for Zettelkasten-style note-taking in Emacs. This week I produced 3300 words of notes spanning 93 files, but I’m a little behind on my reading plan and want to catch up this weekend. There are also a few org-roam features that I’m aware of but need to learn how to use, such as tagging and launching a server that dishes up an interactive mind map
Would love to talk to anyone who has experience with humanities grad school, org-roam, etc!
org-roam is fantastic! Especially when paired with org-protocol, you can add a reffed entry straight from your browser. If you are in school and writing lots of papers, org-ref plays nicely with it as well which is a game changer if you re the sort to write lots of papers. Its one of those things where I wished I had a use case because the workflow was so nice.
Good to get the encouragement about org-ref. I’ve been handling that very manually right now and it’s not great.
On the one hand it’s a little overwhelming to figure out how to use all the relevant bits of the org ecosystem, but on the other it’s nice that I can be productive now and still have room to become more efficient by doing some toolsmithing. I’m excited for my setup to evolve. And, for that matter, am excited for when I have 100k words of notes spanning 500 files. A lot of papers is the goal!
It’s an interesting place to be: there’s still a lot of low-hanging fruit in the field (18th-century Scottish music history) since not many people are working on it and the internet has made so many more primary sources easily available. A lot of the top scholarship was written in the ’80s and was an admirable effort, but of course they were limited by having to physically travel to many far-flung libraries and use incomplete physical card catalogs to access materials.
Pottery, pottery and lots of pottery. Couldn’t go to the studio this week because of work and I really need it
I just learned about kintsugi!
Seeing my best friend for the first time in a about a year. I anticipate {emacs,billiards,alcohol} : -)
Starting my damn vacation.
Finally graduating with my bachelor’s in Computer Science!
I will try to figure out how Rails and React work together, if you wonder or you would like to offer help I am here just ping me. :)
I’m making up hours missed at work during the week. I’m almost done with my Transcranial Magnetic Stimulation (TMS) appointments (supposed to help with clinical depression, but I’m not seeing much of a change (it has around a 65% success rate and I guess I’m one of the 35%)). TMS has been kicking my butt. I get migraines and it has increased the number of headaches that turn into full-blown migraines.
But, I’m having a lot of fun at work. I’m steering the entire company into a whole new direction and it’s really fun. I hack on PHP, C, and C++ code all day. So it’s not like working over the weekend is a drain for me. I’m passionate about what I do, sometimes to a fault.
Writing an IRC bot in Rust that broadcasts messages from a channel to HTTP clients using HTTP streaming, so I can later use it to implement danmaku chat (chat messages overlaid on top of the video flying from left to right, as seen on Niconico and Bilibili) on a small stream that me and a few friends host every January. It’s my first time writing anything using concurrency in Rust, I predictably ran into some issues with borrowing but I somehow managed to solve them using an Arc.
AoC. I managed to stay on track so far so I’m gonna keep it up. I’m doing all problems in F# more out of convenience than anything and I’m finding it very nice; the .NET standard library is huge so it has lots of stuff built in that I’d have to write myself if I were using OCaml, and its strong typing makes it easier to work with complex data structures while still being terse enough thanks to type inference. And you can always use mutation as an escape hatch when you need it.
Playing some more Cyberpunk, I’m enjoying it a lot so far.
Let me know of you need IRC bot help. I have a thing that successfully ties irc and http together asynchronously with irc and hyper crate.
Yeah I’m using the same crates, I’m guessing the code’s probably similar too. I already got the basics down, just have to remove all the
.unwrap()
s from the code and maybe add some logging and all that. I’ll try to finish it by myself but I’d be interested in seeing your version!I also found https://github.com/dbaron/wgmeeting-github-ircbotbabauper useful read. Especially the stateless irc stream handler which makes it really easy to write tests for.
I am definitely going to fix that one bug in khefin that’s now more than three weeks old. Definitely this weekend. Definitely.
Hopefully putting the cherry on a rework/reorg/rename of the flags and envs for a CLI tool I’ve been working on this year (roughly: it rewrites command invocations in shell scripts to absolute paths).
It has been a little tiring and hard to focus on for a few reasons:
$GIT_COMMAND
) but a fair number just emerge from use. To avoid some of my weak/blind spots I just gave these hasty ad-hoc names during initial dev.Initially I was trying to iterate directly on the arg-parsing code, but I eventually settled on writing the manpage to stay focused on the concepts. I think I dragged my feet for long enough that the major patterns have all emerged.
Now, it’s ~time to actually implement the argument processing and do any related refactoring. :)
I did better than I hoped :)
General rename of options, refactor of option code, and most related busywork done.
I’lll be continuing to race Advent of Code at midnight!
I’m also working on pushing my music server to a stable release. This weekend, I’m planning on:
I thought of a way to combine a nix like package manager, with a less obtrusive packaging model. You might imagine it like half way between nix and docker.
Working on the prototype.
stuffing the pantry
mentally preparing for soft lockdown => hard lockdown this sunday
maybe go fatbike cycling in a forest nearby
filling up the tank for the last stressful week before christmas
Converting a home server from CentOS to Alpine. Not much else other than that… Maybe a few random projects around the house.
Family holiday stuff
When I have time, playing around with some more AOC fun. I’ve found that I need to be careful diving into the problems because I enjoy them so much I’ll end up coding for hours, even rewriting code that works to make it better. Much fun. Too much fun
Working some more on adding content to my professional wiki. I am quite enamored with using wikimedia
Playing Cyberpunk 2077 on my PS5. Finally got both yesterday and I am hooked. Spent 20 minutes customizing my character last night.
Also, will be wrapping presents and doing other productive things like rearranging the garage and some activity called “sleep.”
How has the performance been and graphics in general? I was thinking of waiting until they release a version optimized for PS5 (figured some of the bugs would be ironed out by then also).
Performance has been great for me, graphics are decent and I’ve experienced pop-in and random t-poses. Crashes are a regular occurence but not annoyingly so.
Aside from all this, there has been no lag. Unless you are a Deus Ex/RPG fan, I’d wait for the optimized version. I absolutely love the last two DX games and I’m enjoying exploring the world of 2077 via side quests.
I’ve been working on my java vm/aot compiler this week, and will spend some time this weekend. I have a problem that someone among you might be able to help with.
Consider the class Hashtable.Enumerator, which is an inner template of a template. Hashtable has two parameters, K and V, Enumerator adds a third, T. You cannot use that class directly, you have to ask either Hashtable.getIterator() or Hashtable.getEnumeration(), both of which are non-static template methods that also add a T.
K and V are obviously tied to Hashtable. But how about T? It seems obvious to me that the T in Hashtable.Enumerator is intended to be the same as in getEnumeration() and in getIterator(), but I don’t see any rule that ties the three Ts together. Where’s the rule?
I’m not sure I understand the question. The
Enumerator
‘s generic parameter is not explicitly tied at the language level to the generic parameter of the outer class, because you can use it for key and value enumerators. The type parameter in the constructor tells it whether it’s a key or value enumerator and soT
is equal to eitherK
orV
, depending on the value of that. Given that generic parameters are type-erasing in Java, this doesn’t actually matter for the generated code: in the .class file it’s anObject
.For someone writing an AoT compiler; however, this is interesting because the
final
field makes this class a great candidate for compile-time reification. You could eliminate thetype
field and provide two hidden subclasses ofEnumerator
specialised on the two possible values of the field. This requires a little bit of reachability analysis (what are the types passed to the constructor for thefinal
field?) and a bit of logic in your VM to ensure that your reflection code can handle the case that two objects have the same Java type but have isa pointers that refer to different concrete instantiations.How my compiler treats the Enumerator class depends on how much it knows about T.
When I look at the code it seems as if the T in newIterator() is meant to be the same as the T in Enumerator, but I cannot find a rule that lets the compiler assume anything at all, which puzzles me. Since I am frequently within a hair’s breadth of being the world’s worst programmer, I consider it possible that I might have missed something, and asked. But I don’t think I’ve missed anything.
The definition of T comes at construction time. With type-erasing generics, it doesn’t flow through the static type system at all (as I recall, it isn’t even present in the bytecode in the .class files). You have to infer it. Looking at the code, it doesn’t actually matter for code generation of
Enumerator
whatT
is, but it does matter to the callers because they’re going to do a redundant down-cast fromObject
toT
. You may want to do some dataflow analysis to figure out whether, for eachHashtable
a non-K
key or non-V
value can ever be inserted (I can’t remember the semantics of Java generics. If the compiler inserts a cast before the insert, then you just need to find out whether theHashtable<T>
is ever cast to aHashtable
). If it can’t then you can skip the down-cast (that said, with a good representation, down-casts in Java are three loads and two compares, so it may not be worth it.In a language with generics that doesn’t just erase them, it’s useful to do compile-time reification. C++ does this for templates, C# for generics, and so on. Here, you’re generating a specialised version of each function based on the values of the generic parameter. In Java, the type in generics doesn’t typically give you anything useful here but the values of
final
fields do. In a JIT’d version, you’ll likely see the methods being specialised on the value of thefinal
field if they’re hot. In an AoT compiler, you can see if there are branches based on the value of thefinal
field and look at the construction sites to determine whether you want to reify based on those values.The really fun thing for an AoT Java compiler is figuring out where to do devirtualization. Do you support dynamic class loading?
I had another think about the original question, and it was a little mistaken. The compiler complained that it couldn’t compute some blah blah, but nothing actually depended on the particular value that couldn’t be computed, so there was no reason for it to complain to me.
Yes, I do support some class loading, not 100% but enough to support all the java programs at all of the places I’ve worked.
Class loading really is unavoidable. Even printing hello world requires that; openjdk wakes up in the morning, yawns, looks at the environment variables, constructs a class name and loads that to use as foo-to-unicode converter, then it starts parsing command-line arguments using the new converter. Several of the widely-used libraries do similar things, e.g. Class.forName(prefix + “/handlers/” + tagName).newInstance().handle(parsedThing). I personally feel that that’s not in the java spirit. Java is a t-crossing i-dotting language. But it’s done and it has to be handled.
Devirtualisation is fun, particularly when it interacts with reification and inlining it can make the output look so slick.
I just realised that as of today, my code actually does not support that much class loading. Some, but not all I thought. There is a SMoP left.
I imagine in an AoT environment you can handle class loading in three ways:
dlopen
them. I think this is more or less what gcj did.I’d be curious about which approach(s) you’re using. Is your code online somewhere?
I’m close to the first two.
I interpret static initialisation and allow class loading there. Just before the first line of main(), I stop and generate native code, and after that point I (partly have done, partly will) look at the program and classify each class as either loadable or somewhat more devirtualisable. List and AbstractList are examples of the former, ArrayList of the latter.
After that decision, an LTO pass can devirtualise method calls to methods such as ArrayList.contains(), which isn’t marked final but may be known at link time to be effectively final because no classes in the VM extend ArrayList and reimplement that method. And of course, classes can be loaded but loading one that implements/extends an unapproved class will throw an exception. There are some very tricky issues to solve involving static initialisation.
My reasoning is that
It’s a bit fake and will OSGI fans will not like it, but there’s no performance cliff.
Source isn’t available and won’t be for a while. Among other things, I worry about time-draining discussions. There’s an attention-span cliff: People ask questions about this that are simple to ask, but answering the questions or understanding the answers are beyond the cliff. (I’ll answer you, of course. Your attention span is excellent.)
That sounds pretty close to what the .NET Native team did and it seems like a good direction to me. Good luck!
I’m somewhat curious because I was looking for a decent Java AoT compiler when I did the CHERI JNI work. A modern JIT on an (in-order, single-issue) FPGA softcore at 100MHz is painful, but cross-compiling from a nice fast machine is very easy. I ended up using JamVM, but because it’s a pure interpreter I couldn’t run anything other than microbenchmarks and get meaningful results. I’d love to do something a bit more real on the Morello boards when we get them.
Now I wonder how those implementations handle lambdas. Do you know?
Lambdas use class loading and much else. My own compiler reaches 120 stack frames. Doing all that at runtime… maybe their performance cliffs are as steep as my stack is deep.
I haven’t looked at lambdas in detail in Java, but my understanding was that they didn’t modify the JVM, a lambda is just a class that has a single method in its vtable to invoke it (plus a constructor to copy in all of the bound variables). The disassembly that I looked at from a class that contained a lambda just had an inner class representing the lambda with a mangled name for the invoke. I’m not sure why it would need class loading for this, it’s no different than the Java 1.0-style action-listener classes (inner classes with a single method conforming to a single-method action interface). I’d be curious to see examples that required class loading.
It’s a little more complicated, but you have the gist of it. The single-method class you mention is generated by javac using the .class format, but that data isn’t written to a .class file in the file system or a .jar, but rather output as a constant byte array inside the .class file that contains the lambda, and when the lambda is first executed, a compiler-generated method calls ClassLoader.defineClass() with that byte array and a javac-generated name. That’s also a bit of a simplification, IIRC it’s about right for OpenJDK+J9 but OpenJDK+Hotspot has some more moving parts, so that it can execute some lambdas without needing to load the .class. Don’t even look at a class called InvokerBytecodeGenerator, it’s bad for your soul.
As I recall, I first ran into this for java.util.time.format.DateTimeFormatterBuilder.QUERY_REGION_ONLY.
I’m also not sure why it was done this way. I did see something about code reviews… there is a possibility that code reviews in parts of the java source code may be stricter than in others, and that when the authors of the lambda system wrote it, they tried to add extra flexibility so that future changes would not need to change the source code with the highest level of scrutiny. I’m not sure. I’m just speculating.
That’s impressively insane. Do these byte arrays at least have a plausibly sensible name mangling so that you can AoT compile them speculatively and just expose the new classes when someone calls
defineClass
with that byte array?The naming scheme plain enough, but I don’t need to, I just follow about a half-dozen levels of indirection from the invokedynamic opcode. AoT compilation works for lambdas when I use the J9 variant of OpenJDK. No luck with the Hotspot variant.
I think the Hotspot developers must have a very high opinion of their future selves.
Relax and mess around with making video games. Probably work on game engine stuff as well.
Probably play more Wing Commander Privateer too. Never played it when I was a kid and I’m starting to think I missed out. Just need an engine upgrade and an afterburner and I’ll show those pirates who’s boss…
Improved the purchase flow for Graph Galaxy to avoid the misleading, and another cool feature just supported is that we can zoom in and out the graph by pinching now. So this weekend I am going to do nothing and have a rest ;)
Possibly follow-ups to my nixpkgs androidenv rework and more amateur radio shenanigans.
Update: I have successfully transmitted to the entire US and some of Canada by using a potted plant as an antenna.
Received some ZnSe lenses sooner than expected, so I might play around with focusing a thermal camera.
Otherwise, messing with OpenSCAD to see if I can’t figure out a way to generate multiple different structures I’ve had in my mead for a while. If I’m lucky, I’ll even have time to try fabricating them too.
The wife is working all weekend, so other than working on Advent of Code not a whole lot. Maybe I’ll finish my Metaprogramming Ruby book.
<mini book review>
don’t buy if you already know what metaprogramming is and/or you already know the Ruby object model. Having read Ruby Under a Microscope before, most of this book was already covered there. And I HIGHLY recommend the latter, it teaches much more and is just a better book (imo).
</mini book review>
Rust, Linux and VIM.
Ive been learning Rust for the past month or so, working on a command line suite of tools for viewing data from Destiny 2 activities:
https://github.com/mikechambers/dcli
Having a lot of fun with it, although its a ton of refactoring since Im learning so much.
I made the switch over to Linux this week, away from Mac and am really liking it. I decided to also give VIM (Neovim) a try as a main development editor, and while the learning curve seems super high, I can already see how good it can be.
So basically, im spending my weekend writing code in a language I dont know that well, on an OS which im still getting familiar with, in an editor where I have to look at a cheat sheet every time I want to copy some text or look at another file, and I couldn’t be more excited!
Btw, if any people with Rust experience have any comments on the project above (I am sure I am doing a ton of stuff wrong), I am open to any feedback.
Emacs Lisp and Common Lisp.
I have been teaching Common Lisp at a small online mathematics and computer science literature club (#spxy:matrix.org). One of the biggest hurdles while beginning to learn Common Lisp is that the popular books on the subject recommend Emacs and SLIME as the development environment. For someone unfamiliar with both of them, this can be quite a significant learning curve.
While there is Portacle to ease the initial effort to an extent, I am working on my own take on this problem: Emacs4CL. This takes a DIY approach to setting up one’s Emacs + SLIME development environment. It provides a tiny
~/.emacs
file that shows how to customize Emacs and automate package installation. The accompanying README provides a line-by-line walkthrough of the entire~/.emacs
file to help a beginner to Emacs make an informed decision about which lines from the file to keep and which ones to discard in order to better suit their taste and preference. I did a Show Lobsters post too about it a while ago and the feedback has been good so far.Well a day into the weekend, so far I’ve made more progress on the longest ever workbench build in the history of workbenches. It was meant to start a month ago, but got delayed due to delayed materials delivery. With luck it’ll be finished tomorrow.
Who knew only a few hours after I posted the above, I’d be spending 4 1/2 hours to recover a client’s database from backups + recreate missing records from logs, because of a malicious attack.
I’m figuring out how to market SaaS stuff. Spent lots of time building the service, applying it to developer’s repositories, convincing myself there’s value here, submitting patches, PRs, making UX changes etc.
It certainly isn’t an “if you build it they will come” landscape. Directly reaching out to projects that are good fits works but is very manual. Conferences and trade shows are usually not attended by in-the-trenches developers. I’ve heard Coverity spent years just showing their results on projects before gaining traction - it is quite a long timeline.