“…The conjecture that I pose is that many of the properties that would usually require dependent types to ensure at compile time, can be described in Liquid Haskell specifications. …”
This conjecture, I hope is true. And it could very well apply to anything from Janet, to Pascal, to Python or Typescript.
There are seems to be two views on what the type system is for (I am sure, i am simplifying here bit too much):
a) it is a compiler hint for how to translate the intent of the programmer to machine (including the intent of a lifecycle for a variable)
b) it is a compiler hint for how to evaluate the correct usage of the variable (or a collection of variables) of a given type throughout the code
Some programming languages adapted (a) and are slowly switching to incorporate (b). eg. C++
Some languages adopted (b) from the start, and work on adapting (a).
But if I follow the above conjecture, why these two views should be ‘incorporated’ into a single type system . Why not keep that separate, and have essentially 2 type systems expressed differently through out the program.
My first projects are usually lowering CI/CD times, often by improving the unit-test harness. It accelerates the entire team and almost never has downsides.
I have been learning that you have to have a test suite to reduce build times this way
Yup it’s definitely an interesting problem. BTW I think the Julia language coined “the two language problem”, and they specifically “complained about” Python and C in scientific computing.
I think they did a great job in that domain, but Julia’s not a general purpose language.
I don’t think Mojo is either – Lattner says it’s machine learning first. (that implies a lot of exotic features like auto-tuning, auto-differentiation, GPU deployment, etc.)
Graydon wrote about “Ousterhout’s Dichotomy” several years ago – Ousterhout being the author of Tcl which was designed to integrate with C:
I agree that using bash + Python + Go is more annoying than just 2 languages. 3 is worse than 2 ! Oils is trying to fold some of those use cases in, but it’s hard :)
I never used Objective C, but it does seem like a good mix of Smalltalk/Python + C, with an unfamiliar syntax. It never seemed to make it outside Apple’s ecosystem though.
But also Apple did not use it for “shell scripting”; it was more for gluing objects together I suppose.
After watching the podcast, I think Mojo is very cool though. It makes a lot of sense in a lot of ways. The team is extremely accomplished and knows way more than me about languages. And they already did some impressive stuff integrating CPython.
They are working on a downloadable version, so I’ll definitely check it out then. I really wonder what all the caveats will be when you type “import pandas” – is it going to crash? Or it will fall back to CPython? And you copy from CPython to Mojo, or sometimes you can manipulate the dynamic Python objects from static Mojo code?
Or do you have to type “py import pandas” from Mojo, etc. ?
That is the part that was unclear to me, and the podcast sorta addressed that, but not fully. But he also said Mojo is something like 7 months old. So yes the “sell” made people think it was a lot further along than it is.
It sounds like you will definitely be able to write a Rust-ish language in Python syntax, and it will be fast, and deployable to a GPU. But that’s a totally different thing than “import pandas”.
Part of the article that interests me is thinking about the two language problem, ie Unix is C plus Bash. The two language problem is a subset of the static vs dynamic problem. Static is faster and harder. Dynamic is slower and easier. And the problem recurs on multiple levels, ie unikernels vs normal OSes, static linking vs dynamic linking, static types vs dynamic types, compiled apps vs plug-in systems or interpreters, native apps vs Electron apps, etc etc.
My problem with Python today is that if I just want to run some scripts, Bash has less boilerplate. And if I need speed, Go is faster. So Python is getting squeezed out of the middle of a two language system for me. But TFA seems to posit that Python can grow out to cover more static ends of the spectrum. It’s an intriguing idea.
I’ve been disappointed with how Sonos products have lost functionality over the years (although the fault may lie with some of the services they pair with, not sure), so I greatly enjoyed reading these different ways of breaking them.
Another more positive thing is that it’s kind of wild to me how many talented people are around in the modded minecraft community. People were reversing the Java payloads within minutes of getting their hands on it.
Kind of expected if you send obfuscated malware to a community that once deobfuscated the entire vanilla game without assistance from Mojang.
Incident journalist here. It’s been a fun 24 hours :P
I think the thing to highlight here is how targeted this was at the modded Minecraft community. We’ve seen malware be uploaded to mod repos before, but they were always isolated instances, didn’t attempt to replicate, and were quickly dealt with.
This attack intentionally crawls the entire file system looking especially for Minecraft mods to propagate itself to.
In general, I’d say the modded mc community’s development practices are pretty nascent – no reproducible builds, little code signing, etc. It’s all been based on trust and authors’ reputations over the past decade+. The followup here is to look into what we can do as a community to build that muscle more.
I don’t expect Microsoft or Mojang to do much here, as they’ve always had a laissez-faire approach to the Java modding community.
Please don’t take anything I say here as gospel, I’m just rambling after some long hours. See the github repo for the latest updates.
People at my office seem to think that they’re going to be able to “switch to Mojo” and just keep on writing Python code, and somehow get amazing speed ups. That sounded too good to be true to me (especially as we’re mostly using pandas, numpy etc which are already using C or whatever under the hood).
Seems like the faster-than-Python part of Mojo is the stuff that’s in the “superset” (ie not vanilla Python). Is that about right?
My impression at Google was that there definitely was a tendency to produce highly complex, highly integrated vertically systems:
a good, thoughtful, polished design doc is produced for a complex, all-encompassing solution (the perf process incentivizes design docs to become a kind of performative art).
then the outlined solution gets executed, solidly and professionally, with due repayment of tech debt and iteration.
it becomes both too good and too complex to replace, so it ossifies, until a Great Deprecation comes.
With perf happening every half a year and some distance to actual customer needs, this gets waterfall-ish, producing solutions that in many cases could be better with more ad-hoc and more chaotic process. Actual engineering feedback and true iteration might get lost in the process.
Thanks, I’ll try it out! Also, thank you for reading, and for writing about Alloy in the first place – it was your blog post that made me aware of the tool.
When I was big into Minecraft as a tween and young teenager, I had a mod phase (where I installed a bunch of junk, not wrote mods myself). Looking back, even though I was quite tech ‘literate’ for my age, I took 0 care when downloading and installing mods. They were all hosted on weird CDNs, I’d download random .JARs and random EXEs to help me install mods. Its a wonder that, AFAIK, I never got a virus or malware. And in hindsight, this seems like a great vector for kids who don’t know any better and just want a virtual voxel girlfriend.
How do you sandbox Minecraft with the necessity for this like all the 3d graphics drivers and stuff to carry on working? Just run it as a different unprivileged user account?
This conjecture, I hope is true. And it could very well apply to anything from Janet, to Pascal, to Python or Typescript.
There are seems to be two views on what the type system is for (I am sure, i am simplifying here bit too much):
a) it is a compiler hint for how to translate the intent of the programmer to machine (including the intent of a lifecycle for a variable)
b) it is a compiler hint for how to evaluate the correct usage of the variable (or a collection of variables) of a given type throughout the code
Some programming languages adapted (a) and are slowly switching to incorporate (b). eg. C++ Some languages adopted (b) from the start, and work on adapting (a).
But if I follow the above conjecture, why these two views should be ‘incorporated’ into a single type system . Why not keep that separate, and have essentially 2 type systems expressed differently through out the program.
I have been learning that you have to have a test suite to reduce build times this way
It’s been exhausting as is, I don’t know if I have the mental effort for this case too right now. 😔
This seems too profound for me.
Yup it’s definitely an interesting problem. BTW I think the Julia language coined “the two language problem”, and they specifically “complained about” Python and C in scientific computing.
I think they did a great job in that domain, but Julia’s not a general purpose language.
I don’t think Mojo is either – Lattner says it’s machine learning first. (that implies a lot of exotic features like auto-tuning, auto-differentiation, GPU deployment, etc.)
Graydon wrote about “Ousterhout’s Dichotomy” several years ago – Ousterhout being the author of Tcl which was designed to integrate with C:
https://graydon2.dreamwidth.org/3186.html
https://graydon2.dreamwidth.org/189377.html
I agree that using bash + Python + Go is more annoying than just 2 languages. 3 is worse than 2 ! Oils is trying to fold some of those use cases in, but it’s hard :)
I never used Objective C, but it does seem like a good mix of Smalltalk/Python + C, with an unfamiliar syntax. It never seemed to make it outside Apple’s ecosystem though.
But also Apple did not use it for “shell scripting”; it was more for gluing objects together I suppose.
Yeah I can’t tell what will happen, but it won’t be that simple, at least not soon.
Here is my speculative 4-level hierarchy of implementing Python-like languages, with running NumPy and Pandas dead last - https://lobste.rs/s/qbcjsl/mojo_future_ai_programming#c_a5vgqv
After watching the podcast, I think Mojo is very cool though. It makes a lot of sense in a lot of ways. The team is extremely accomplished and knows way more than me about languages. And they already did some impressive stuff integrating CPython.
They are working on a downloadable version, so I’ll definitely check it out then. I really wonder what all the caveats will be when you type “import pandas” – is it going to crash? Or it will fall back to CPython? And you copy from CPython to Mojo, or sometimes you can manipulate the dynamic Python objects from static Mojo code?
Or do you have to type “py import pandas” from Mojo, etc. ?
That is the part that was unclear to me, and the podcast sorta addressed that, but not fully. But he also said Mojo is something like 7 months old. So yes the “sell” made people think it was a lot further along than it is.
It sounds like you will definitely be able to write a Rust-ish language in Python syntax, and it will be fast, and deployable to a GPU. But that’s a totally different thing than “import pandas”.
Part of the article that interests me is thinking about the two language problem, ie Unix is C plus Bash. The two language problem is a subset of the static vs dynamic problem. Static is faster and harder. Dynamic is slower and easier. And the problem recurs on multiple levels, ie unikernels vs normal OSes, static linking vs dynamic linking, static types vs dynamic types, compiled apps vs plug-in systems or interpreters, native apps vs Electron apps, etc etc.
My problem with Python today is that if I just want to run some scripts, Bash has less boilerplate. And if I need speed, Go is faster. So Python is getting squeezed out of the middle of a two language system for me. But TFA seems to posit that Python can grow out to cover more static ends of the spectrum. It’s an intriguing idea.
Huh. I stand corrected!
py-spy is cross platform, pystack seems Linux-only.
I’ve been disappointed with how Sonos products have lost functionality over the years (although the fault may lie with some of the services they pair with, not sure), so I greatly enjoyed reading these different ways of breaking them.
Another more positive thing is that it’s kind of wild to me how many talented people are around in the modded minecraft community. People were reversing the Java payloads within minutes of getting their hands on it.
Kind of expected if you send obfuscated malware to a community that once deobfuscated the entire vanilla game without assistance from Mojang.
Incident journalist here. It’s been a fun 24 hours :P
I think the thing to highlight here is how targeted this was at the modded Minecraft community. We’ve seen malware be uploaded to mod repos before, but they were always isolated instances, didn’t attempt to replicate, and were quickly dealt with.
This attack intentionally crawls the entire file system looking especially for Minecraft mods to propagate itself to.
In general, I’d say the modded mc community’s development practices are pretty nascent – no reproducible builds, little code signing, etc. It’s all been based on trust and authors’ reputations over the past decade+. The followup here is to look into what we can do as a community to build that muscle more.
I don’t expect Microsoft or Mojang to do much here, as they’ve always had a laissez-faire approach to the Java modding community.
Please don’t take anything I say here as gospel, I’m just rambling after some long hours. See the github repo for the latest updates.
People at my office seem to think that they’re going to be able to “switch to Mojo” and just keep on writing Python code, and somehow get amazing speed ups. That sounded too good to be true to me (especially as we’re mostly using pandas, numpy etc which are already using C or whatever under the hood).
Seems like the faster-than-Python part of Mojo is the stuff that’s in the “superset” (ie not vanilla Python). Is that about right?
My impression at Google was that there definitely was a tendency to produce highly complex, highly integrated vertically systems:
With perf happening every half a year and some distance to actual customer needs, this gets waterfall-ish, producing solutions that in many cases could be better with more ad-hoc and more chaotic process. Actual engineering feedback and true iteration might get lost in the process.
Thanks, I’ll try it out! Also, thank you for reading, and for writing about Alloy in the first place – it was your blog post that made me aware of the tool.
When I was big into Minecraft as a tween and young teenager, I had a mod phase (where I installed a bunch of junk, not wrote mods myself). Looking back, even though I was quite tech ‘literate’ for my age, I took 0 care when downloading and installing mods. They were all hosted on weird CDNs, I’d download random .JARs and random EXEs to help me install mods. Its a wonder that, AFAIK, I never got a virus or malware. And in hindsight, this seems like a great vector for kids who don’t know any better and just want a virtual voxel girlfriend.
That sounds like an aspiration. It doesn’t sound “soon-to-be open sourced” for any definition of “soon” that I like.
This is a document for people in SRE who presumably don’t need a definition of what SRE is.
How do you sandbox Minecraft with the necessity for this like all the 3d graphics drivers and stuff to carry on working? Just run it as a different unprivileged user account?
SRE = Site Reliability Engineering/Engineer