Threads for weaksauce
-
Gotta say magit is the best git gui out there and probably one of the best pieces of software ever written.
-
-
This is a tool I use pretty regularly.
On most sites I use, I automate something. Since I don’t like to keep two versions of the code (plain and in bookmarklet form), I wrote this editor to convert the code in both directions.
You can click on the question mark to get more infos about bookmarklets in general, this editor in particular, and some examples.
-
-
-
Is that a simple URL rewrite?
-
I pasted it in this, then took that long single line over to Beautifier, and it’s 75 lines long.
-
-
It’s from this generator, though I might try @no_gravity ‘s, and I’ve seen but not tried another.
-
This works for me:
let your_instance="masto.ai"; document.location=( "https://" + your_instance + "/authorize_interaction?uri=" + encodeURIComponent(document.location) )Before you turn it into a bookmarklet, set your_instance to the Mastodon instance you use.
-
-
-
-
-
-
I use bookmarklets on pretty much every site I visit regularly, and I never had a problem with CSP.
Can you give an example of a bookmarklet and a website that do not work together?
-
-
That wasn’t the case for ~5 years that firefox wouldn’t run bookmarklets on CSP sites at all, according to the history here: https://bugzilla.mozilla.org/show_bug.cgi?id=866522
Sounds like they did finally fix the case that doesn’t need subresources, though still haven’t fixed it for those that do.
-
-
-
-
-
amw-zero
4 months ago
|
link
| on:
From Ruby to Node: Overhauling Shopify’s CLI for a Better Developer Experience
First, it’s crazy that Shopify even allowed a non-Ruby tool, given how much they invest in the Ruby ecosystem. I know a few people who have worked there, and while they do use some other programming languages, the vast majority is Ruby, and they are pretty forceful about using Ruby whenever possible.
Second - why not just say “we switched to Node because we flipped a coin?” None of the reasons given in the article are compelling.
Moreover, the Hydrogen team, which had built some tools on Node, started to consider building a new CLI instead of building Hydrogen workflows into the Shopify Ruby CLI so their users didn’t need a Ruby runtime in their system. Hydrogen developers expect the npm install command to resolve all the dependencies that they need to work on a project. With Ruby as a dependency, that mental model breaks and they can easily run into issues where the CLI refuses to run because of additional steps needed.
I would have just given devs a script to setup their local env vs. rewrite the whole tool.
-
So they don’t need a ruby runtime. But they need a node runtime. Lol.
I would have understood if they had used the static binary excuse (and even then, mruby). They could have been honest and said “creator left the time, we don’t like ruby in this team, but we need a story to sell node to engineering managment”
-
Sounds exactly right to me. It’s silly - they even state outright that
Ruby is the language most developers are familiar with
followed by
Shopify wants internal teams to contribute new ideas into the CLI.
and then the illogical conclusion
We were left with either Ruby or Node.
followed by some blah blah nonsensical bit about Node allowing multiple versions of the same module in one project (which is highly questionable IMO).
-
-
-
-
you could write a small tool to append to a log file of your observations. maybe it’s an extension for vscode or whatever you’re using to append to a notes file similar to an org file. maybe it notes where you are in the source code with filename line number and sha of the commit(though that is probably not super useful since your code may differ from what was last committed) and whatever note you have.
could have the tool have a concept of “working on” that starts a new note under the working on name and appends to that. all could be under the project directory under
.notes/sounds like a useful tool actually.
-
Can anyone who’s tried sorbet comment on why you’d want to use this instead of crystal?
-
your question is ‘why x instead of y’, but these two are not direct competitors/alternatives, though both are related to ‘type checking ruby(-esque) code’.
sorbet is a ‘checker’ that can be used on top of an existing ruby codebase, which also means it can be gradually adopted. this is similar to mypy in the python world.
crystal, on the other hand, is a separate (though similar looking) programming language with no direct interop with the ruby ecosystem. instead, it compiles to machine code, as opposed to running inside a ruby interpreter/vm.
-
-
Thanks. “Ability to gradually migrate” does answer the question
-
-
-
I am more interested in that exactly: What have people’s experiences been between RBS and Sorbet? They seem to approach the same problem, but I suspect they’re not entirely overlapping. As such, I find myself wanting to gather the experiences of someone who has tried both before deciding which I might gradually introduce into a project.
-
I wrote an online poker virtual table in ruby without any typechecking at the start of the pandemic. It went swimmingly.
After playing for a few weeks I realized about a dozen tweaks I wanted to make. Diving back into the code was a little difficult and I grabbed Sorbet (my first go at it) and I found it really helped me keep things straight. I used the types sparingly at first, mainly to document my assumptions about parameters for certain functions. In some places the type signatures got complicated and I took that as a hint to refactor. Decent experience all around. The worst part was the way it loaded gems when probing for type signatures in my dependencies. Thankfully that was a smaller project, probably 2k LoC with a half dozen dependencies. I can’t imagine how a large rails application would fare in that process.
Later RBS was released and I figured I’d port my game over to it since the official nature of RBS seemed to challenge Sorbet and the future might demand I make the change. I didn’t like any part of it. The definitions being in a separate file was probably the worst. It meant that those useful notes about my assumptions (in the form of type signatures) were an extra step away. The error messages coming from the Steep tool were significantly less understandable than the Sorbet messages. It ran slower in most cases too.
My current day job doesn’t use ruby but if it did I wouldn’t necessarily advocate for my team to adopt either tool. If someone else did, I’d be happy to help bring in Sorbet and I would argue strongly against RBS. The experience of no type checking was better than RBS.
-
-
-
-
-
I particularly enjoy the Change In/Around commands:
ci"replaces the inside of whatever double-quoted text block you are in,ca'replaces the single-quoted text block including the quote characters, and there are variants for various brackets, braces, parens, and paragraphs!-
another fun fact about ci” is that you don’t even need to be inside a double quoted string or whatever… it will go from where you are on the line to the next double quoted string and let you change that.
so ci” on this at the | :
lor|em “ipsum” foo bar
would change it to this with you being in insert mode
lorem “|” foo bar
-
Best paired (…) with https://github.com/tpope/vim-surround which is the first plugin I miss on a fresh installation of vim.
-
See :h text-objects for more details. You can also create your own using
omap.-
I think this is a great introduction to writing your own text objects:
https://web.archive.org/web/20201108142554/https://vimways.org/2018/transactions-pending (web archive link because the site currently has an out-of-date certificate)
I mention is below, but kana’s vim-textobj-user is also very helpful.
-
-
If you like text objects, I definitely recommend checking out these. (People may already know about them, but just in case…)
- https://github.com/kana/vim-textobj-user/wiki (a list of user-defined text objects made with the help of kana’s vim-textobj-user)
- https://github.com/machakann/vim-sandwich (a supercharged variant on vim-surround—the docs could be clearer, and more complete, but the plugin is very powerful)
-
-
Yup, we should learn about text objects sooner rather than later and hen get the kana/textobj-user plugin which will also give you a whole set of new text objects to play with. There are also alternate approaches like targets.vim.
-
-
g commands when using it from the colon mode (command mode? i’m not sure what that’s called) same way as you’d do a find and replace in a file
something like
%g/^ *$/dwill delete the empty lines in a file or%g/WARN/dwould delete warning lines in a loguseful for deleting all the lines that you don’t want to deal with in a log file or you can also perform other operations with it by replacing
d.
-
Mine is moving ctrl to my capslock key, and then becoming proficient with the various
c-prefixed keys like<c-w>for window navigation and<c-p>or<c-x><c-f>and others for completion.-
-
I agree! I have my caps key mapped to esc on tap, and ctrl on hold.
If I had to only pick one between ctrl and escape I would pick ctrl for remapping caps lock. The more I use my mappings that begin with ctrl, the more I appreciate having a ctrl key within easy reach. Also useful for other programs. And
<c-[>sends the escape key, so it’s not too hard to press escape when needed. -
I don’t think there are many situations where ctrl-c doesn’t do the same thing as escape. It’s not a bad muscle memory anyhow for a similar concept everywhere else. I don’t remember the last time I’ve reached for escape in vim…
-
-
-
-
-
No, in all seriousness, the documentation sucks on this… So I’ll explain it how it was explained to me by a colleague:
Vim stores edit history and undo as a tree. For example, if you do
A,B,C,D, undo, undo,E,F, undo, undo, undo,G, vim will store this history:D F | | G C E | | | | +----+---+ | | | B | | | +----+-----+ | Ag+andg-will navigate between the “tips” of this tree, so betweenD,FandG. One goes back, the other forward. I never know which one is which betweeng+andg-to go the way I want, so I always try both until I realize it’s going the right way :) .-
-
Especially when you throw in a visualizer for it like mundo. Emacs has it as well.
-
-
-
-
-
JetBrains IDEs have a local history feature for this. It’s fantastic, as it shows you a merge view between your current state and the selected history state. So you can still edit the current state and cherry-pick bits from the diff trivially.
-
-
-
Trying to make an Emacs-package + firefox extension which will allow me to “archive” my firefox tabs in an org-mode subtree. I am sick of having 30 tabs open out of which I only interact with 5 in a day, but am too paranoid to delete them. I am aiming for a
consult/avy-bufferlike UI for my tabs; so I can just archive a tab into a plain text file, and then go through my list of archived tabs. Maybe tag tabs as well, like Chrome’s tab groups. I’ve tried a lot of FF addons which provide a tree/groups for tabs, but I’ve found them to be too buggy and have often lost huge lists of tabs. I want my tabs in a plain-text file which is under VC so I can procrastinate them in peace.PS I use EXWM, so when I talk about
consult-bufferlike UI, I mean actual Emacs completion UI, not somethign in Firefox like Tridactyl.-
Make sure to check the native messaging part of the WebExtension. It worked for my “what was that github repo from X days ago” use case.
-
I’ve been looking into that. I was unable to figure out how to get Firefox addon talk to Emacs. Because native messaging needs an app which accepts input from stdin and outputs to stdout, and this “native app” is started from browser’s side. But I couldn’t find a way to launch an emacsclient which can behave like this. So I’ve decided to write a small script (“native app”) which:
- Starts on browser’s request
- Along with stdin, watches a file for input
- Browser sends its messages via stdin. Emacs sends its messages by writing to the file.
- Browser gets its messages via stdout. “native app” will directly call
emacsclient --eval '(some-elisp message)'to send messages to Emacs
I couldn’t write any code to verify this though; I hope it’ll work.
-
-
FWIW I recently found TabStash and it’s working quite great for me as of now, interface-wise. It let’s you easily archive tabs into bookmarks - so not into org-mode, but maybe it could help you solve part of your plan, e.g. you could sync just the bookmarks it created to org-mode, or fork the plugin and tweak its source to your liking.
-
Why not just have a native messaging webextension that calls a small script that appends to a text file? if you want to have it in some special thing that needs to be processed by emacs for whatever reason you could have a minor mode or just a command that any time you switch to a particular buffer or run that command it takes from that file and merges it into the “master” file and then truncates the file or delineates the file in some fashion to indicate where to start for the next import.
-
-
Understanding this code is over my head right now.
Can someone recomend me what is the standard method to write a simple media player?
Is it genrally like this:
- Read the frame, display on a canvas. Basically pixel by pixel
- Sleep a very small amount of time.
- Read next frame, display
Of course, a lot of thing can be optimize, but is that a simplify version of how a media player work?
-
https://github.com/leandromoreira/ffmpeg-libav-tutorial seems to be a useful tutorial about this. Though, have to admit, it’s been more than 10 years that I touched libav*.
-
-
basically yeah. though the way codecs work is that they don’t store all the data for every frame. they take a difference of frames and then calculate that. there will be different types of frames stored for different purposes and every so often you have a “check” frame that is kindof a reset frame that stores the full picture instead of a differential picture and that frame is used for the intermediary frames between those check frames (I frames vs B frames vs P frames) https://ottverse.com/i-p-b-frames-idr-keyframes-differences-usecases/
and typically you would not draw the frame directly but use some kind of buffering mechanism so that the frame can be shown on the screen instantaneously.
-
I think the docker desktop license thing is grossly overstated. if you are rocking in a year a 5,000 dollar a month license for all of the max hypothetical 249 employees at the max business rate of 21 dollars each is not all that compelling considering it’s “business-critical software” per them. the real scenario is one of the lower tier licenses probably work and it’s more than likely way lower than 250 employees that use it. if you have 250 employees it’s more than likely you are pulling in a lot too.
-
Kitten-proofing the apartment. We got two kittens this week, and every day they discover something new to get into.
Also, completely taking apart my NAS box and trying to figure out if it has a short, and where it is. I tried moving it (kitten-proofing), and, when I laid it on its side, its PSU died. It smelled like it completely burned out. So I bought a new PSU, and it worked briefly… until I flipped the tower on its side again, and now it won’t start anymore, again. Hopefully this one’s not dead.
-
-
The choice is between an Electron app or no macOS app.
Really? Were there no other cross-platform GUI toolkits you could have used?
-
I wrote and maintain an Electron application. If I weren’t using Electron, if the application would exist at all, it would be Linux-only. There’s a very simple reason why: we use interactive SVGs a lot throughout the application, because we need an image that resembles a keyboard, and SVGs make that easy. GTK can’t do that, QT can’t do that. They can display SVGs, sure, but they don’t support interactive SVGs, only via a webview. Since about 90% of the stuff the app does is tied to interacting with that SVG, there’s no point in using a native toolkit, when most of it will be spent in the same WebKit anyway.
We could, of course, build our own widgets instead of using SVGs, but that’s way more time investment than we can afford. We - well, I - explored the native options. None of them would’ve allowed me to build the app as it is today. Electron did.
Sometimes there are good reasons for using Electron.
-
-
Honestly, I have a dim view of cross-platform toolkits having used them and developed on them. UX-wise, they’re about as bad mouthfeel-wise as Electron. I had to do quite a bit of work to get Qt to have not awful mouthfeel on Mac OS. I find most of the advocates of cross-platform UI toolkits tend to be on Linux, which was historically a pretty low bar UX-wise and clamouring for any application. You’d get better performance, but it’s not a strong argument from the UX side.
-
Honestly, in my experience they’re worse than Electron apps. At least standard text input shortcuts work in Electron! The “cross-platform” toolkits tend to look non-native, just like an Electron app (although often more dated), and have very… weird shortcut support, e.g. macOS Emacs-style keyboard input.
-
-
What would you recommend? I don’t mean this adversarially, it’s just that every time I’ve looked for a good cross-platform GUI toolkit, I’ve come back disappointed. I hate working with Qt because Qt bindings vary in quality across languages and I’d rather not use Qt’s stdlib over the C++ stdlib when writing C++ because I have much more experience with the C++ stdlib. Gtk similarly has some pretty poor bindings in certain languages. Tk is probably the toolkit I’ve enjoyed using the most, but it’s rapidly losing mindshare from what I can tell.
-
I agree with you that the state of cross-platform GUI toolkits is bad. I love GTK on Linux, and as far as I can tell, its language bindings are consistently good, even when they’re third-party. But GTK support on Windows is second-class, and on MacOS has historically been terrible (but is maybe getting better?).
When I was looking at a toolkit to use for a cross-platform graphical Common Lisp app, the best I could find was Tk, despite its limitations.
-
-
I think so? There are options you can use, but there are no really good options. https://blog.royalsloth.eu/posts/sad-state-of-cross-platform-gui-frameworks/ is a nice survey.
-
They missed some third-party XAML implementations like https://avaloniaui.net/ It’s going to be closer to javafx, but with a great RAD environment (visual studio)
I hope MAUI will get a community Linux back-end. That would make it a good alternative too.
-
-
i mean you have to buy into the react paradigm but react native can compile to windows and mac in addition to ios and android
-
I guess people really haven’t tried how fast you can get stuff running with QT. Yeah it’s not completely native (and it can also be used the electron way since some version), but that’s not something you get with electron either. To be fair, you have to use c++ (with QT additions) or python for it..
-
I inherited a Qt project once. It was awful. I’ve never used Electron, but I know enough about it to pick it over Qt in most circumstances.
Not sure there are many other options if you’re targeting desktop. Proton looks promising. Microsoft’s React Native for Windows and Mac does as well. Both are similar concepts with ostensibly less overhead than Electron. Anyone here try those?
-
-
I use mailmate for osx. not what i’d call a beautiful app but it’s lightning fast for search and sort over many accounts which is important to me.
-
Because some Gems have become unfathomably broken and starting from a clean slate is the only thing that seems to work. I’d say I’ve had to do it a good 5 times.
-
Ah, that may be one reason people are trying to uninstall Ruby.
-
-
if you update your system beyond what the native C extensions to the gem that were compiled to the system expect you have to rebuild the gems again… that’s a very common thing in ruby and not really the fault of ruby at all. the price of admission to have native extensions. I expect python to have a similar issue. ruby probably needs to be reinstalled too come to think of it since that’s built expecting some native libraries to be there.
-
-
I’ve been looking for good cross-platform FOSS alternatives to GitHub tools or VS Code for pretty git interfaces, and gitui is the winner for me so far. It’s definitely made it easier for me to manage git repos even with its incomplete feature set, and as this release demonstrates it is quickly gaining functionality, making it even more pleasant to use.
I will mention a specific pain point for me: gitui supports creating tags and pushing tags, but it doesn’t appear to support annotating tags. For git commits it supports opening the commit message in your editor if you want to write more than one line, why not do something similar for annotating tags?
-
i don’t know if you’ve tried magit but it is hands down the most phenomenal way to use git in my opinion. downside is that you have to use emacs but you could probably tie it to a pared down version of it which i might just try to get going this weekend. despite it being tied to emacs it’s well worth the time to load up and use it. spacemacs has a decently polished layer that includes magit and vim customizations that make it all work well together.
-
gitui does not force any particular text editor on you 👍
-
Neither does magit, in fairness, except for the editing of commit messages. And even then, you can use vi keybindings.
I intend no snark by pointing this out; I have colleagues who used other editors and only touched Emacs for magit.
That said, it’s great to see a good GUI for Emacs building momentum. I’ve seen so many new and junior Devs utterly confused by Git. (This is in part to do with how it’s usually taught, but also that the CLI is IMO quite confusing to learn).
-
-
-
I’m still holding out for a gesture-based interface to Git.
They’d all be rude gestures, mind you ;)
-
-
-
it’s great to see a good GUI for Emacs building momentum
gitui is not “for” emacs. its a general terminal application
-
Ah, braino. I meant “for git” :-/
-
-
-
-
-
-
Hi glad you like it! I never annotated a tag 🙈 but it sounds like a reasonable feature request, please file it over on github👍
-
-
-
Not sure if posting this fits the rules, but I semi accidentally stumbled upon this a while ago, and it markedly improved my life, so I figured it might be useful to share :)
-
-
I am not a very heavy magit user, but, for the features I use (staging in chunks, instant fixup, reword, etc) it feels surprisingly close. Before this, I had an
mgshell command which fired up Emacs+magit when I wanted to do git stuff. Now I do all that from VS Code without a context switch.Interactive rebase is a bit more clunky than in Emacs though. On the positive side,
gis mostly not needed any more, the state updates reactively. -
-
-
-
I love Postgresql, and I’m really grateful for posts like this to balance my opinion and present a well-argumented counter-argument. However, when I read posts like this I mention a pattern: all these downsides seem to be relevant to really huge, perfomance-sensitive projects with enormous loads. The kind of projects where you likely have over a hundred of developers, and probably should move away from one master relational database to micro-services and message queues as a backbone of your architecture.
What I’m saying, this kind of criticism just highlights the fact that Postgresql is probably the best choice if you’re smaller than that, both in terms of load and team size.
-
I almost hit the XID wraparound (I was 2-3 days from an outage; it was bad enough that AWS emailed me) when my company only had 4-5 devs and 100 customers. And I’ve hit connections-related performance issues at least four or five times through the last five years, at relatively modest scale (<100 application servers, in an overwhelmingly read-heavy workload). This affected us as recently as yesterday, as we are bringing a Timescale DB (which is a Postgres extension) into production and we are faced with the prospect of tiered pgBouncers or breaking the Timescale access into its own microservice.
I love Postgres but these are real and they can hit earlier than you expect.
-
I love Postgres, and I was strongly considering moving to it…but for our use case, it simply requires too much care and feeding. We ship databases to customers in appliance form, meaning all the maintenance has to be invisible and automatic and Postgres simply isn’t there.
-
Having worked for a company that did that and been responsible for Postgres tuning, I say it can be done. In nearly 15 years of shipping out a postgresql db as part of an appliance, I have not seen any of these problems.
Edit: Except for long upgrade times. That one is a PITA.
JADP.
-
I’d love to hear about your experience if you have time.
Also, I’m drawing a blank on “JADP”…
-
Just Another Data Point?
-
Just A Data Point. A weird acronym I picked up from old-timey online fora like USENET, and The Well.
I probably can’t say too much more about my experience postgres tuning as.
- It was for a company, and might be considered propietary information.
- It was about 5 years ago and I really don’t recall all that well what I did.
sorry, just know that these are really rare problems if you’re dealing with the limited scale inherent in incorporating postgresql as part of an appliance. Most of them deal with syndication, or ginormous tables. They’re web-scale problems, not appliance scale problems
-
-
-
-
-
-
-
Definitely echo that these problems (and others) can hit you way before you get to 100 devs. We were running into the pains mentioned in this article (which admittedly is a more general crritique of SQL databases, and lands on MySQL over Postgres) at more like 10 developers.
It absolutely isn’t that hard to run into the pitfalls of SQL databases at relatively small scale, especially if you’re using them for OLTP workloads in services where uptime/response times matter.
-
-
all these downsides seem to be relevant to really huge, perfomance-sensitive projects with enormous loads. The kind of projects where you likely have over a hundred of developers
A number of these issues affect “normal” users of PostgreSQL as well:
-
Replication is something you may want even on smaller use cases.
-
“Wasted” space from the “update is really delete + insert”-paradigm can be a problem even on fairly small use cases (i.e. tens of millions of rows). It can make some simple operations rather painful.
-
Lack of query hints is pretty annoying, especially for smaller users who don’t have a dedicated DBA with a Ph.D. in the PostgreSQL query planner. It’s also a massive help in development; want to try a new index? Now it’s a
drop index,create index, wait, analyse`, wait some more, run some queries, discover that didn’t do what you expected, drop the index, create a different one, wait, etc. It’s very time-consuming and much of the time is spent waiting.
-
-
Nice example: “complaining” that it is hard to tune it for a million concurrent connections.
Haven’t read it to the end yet, almost hoping to see an ending like “of course I’m happy to have a free DB that gets me in trouble for a million concurrent connections instead of breaking my bank at 1000 connections or when somebody touches advanced debugging like Oracle or ”
-
-
I agree with Simon that LLMs are akin to the introduction of mainframes in the 50s on clerical work (think of Hidden Figures).
I can see two different things happening. First, just like a calculator, LLM will be able to enhance the work of a skilled person. Just like a calculator/mainframe/computer helped with streamlining engineering work. Moreover, I am excited to see what new avenues these models could open up in the future.
On the other hand, the introduction of mainframes has made certain positions to be completely redundant (like human computers). It will be interesting to see how LLMs will impact positions that we believed were safe from automation, like copywriters or other creative writing work. Also, it will be interesting to see how it will change/impact English Language Arts instruction. Just think about your math teacher telling you that you had to learn multiplication tables because you were not supposed to carry a calculator with you all the time. The idea of the 5-paragraph essay is probably out of the window with LLMs, as most of these models can write an essay at the college level.
The things that make me excited about LLMs are the projects that integrate them with other services. LLMs make fantastic UIs for low-stakes interactions. I remember when the semantic web hype was at its peak, the vision was that companies would provide web services to interact with their models and anyone would be able to build UIs that connect to them and, especially, write tools that query dozens of data source and merge the responses. Some of the things I’ve seen where LLMs translate natural-language queries into JSON API requests make it seem that we might be close to actually providing this kind of front end.
The hallucination problem means that I wouldn’t want them to be responsible for every step (sure, go and plan a trip for me querying a load of different airline and hotel company data sources, but please met me check that you actually got the dates right), but there’s a lot of potential here.
It’s probably going to end up being used a lot for programming language docs.
I know how to get through https://www.python.org/doc/ by this point
But if I pick an unfamiliar language like https://ruby-doc.org/ , I would probably prefer to query it in natural language
Though I also hesitate because of the hallucination issue. I know teams have been working on attribution
If they can solve that problem it would be great
My hunch is though that it breaks some properties of the model, and blows up the model sizes
I think that hallucination/confabulation is inherent in the framing of the tool. Decoding notes from a few days ago, I think that there are formal reasons why a model must predict a superposition of possible logical worlds, and there will always be possible logical worlds which are inconsistent with our empirical world.
Briefly, think of text-generation pipelines as mapping from syntax to semantics (embedding text in a vector space), performing semantic transformations (predicting), and then mapping from semantics to syntax (beam-searching decoded tokens). Because syntax and semantics are adjoint, with syntax left of semantics, our primary and final mappings are multi-valued; syntax maps to an entire subspace of possible semantics, and semantics maps to a superposition of possible text completions.
I think this is precisely the kind of thing that would benefit from proper integration with a query API. If the LLM is able to do a rough mapping to some plausible API names and then query the docs to find if they’re real and refine its response based on that then it doesn’t matter if they hallucinate, the canonical data source is the thing that fixes them. If the LLM can then generate examples, then feed them into a type checker for the language and retry until they pass, then it could be great for exploring APIs.
i am an expert on ruby and gave it a small prompt and the answer was something akin to a highschooler bullshitting their way through a book report instead of a cogent answer to the question. it will probably get better but right now it’s not there for that case. i read the docs for the item and the answer was clear.
Yeah for topics I know well, I get the same thing. Sometimes it’s pretty good, but pretty often it either
But I was thinking about the case where you train it on Ruby docs specifically. I think the hallucinations become much rarer then, which is mentioned in the OP’s blog post.
There is apparently a way to fine tune the model based on a particular corpus.
Either way, I still think people will end up using it for programming language docs. I’ve tried it on stuff I don’t know as well, and then I google the answer afterward, and it saves me time
I think this is also possibly because Google is pretty bad now. But even though there’s a lot of annoying hype, and huge flaws, it seems inevitable.
The good thing is that there will be a lot of open source models. People are working on that and they will train it on PL docs, I’m sure