We, generally speaking, want these assistant AIs to be truthful. OpenAI tried to train their AI to be truthful. But the best way they could find to do that is to show it a bunch of training examples of true and false things and ask it to complete the text appropriately.
The AI didn’t learn to be truthful from that. It learned instead to sound vaguely plausible, which is subtly but importantly different. When this aligns with the truth, great. When it doesn’t, that’s a misalignment between what the we want and what the AI “wants”.
I posted a rather garbled explanation last night to Twitter (I had been out all day and had a few beers, and I have only a vague knowledge of law at the best of times), so I really value Simon’s far more coherent explanation!
Notably, it looks like this lawyer was actively maliciously using ChatGPT, since it’s quite hard to get to produce the kinds of output we saw (especially the full “quotes” from the fictional cases). It doesn’t seem like he was mislead by an overly friendly algorithm.
While I have no doubt GPT will never be as creative and insightful as a human, it does strike me that legal writing in particular is extremely constrained and formulaic. It does seem like a legal-oriented AI could do at least a somewhat good job of scanning the available research and generating arguments. Of course, that won’t be ChatGPT!
My limited defense of pkg is that you end up with a lot of stuff in the root of a project anyway, so for an application, it can be nice to shove things down one level. But it’s probably not necessary for most applications.
I know Simon is popularizing “a calculator for words” but I much prefer “a kaleidoscope for words”. It makes cool new patterns! But don’t try to see what’s outside like it’s a telescope.
Integrating these things with search engines is a totally irresponsible idea because they work nothing like search engines and are very bad at sticking to the facts. It’s irresponsible to make a tool that looks like a hammer but works like a mail gun.
I really wish these stories would kill off this notion that these “AI” systems understand anything. They do one thing, and one thing only: try to create a statistically plausible list of words.
If you go to chat-gpt and ask it to just repeat a single letter - say ‘A’ - as long as possible it will eventually start producing a pile of unrelated text, because eventually token gap between your request and the next word it should emit becomes too long and ceasing being involved in the “predict the next word in a sentence” game.
Why are we adding __Bound here? it isn’t unnecessary.
If you have
void foo(size_t N, int ptr[N]);
It does not need an additional annotation. That just works as you would expect. The only time you need to explicitly use these attributes is if
The attributes are needed for all the cases where that syntax does not work.
For example, you can’t make
void foo(size_t N, size_t M, int ptr[N*M])
work with the existing syntax because you break existing compilers in a way that can’t easily be opted out of via a macro.
The same happens with the gcc extension to pre-declare parameters
void foo(size_t N; int ptr[N], N)
You can’t easily make that work via macros in compilers that don’t support gcc’s extension.
But you’re also focusing on just parameters, whereas this set of attributes applies to the type, and can be placed more or less anywhere a type is used, for example, the fairly critical case of a return value
void* __sized_by(N) malloc(size_t N);
or fields in structs, where
struct S {
int * __counted_by(10) buffer;
};
is not the same as
struct S {
int buffer[10];
};
and other things just aren’t representable
struct S {
int N;
T* buffer __counted_by(N);
}
Arguably it shouldn’t be a stretch to make flexible struct arrays a real syntax, e.g.
struct ArrayThing {
int N;
Thing things[N]; // Where currently you would just have []
};
But that kind of change still wouldn’t allow
struct Matrix {
int width;
int height;
float values[width * height]; // or a out of band pointer, etc
}
This RFC is designed to be applicable to existing C codebases, and more importantly existing C ABI, so it has to be general and support the many absurd things that happen in existing code, so the bounding expression is a more or less arbitrary, and can reference members, etc.
Graphemes are the closest thing to how humans (or at least the majority of humans who are not Unicode experts) normally/“naturally”/informally think about text. So in an ideal world, the most humane user interface would state “max length n” and mean n grapheme clusters.
For some Western European scripts, code points also get kinda-sorta-close-ish to that, and are already the atomic units of Unicode, so also are an acceptable way to do things (and in fact some web standards are defined in terms of code point length and code point indexing, while others unfortunately have JavaScript baggage and are instead defined in terms of UTF-16 code units).
This is a nice idea. Just leverage BEAM and OTP to make Kubernetes obsolete you mean? In that sense, I’m continuously considering using less AWS and more OTP. I am not in love with erlang and elixir as languages, but they pay back when complicated stuff arises.
I think the static guarantee is the interesting bit (and the reason that the comparison to Rust makes sense). It’s not just allocating values on the stack, but allocating values on the stack and guaranteeing that references to those values can’t outlive the stack frame. (Speaking loosely here, since the allocation regions aren’t literally the call stack, and exclaves mean you can allocate in other regions, but you get it.)
5 (codepoints) is sensible in exactly one case: communicating offsets for string replacement/annotation/formatting in a language and unicode version and encoding agnostic way.
Here’s the strategy I used when switching from Emacs to Vim:
Look up all the letters and symbols in Vim and what they do. Categorize then and write them all down on a piece of paper (it just has to be enough detail for you to remember). Then keep the piece of paper next to you as you edit, and refer to it whenever you’re trying to do something that feels clunky with the commands you know.
I was very surprised how effective this was. After a week I was as fast at editing in Vim as Emacs (after a decade of Emacs use; this is mostly Emac’s fault), and after two weeks I was pretty fluent in Vim.
I’d finance another few seasons of Firefly. :D
This is an AI alignment problem.
We, generally speaking, want these assistant AIs to be truthful. OpenAI tried to train their AI to be truthful. But the best way they could find to do that is to show it a bunch of training examples of true and false things and ask it to complete the text appropriately.
The AI didn’t learn to be truthful from that. It learned instead to sound vaguely plausible, which is subtly but importantly different. When this aligns with the truth, great. When it doesn’t, that’s a misalignment between what the we want and what the AI “wants”.
I posted a rather garbled explanation last night to Twitter (I had been out all day and had a few beers, and I have only a vague knowledge of law at the best of times), so I really value Simon’s far more coherent explanation!
Notably, it looks like this lawyer was actively maliciously using ChatGPT, since it’s quite hard to get to produce the kinds of output we saw (especially the full “quotes” from the fictional cases). It doesn’t seem like he was mislead by an overly friendly algorithm.
While I have no doubt GPT will never be as creative and insightful as a human, it does strike me that legal writing in particular is extremely constrained and formulaic. It does seem like a legal-oriented AI could do at least a somewhat good job of scanning the available research and generating arguments. Of course, that won’t be ChatGPT!
I rather like that!
Perhaps the wording could be improved: gpt is designed to produce statistics.
Well, shoot, so is evolution and people. Well, we have usually had longer training and better heuristics :)
My limited defense of pkg is that you end up with a lot of stuff in the root of a project anyway, so for an application, it can be nice to shove things down one level. But it’s probably not necessary for most applications.
I know Simon is popularizing “a calculator for words” but I much prefer “a kaleidoscope for words”. It makes cool new patterns! But don’t try to see what’s outside like it’s a telescope.
Integrating these things with search engines is a totally irresponsible idea because they work nothing like search engines and are very bad at sticking to the facts. It’s irresponsible to make a tool that looks like a hammer but works like a mail gun.
How could one prove that any system that outputs only text (i.e. a human at a keyboard) is anything more than that?
Rants have improved my technical ability before.
They are a calculator for words only (albeit an extremely capable and useful one).
That said, it doesn’t need to be AGI to be disruptive.
I really wish these stories would kill off this notion that these “AI” systems understand anything. They do one thing, and one thing only: try to create a statistically plausible list of words.
If you go to chat-gpt and ask it to just repeat a single letter - say ‘A’ - as long as possible it will eventually start producing a pile of unrelated text, because eventually token gap between your request and the next word it should emit becomes too long and ceasing being involved in the “predict the next word in a sentence” game.
Why are we adding __Bound here? it isn’t unnecessary.
If you have
It does not need an additional annotation. That just works as you would expect. The only time you need to explicitly use these attributes is if
The attributes are needed for all the cases where that syntax does not work.
For example, you can’t make
work with the existing syntax because you break existing compilers in a way that can’t easily be opted out of via a macro.
The same happens with the gcc extension to pre-declare parameters
You can’t easily make that work via macros in compilers that don’t support gcc’s extension.
But you’re also focusing on just parameters, whereas this set of attributes applies to the type, and can be placed more or less anywhere a type is used, for example, the fairly critical case of a return value
or fields in structs, where
is not the same as
and other things just aren’t representable
struct S { int N; T* buffer __counted_by(N); }
Arguably it shouldn’t be a stretch to make flexible struct arrays a real syntax, e.g.
struct ArrayThing { int N; Thing things[N]; // Where currently you would just have [] };
But that kind of change still wouldn’t allow
struct Matrix { int width; int height; float values[width * height]; // or a out of band pointer, etc }
This RFC is designed to be applicable to existing C codebases, and more importantly existing C ABI, so it has to be general and support the many absurd things that happen in existing code, so the bounding expression is a more or less arbitrary, and can reference members, etc.
Graphemes are the closest thing to how humans (or at least the majority of humans who are not Unicode experts) normally/“naturally”/informally think about text. So in an ideal world, the most humane user interface would state “max length n” and mean n grapheme clusters.
For some Western European scripts, code points also get kinda-sorta-close-ish to that, and are already the atomic units of Unicode, so also are an acceptable way to do things (and in fact some web standards are defined in terms of code point length and code point indexing, while others unfortunately have JavaScript baggage and are instead defined in terms of UTF-16 code units).
This is a nice idea. Just leverage BEAM and OTP to make Kubernetes obsolete you mean? In that sense, I’m continuously considering using less AWS and more OTP. I am not in love with erlang and elixir as languages, but they pay back when complicated stuff arises.
I think the static guarantee is the interesting bit (and the reason that the comparison to Rust makes sense). It’s not just allocating values on the stack, but allocating values on the stack and guaranteeing that references to those values can’t outlive the stack frame. (Speaking loosely here, since the allocation regions aren’t literally the call stack, and exclaves mean you can allocate in other regions, but you get it.)
Why does input validation care about grapheme length? It seems like the most irrelevant type of length.
5(codepoints) is sensible in exactly one case: communicating offsets for string replacement/annotation/formatting in a language and unicode version and encoding agnostic way.Here’s the strategy I used when switching from Emacs to Vim:
Look up all the letters and symbols in Vim and what they do. Categorize then and write them all down on a piece of paper (it just has to be enough detail for you to remember). Then keep the piece of paper next to you as you edit, and refer to it whenever you’re trying to do something that feels clunky with the commands you know.
I was very surprised how effective this was. After a week I was as fast at editing in Vim as Emacs (after a decade of Emacs use; this is mostly Emac’s fault), and after two weeks I was pretty fluent in Vim.
Please don’t post rollups and newsletters. Specific articles with specific information are way more useful and easier to discuss.