Interesting, have never heard of recutils before now. I’m a big fan of plain text, and I assume on a modern machine something like this is plenty fast for most uses.
I’m curious why you wrote these bindings. What do you use recutils for?
As for why I wrote these bindings…well, I know I like Go and I know I like recutils. I wrote these bindings in anticipation of times when I’ll want to write a Go program, want to use recfiles, and not want to worry about coding up the database layer. I don’t have a use case for this library right this moment.
As for what I use recutils for…well, I use it when I’ve got shell scripts that need to maintain state/data between runs AND I don’t want users to have to set up a database before getting to use my scripts.
Not having used recutils myself, I think the biggest differentiator is that recfiles are plain text and don’t need a binary “interpreter”. There’s also the command-line aspect, you have specific commands and arguments for stuff like “select titles where author matches foo” without having to write SQL.
Here’s an old Lobste.rs discussion I stumbled on yesterday.
sqlite has a pretty small set of builtin functions. If you want to do some string manipulation on cells, like say URL escaping, then I think you’re basically stuck with writing C extensions and rebuilding sqlite.
Not sure if recutils does better, but you can usually hack something with shell.
You can also use both sqilte and recutils in a shell script, so it’s not either/or.
Putting the constant part of the comparison before the variable like this is a defense against accidental assignment, if you fat-finger the operator. But that only matters in languages where it’s possible to perform assignment in comparison expressions, like C. It’s not possible in Go, so this reads awkwardly.
if err != nil
This form is preferable, and totally safe.
func whatever() (ReturnType, error) {
var retval ReturnType
var err error
// code
Predeclaring variables in this way is also a bit of a cargo cult from languages where it is necessary, but in Go it isn’t necessary. Keeping declarations close to usage generally aids in comprehension and maintainability, and Go’s := operator has affordances specifically to help with this stuff. Better to avoid predeclarations.
Your Save function will leave the file on disk in an incomplete state if any error is encountered. Better to serialize to memory, and write to disk in a single motion, on complete success.
Conventionally, if err != nil, then all other return values are invalid. In practice, that means you need to check if err != nil first. So this code should read as e.g.
Ya, the nil != err is definitely a carryover from coding in C
For functions, I usually do try to keep declarations close to usage except for function return values. Personally, I’ve always found it more readable for those to be predeclared at the top of the function
Your Save function will leave the file on disk in an incomplete state if any error is encountered
Ah, good catch. I’ll make a note of that case to address it
I usually do try to keep declarations close to usage except for function return values. Personally, I’ve always found it more readable for those to be predeclared at the top of the function
This creates some dangerous situations. Take your Load function, for example.
func Load(path string) (Database, error) {
var database Database
var err error
parser, err := newParser(path)
if nil != err {
return database, err
}
If you encounter an error, convention dictates that all other return values are (usually) returned as their zero values. That would mean return nil, err here. This defends against accidentally returning a valid database value with a non-nil err value. The code as written is vulnerable to this mistake. You want
parser, err := newParser(path)
if err != nil {
return nil, fmt.Errorf("parser for path: %w", err)
}
Later,
...
return database, err
}
At this point you can reliably assert that err is nil. Therefore, you want to return database, nil here, to avoid accidentally returning a non-nil err value with a valid database value. You want
return database, nil
Predeclared return values generally introduce more cost than benefit.
I wonder if it’s worth doing a side-by-side with other established projects like textql or q. Obviously this particular post is specifically about the Go bindings, but there could be broader potential here.
Almost none. Alphabet for the attribute names is very limited ([a-zA-Z%][a-zA-Z0-9_]*) and attribute values end with the line-end. If the line-end is part of the value, the value just continues on another line with “+” prefix. This works quite well. The line-ends can be also escaped by \ and then they are not part of the value. This is also useful, but it is a blurry part of the specification. How should be an equivalent of the CSV
You know…I hadn’t thought of that. Am I incorrect in calling it bindings? I’m open to the idea that bindings has a clear definition that this doesn’t fall into. Would it be more accurate, instead, to say a “Go library for GNU Recutils”?
Interesting, have never heard of recutils before now. I’m a big fan of plain text, and I assume on a modern machine something like this is plenty fast for most uses.
I’m curious why you wrote these bindings. What do you use recutils for?
In my experience, I agree. It runs plenty fast.
As for why I wrote these bindings…well, I know I like Go and I know I like
recutils. I wrote these bindings in anticipation of times when I’ll want to write a Go program, want to userecfiles, and not want to worry about coding up the database layer. I don’t have a use case for this library right this moment.As for what I use
recutilsfor…well, I use it when I’ve got shell scripts that need to maintain state/data between runs AND I don’t want users to have to set up a database before getting to use my scripts.Just being curious, but what is the use case for recutils when you have Sqlite?
Not having used recutils myself, I think the biggest differentiator is that recfiles are plain text and don’t need a binary “interpreter”. There’s also the command-line aspect, you have specific commands and arguments for stuff like “select titles where author matches foo” without having to write SQL.
Here’s an old Lobste.rs discussion I stumbled on yesterday.
https://lobste.rs/s/b1k1hi/gnu_recutils
It discusses this article that has more examples
https://labs.tomasino.org/gnu-recutils/
Yep, that’s a pretty significant chunk of the value proposition for
recutilssqlite has a pretty small set of builtin functions. If you want to do some string manipulation on cells, like say URL escaping, then I think you’re basically stuck with writing C extensions and rebuilding sqlite.
Not sure if recutils does better, but you can usually hack something with shell.
You can also use both sqilte and recutils in a shell script, so it’s not either/or.
Some notes on style.
Putting the constant part of the comparison before the variable like this is a defense against accidental assignment, if you fat-finger the operator. But that only matters in languages where it’s possible to perform assignment in comparison expressions, like C. It’s not possible in Go, so this reads awkwardly.
This form is preferable, and totally safe.
Predeclaring variables in this way is also a bit of a cargo cult from languages where it is necessary, but in Go it isn’t necessary. Keeping declarations close to usage generally aids in comprehension and maintainability, and Go’s
:=operator has affordances specifically to help with this stuff. Better to avoid predeclarations.Your Save function will leave the file on disk in an incomplete state if any error is encountered. Better to serialize to memory, and write to disk in a single motion, on complete success.
Also, some issues in error management, e.g. in parser.go
Conventionally, if err != nil, then all other return values are invalid. In practice, that means you need to check if err != nil first. So this code should read as e.g.
Ya, the
nil != erris definitely a carryover from coding in CFor functions, I usually do try to keep declarations close to usage except for function return values. Personally, I’ve always found it more readable for those to be predeclared at the top of the function
Ah, good catch. I’ll make a note of that case to address it
This creates some dangerous situations. Take your Load function, for example.
If you encounter an error, convention dictates that all other return values are (usually) returned as their zero values. That would mean
return nil, errhere. This defends against accidentally returning a validdatabasevalue with a non-nilerrvalue. The code as written is vulnerable to this mistake. You wantLater,
At this point you can reliably assert that
erris nil. Therefore, you want toreturn database, nilhere, to avoid accidentally returning a non-nilerrvalue with a validdatabasevalue. You wantPredeclared return values generally introduce more cost than benefit.
You can put the name of the return variable in the function signature.
You should add a
go.modfile if you want people to use this.Also a license!
This is revolutionary!
Disclaimer: @jcstryker is my friend being a hypeman, not me on an alt account lol – thanks, bro
I wonder if it’s worth doing a side-by-side with other established projects like textql or q. Obviously this particular post is specifically about the Go bindings, but there could be broader potential here.
I read recfile documeantation and did not found which character/should be escaped
Do you mean which characters in the actual database need to be escaped, or which characters in a query need to be escaped?
originally I wanted to know the character in the database that must be escaped, but now i’m curious of the character in the query too.
Maybe the percent sign?
Almost none. Alphabet for the attribute names is very limited (
[a-zA-Z%][a-zA-Z0-9_]*) and attribute values end with the line-end. If the line-end is part of the value, the value just continues on another line with “+” prefix. This works quite well. The line-ends can be also escaped by\and then they are not part of the value. This is also useful, but it is a blurry part of the specification. How should be an equivalent of the CSVbe serialized as a Recfile?
I wrote a Recfile parser and generator for Relational pipes (Integrating Relational pipes with GNU Recutils, Reading apt (Debian package system) results) And I am currently dealing with the line-escaping part… I will ask at the mailing list if I do not find any solution. (of course, you can add a space after
\, but it changes the attribute value)Very nice, I’m partial to a bit of recutils. I was actually looking for a Go recutils library a few months back.
I have to admit though, that when I read “bindings” I thought this was CGo bindings…
You know…I hadn’t thought of that. Am I incorrect in calling it bindings? I’m open to the idea that bindings has a clear definition that this doesn’t fall into. Would it be more accurate, instead, to say a “Go library for GNU Recutils”?
I think I would say it’s “A Go implementation of GNU Recutils”
Precisely. Unless you use it through librec, this is another implementation, not bindings.
Reminds me that I should finish the Guile bindings for librec one day…