1. 24
  1.  

  2. 7

    So, other than the host framework and language, what makes this different to Linq-to-SQL, or the Entity Framework Code-first migrations? (Not to bash it, but I’m curious how it compares to other attempts at something similar in this space).

    1. 4

      My thoughts exactly. (I’ve never used C# but I’ve been envious of LINQ, which seems super clever, and of the language’s underlying secret sauce that the language lets you declare a function parameter such that the function actually gets passed the AST of the argument, sort of like a run-time macro.)

      1. 6

        Hey, one of the authors of this work here. Entity Framework is indeed one of our inspirations. To me, they key difference is that we aim to retain TypeScript syntax and semantics, whereas LINQ (at least with C#) has this language extensions as well. Also, because ChiselStrike is a runtime with integrated data layer, we are looking for ways to feed dynamic runtime feedback with a query profiler, for example, for stuff like index selection. Basically, we are trying hard to let developers express themselves in TypeScript, and do the heavy-lifting at the runtime layer.

        Hope this answers the question!

        1. 3

          Just wanted to say: love the application of compiler tech to a common pain point in the dev process.

          1. 3

            I think it does, yes. I’ll be interested to see how it grows over time, and what sort of effort it’ll take to write effective ChiselStrike to Data Source adapters.

            The other part of the reason I ask about this is because Entity Framework is mostly an ORM, as I understand it.

            And when you’re expressing database logic, it can be super easy to trip up on the difference between code that is run in the DB, and code that isn’t run in the DB. For me, when I was first learning LINQ, I did a ToList().Count(), rather than just a count, for reasons I don’t recall, and that resulted in pulling 70K rows into the memory of the web server from the database server, and counting them there. It was a rookie mistake, and I was a rookie at the time, but I’m curious if there will be any sort of typing barrier to help catch things like that, like how .NET has IQueryable vs IEnumerable.

      2. 3

        Just a reminder that the DB that did this best was (is) RethinkDB. Such a shame that the company failed, but it’s still available as an OSS project.

        https://rethinkdb.com/

        1. 4

          It’s a bit different? Looks like RethinkDB is a database with a query language that isn’t just “use JS primitives like map and filter and we’ll figure the search out”, which appears to be what ChilselStrike is going for.

          Here’s a RethinkDB query, from their docs

          r.table('authors').filter(r.row('posts').count().gt(2))
          

          That looks like a pretty standard ORM pattern - you probably don’t need RethinkDB to write queries that look like that. I guess the ChiselStrike version would be more like

          await Authors.findAll().filter(author => author.posts.count > 2)
          

          And their magic is that the bits of the query that can be expressed as efficient SQL are, and the rest is done in JS, and as a developer you don’t have to think about it.

          1. 1

            I had kind of gotten the impression that it was defunct, I’m surprised to see a release in April.

            I had a lot of hope for it when it came out but when I tried using it, it didn’t seem all that performant.

          2. 2

            Automatic query projection: The compiler currently only inspects the lambda passed to the filter function. However, if we inspect whole endpoints, we can — via escape analysis — determine what properties of an entity are actually used, and perform automatic projection. That is, the runtime can simply not SELECT unused columns in the underlying SQL query.

            Interesting idea but not necessarily viable to do automatically since you can’t be sure clients aren’t using some undocumented/untyped fields in a response.

            Overall good article though. This is the first time I’m starting to understand what chiselstrike is.

            1. 1

              Right, so the idea is that if we can prove in the endpoint handler that an object never escapes, we can project the query automatically. For example, imagine the following (pseudo) code:

              export default async function (req: Request) {
                  const users = User.findAll();
                  return users.map(user => user.username);
              }
              

              we know that only the username property escapes and, therefore, can project the query just for that column in the database.

              1. 2

                Thats exactly what I’d be worried about. Why would you only return the one column? Nowhere in there did I indicate I wanted only one column returned or that I expect the clients to only use the username field.

                1. 1

                  Sorry for not being clear, in that example endpoint, I specifically wrote code to return a response with a list of usernames (that map() call there does it).

                  IOW, what that endpoint would return is something like:

                  [
                    "penberg",
                    "eatonphil"
                  ]
                  

                  That is, in the example, the logic of the endpoint is to return only some specific properties, not whole objects. Clients cannot, therefore, expect any other properties to be there because it’s not in the contract of that endpoint.

                  If you want an endpoint with more stuff, then you’d return a list of user objects:

                  export default async function (req: Request) {
                      const users = User.findAll();
                      return users;
                  }
                  

                  and now no projection can happen, obviously, and you will get all the properties that User has.

                  (And if this explanation still wasn’t clear, happy to continue discussion at our Discord, for example.)

                  1. 2

                    Oh I’m sorry I missed the map. I was thinking it was a find/filter. Yes that makes sense.

            2. 1

              How does RCT3 compile its paths?

              1. 1

                How does ChiselStrike handle migrations? Is that a separate problem for a developer to solve? If so then it seems like you can’t really escape knowing about the underlying database. If migrations are handled by ChiselStrike I’d be really interested in how it works - there’s a hint in the blog post about being able to figure out which indexes are needed based on the queries it sees, which implies that it might perform DDL.

                I think it’s a really interesting idea. I suppose one destination is that ChiselStrike becomes the data store as well.

                It’s also interesting to compare with EdgeDB, too, who I think are coming at things from a different angle. Their approach is more like “let’s make a better data model and search language”. I quite like what they’re doing, though adopting a whole new database [0] is possibly more of a leap than using ChiselStrike over a known and proven database.

                [0] kind of, the boundaries between EdgeDB and underlying Postgres are blurry to me, probably due to my lack of understanding

                1. 2

                  ChiselStrike handles a large variety of migrations itself. You can add fields, remove fields, change defaults, etc. Here’s one example of that being done and deployed instantly in the ChiselStrike platform: https://www.youtube.com/watch?v=bxI7VcY9_gg

                  We’re looking into supporting other things with potential side effects too, like type changes.

                2. 1

                  It’s interesting to note that it’s also possible to implement a similar but more limited kind of compilation entirely at runtime using Function.prototype.toString(). For example - https://github.com/jackriccomini/arrowquery.js