1. 1
    1. 1

      I welcome insights and perspectives from individuals whose professional responsibilities primarily involve the creation and management of internal tools. Your expertise and experience in this area would be of great value and highly appreciated.

      1. 1

        Strictly ‘cause you asked for it - not a judgment or endorsement on this project:

        I hail from the mirror-verse. My $WORK responsibilities involve the creation and management of internal tools; however, the one-two punch of the “javascript” tag and npx command to bootstrap the project is a death-knell (to mix metaphors) in my world.

        In the mirror-verse, if folks need database access, they get a locked down database account. They want a form to access a database - they get to build it (and maintain it) themselves - perhaps using this project!

        1. 3

          Pardon my ignorance, what is mirror-verse, is it the game? So your problem is it being JS? Would it help if you knew that the actual underlying code is Rust but it’s compiled to wasm and packaged as a JS/TS lib?

          1. 2

            I can’t speak for @crunxxi , but I also feel like I come from the mirror-verse. What I mean by that is I am “the mirror” (the opposite) of the audience this tool tries to target. I maintain data and give people access to build on it.

            For me the issues aren’t how quickly we can build the tool. The issues are:

            1. Who owns the data that this relies on.
            2. How do I give access in a minimal way that isn’t a risk. I need to worry about security, privacy, and someone messing with the data in ways that can break other things.
            3. Can we give that access in a consistent way so when others need it we aren’t just re-inventing the wheel. (Credential handling, punching holes in VPNs, defining permissions, etc…)
            4. How do we track who is using this and what they are doing? If we need to change something we’ve also now broken this random tool and need to tell whoever owns it. Maybe even (sigh) wait on them to update first.
            5. How do we track the amount of load it generates? If the system is having issues we need to be able to reach out.
            6. Do they UNDERSTAND the data? subZero will introspect columns, but the users won’t know the caveats of what is in some field somewhere. Names can be very misleading. I usually will need to give them documentation or onboard them or at least be able to field questions. If they can update that data without talking to me I’m going to be terrified.

            Building the tool is NOT the hard part. It’s all the junk around it when you don’t own the database. If subZero does everything through introspection then it’s going to constantly bump up against fine-grained access issues if we don’t give it full access.

            To be clear: this does not invalidate the product. It actually looks really slick. I bet alot of people will find it useful. I’ve personally been at startups where it’d be nice. But it will be most useful to small teams where everything is open and can be grabbed without permission. Places where us “mirror-verse” people haven’t been hired yet.

            1. 1

              Thank you for the “mirror” explanation and thank you very much for the detailed explanation of your pain points, this is very valuable to me.

              I maintain data and give people access to build on it.

              This feels like a my failure on the messaging part since what you describe as your job should fit the customer profile. What if i describe (the core of) subzero as “extensible PostgREST with more features and support for other databases besides PostgreSQL”? Would you say that you are not on the other side of the mirror now?

              1. Who owns the data that this relies on.

              subzero does not need to connect to the database with admin privileges you can use whatever locked down credentials you are given (that is if i am understanding it correctly as in you don’t control the db, you just have some limited access to it)

              1. How do I give access in a minimal way that isn’t a risk. I need to worry about security, privacy, and someone messing with the data in ways that can break other things.

              The way it works is you can either rely on the permissions set in the db for various roles (combination of grants + RLS) or you can set permissions in the same style but as a configuration. In either case, each user/role starts with no access to anything and you define to what columns/rows they get access to.

              1. Can we give that access in a consistent way so when others need it we aren’t just re-inventing the wheel.

              This is probably the best part, The REST api shape is the one from PostgREST (so there are lots of SDKs/url-builders for other languages) but the underlying db can be anything that is using SQL (at this moment it’s PG/MySQL/Sqlite/Clickhouse)

              1. How do we track who is using this and what they are doing? If we need to change something we’ve also now broken this random tool and need to tell whoever owns it. Maybe even (sigh) wait on them to update first.

              subZero is not a black box that you have no visibility in. At it’s core it’s a lib that you feed a http request and it gives you back a SQL query that you run against the db which returns the data the original http request asked for and you use this in the context of a JS/TS/Express server. So you can log whatever you feel you need to log when the http requests comes in and when it sends out the data.

              1. How do we track the amount of load it generates? If the system is having issues we need to be able to reach out.

              Similar as the explanation above, subzero is a lib you use inside an express server so there is nothing liming you from using something like prom-client to add a prometheus endpoint for metrics.

              1. Do they UNDERSTAND the data? subZero will introspect columns, but the users won’t know the caveats of what is in some field somewhere. Names can be very misleading. I usually will need to give them documentation or onboard them or at least be able to field questions. If they can update that data without talking to me I’m going to be terrified.

              It would be relatively easy (a feature for subzero) to generate a OpenAPI document based on the database schema (and your annotations) that can serve as an automated documentation for your users. Access to your users can be as limited as you want, this is all configurable. Also i feel like you are looking at subzero as something that gives an UI to the users, it’s not, it only generates the backend API, you build the UI on top of it.

              If subZero does everything through introspection then it’s going to constantly bump up against fine-grained access issues if we don’t give it full access.

              It seems like you think of introspection as “whatever subzero can see/introspect from the db will be available to the api users to read/modify” and it’s not at all like that. This part is quite configurable/hackable of what subzero sees (or needs to see) and what it allows the api users to do. Also, it does not need full access to the db (not even the ability to introspect).

              PS: If I have not scared you already with this wall of text and you think you can extract some value from this I would love to connect. Let me know here and i’ll reach out over email/discord