1. 2

    Folks building new software (and not using hosted auth services like auth0 or Google accounts): are you still building your own auth system from scratch or are you using an existing open source system like keycloak or kanidm?

    1. 9

      As the main back-end guy at my company, I decided to use Keycloak for our system’s authentication. It has pluses and minuses.

      The minuses: Up-front development and setup cost was MUCH higher than a simple username/hashed-password table. I spent a fair bit of time fussing with Java OAuth libraries and Spring Security before I got it working the way I wanted. Stuff like “let users manage their own API keys for non-interactive clients” turns into a minor ordeal involving multiple OAuth tokens and fake Keycloak user identities. Customizing the registration flow is a hassle compared to implementing it as a couple extra pages in our main web app. The admin UI is a little clunky (though, it must be said, still far nicer than any of the other auth packages I evaluated). Its role-based access control setup is not flexible enough for our application and I ended up building my own authorization layer.

      On the plus side, adding a working “log in with Google” button to our app took me all of 5 minutes. Adding “log in with Apple” took longer but was still pretty quick (and most of the blame for it taking longer is on Apple, not on Keycloak). When we wanted to set up a Grafana instance and let employees log in with the same credentials they use for our web app, I just configured Grafana to use our Keycloak server as its OAuth provider, configured Keycloak to only allow access to users with our “employee” role, and it was done. Other internal tools sit behind an oauth2-proxy instance that uses Keycloak for login, and the tools don’t need to know anything about where the user data lives. My expectation is that we’ll be adding other such services over time, some end-user-facing and some not.

      On balance, I think it was the right choice for us, and the advantages will grow over time. But the minuses are significant enough that I can’t say that with absolute confidence; I could envision an alternate reality where it took me less time to do all the “easy in Keycloak” stuff from scratch than it took to mess with Keycloak.

      1. 1

        Thanks for the details!

        Up-front development and setup cost was MUCH higher than a simple username/hashed-password table.

        I think this is always the case with every separate auth system to be fair.

        The admin UI is a little clunky (though, it must be said, still far nicer than any of the other auth packages I evaluated).

        Since you mentioned you evaluated others… any info to share on specifically which other auth systems you ended up not going with?

        1. 2

          I think this is always the case with every separate auth system to be fair.

          Definitely, and going into it I expected it to take longer than a simpler system. I just didn’t appreciate how much more work it was going to be. There were lots of little fussy details that weren’t apparent when I did my initial proof-of-concept implementation as part of the evaluation process.

          Others I seriously evaluated were CAS (which seemed capable but just getting it up and running at all was a nigh-impenetrable configuration nightmare and it didn’t look like it’d get any easier afterwards), OpenAM (seemed okay-ish, but it seems like they started off with a commercial package, half-converted it to a community project, then got bored and released their work in progress, which didn’t give me lots of confidence about its long-term viability), and WSO2 (my second choice after Keycloak, but quite resource-hungry and forces you to muck around with XML to configure permissions).

          Those aren’t the only auth packages out there by a long shot, but they were the only ones I found that met our specific product requirements. Definitely worth figuring out what you really need out of such a system before looking for candidates.

      2. 4

        Building from scratch every time. But I do reuse code from project to project so it’s not entirely “from scratch” as that wouldn’t make any sense. Much auth code can be reused as it’s the same basic functionality with only minor things that change from project to project.

        1. 2

          the second

          1. 1

            Which one are you using?

            1. 2

              Keycloak. Sorry for the dry reply back there.

              Lead the IdP/IAM implementation (customer facing both B2B and B2C type of UX) in my last gig and used Keycloak as its core. Has a fairly impressive out-of-the-box completeness and extensibility is achievable albeit awkward IMO.

              In my current gig I found a Keycloak already in place. Honestly I would rather try something a bit less bulky and have been looking to Ory but haven’t done anything noteworthy yet.

              My main issues with Keycloak have been mostly around extending it. Has that old-school enterprisy Java coding experience going, as it runs on JBoss and does a poor job in hiding that. Clustering on K8S was more hassle than it should too. I think they’re “rewriting” it under Keycloak X, which is both a good and a bad thing. Good because it is targeting a more lightweight “cloud-native” runtime but breakage is sure to happen. Gave me zero issues in production for 3 years with low traffic (dozens TPS tops) but very high criticality (all user and all “microservices” used it for auth)

              I understand it depends on context but rolling out a “user/pass” form like it’s the 00’s really rubs me the wrong way. OIDC comes with a lot of value and I find it frustrating when I use something that doesn’t support auth federation: thinking on self hostable open source stuff but also commercial products

              It’s a lot of work to implement something like Keycloak from the ground up… even if you’re using some framework for it, registration/login/recovery/sso/browser support/email/mfa/“social login”/… in a profissional, product development setting, I would default against that.

        1. 31

          I disagree with this article. Selecting the right programming language is all about understanding the problem at hand and selecting the best tool for the job. One of the requirements is to understand the long term effects of your choice and plan ahead. If you later need to change the entire structure because you decided to use an ill-suited programming language/platform, you’re in for at world of hurt when the entire thing has to be re-done in something else. It’s not premature optimization.

          1. 14

            The best way to understand the problem at hand is to try solving it. Selecting a language that is good enough for most cases is a great way to go ahead and start finding what problems are hard in that problem, and which are not. This doesn’t apply to every problem, since some are well studied and have many resources for you to look at, but hopefully, in those cases you would be using solutions that have already been made.

            If you choose a language believing that performance will be a major problem, and the real problem ends up being maintainability, then you have made a premature optimization, that some may be unwilling to rectify by moving to another language due to a preconception that performance is a problem, as them “solving” it with a language choice will have “solved it” and that’s why they haven’t seen it. Solutions to problems that don’t exist in a first place are hard to get rid of, because people believe that they haven’t seen the problem because of the solution.

            Also, worth noting that a majority of software (that people can choose a language for) written nowadays (web servers) are not particularly performance sensitive as long as you can throw enough money at it, and as the article said, you’d often need additional manpower to create software in a more performant language, and the crossover point for that is quite far from what most companies experience. You have to know that the problem that you are solving cannot be scaled horizontally to justify the extra cost of developing it in a performant language, and that, once again, requires knowledge in the problem, which you don’t necessarily have.

          1. 27

            A reminder as to why older geeks, like myself, have precisely zero trust towards Microsoft when it comes to open source, or a customer-first mentality in general:


            More recently, they’ve shown their true colours with the Minecraft Education Edition, that’s only available to students at schools affiliated with Microsoft. I’ve tried (repeatedly, at the behest of my literally teary-eyed children) to pay money for Minecraft EE, only to be repeatedly rebuffed.

            The answer I was given is that they’re afraid that the low price of EE licensing might cannibalize their regular Minecraft sales. So they’ll only offer it for sale to students of schools where they’re already making up the difference on bulk licensing.

            Perhaps I’m wrong; I certainly hope I am. But I eye things like Microsoft’s acquisition of Github, and their embrace (to be followed by extending and extinguishing?) of Linux in Windows, with great skepticism. They certainly weren’t to be trusted in the 90s and early 2000s, and their treatment of Minecraft users suggests nothing has changed.

            1. 5

              You’re not alone! I am very afraid that the younger generations fail to study history and understand that Microsoft’s so-called “friendly” attitude towards the open source community is nothing more than a new business strategy.

              Things are actually much worth today than in the past. In the past it was clear and obvious that Microsoft was very hostile towards all open source, now they hide it, which makes it worse.

              1. 2

                I agree. I’m more upset that people aren’t upset about this. Microsoft has never been the good guy. They’ve open sourced some stuff here or there, but they still track you just as hard as Google or Facebook in every way imaginable.

                If the Pi team wanted to offer VS Code, why not do it from an official repo, or add a repo for vscodium?

                Maybe it’s just because I lived through the Windows 95 era and all the garbage Gates and his company tried to pull to completely crush Linux. But nothing has really changed. Sure Windows 10 doesn’t blue screen of death once every 2 ~ 3 days, but instead you get ads on everything from the start menu to lockscreen, endless amounts of telemetry, Cortana … why are so many people so quick to give Microsoft a free pass? This distribution literally has a Microsoft repo in it. Depending on the repo priority, it could offer Microsoft versions of other packages on your system.

                Also, have people just forgotten about youtube-dl and github? I don’t understand all the Microsoft apologists. Keep Linux MS free. It’s not a hard ask. It’s just common sense.

                1. 3

                  Keep Linux MS free. It’s not a hard ask.

                  It is a hard ask, because that would break Freedom Zero.

                  1. 2

                    I don’t think that @djsumdog’s comment was a legal request, but a “social” one.

              1. 3

                If only 10% of the work went into fixing the unbelievable poor state of the PHP documentation.

                1. 1

                  This isn’t working for me with local host names only FQDM. With a local host name, clicking the “HTTP” button to continue to the http version does nothing.

                  Also, I think it would be very beneficial with a whitelist, like HTTPS everywhere has.

                  1. 8

                    Using plain PHP templates is a bad idea because

                    • it relies on the fact that PHP treats accesses to undefined variables as something relatively normal
                    • it doesn’t do any kind of string quoting/escaping by default and there’s no way to add default processing. Yes, you could be using htmlspecialchars everywhere, but forget it once and you have an XSS at your hand. Proper template engines escape by default. Forget to mark something as HTML and you have visible markup on the page which is way better than XSS.
                    • PHP templates allow unmitigated access to global state and due to the they how PHP keeps request state as mutable global dictionaries, this means that PHP templates can even mutate request state at will.
                    • include() puts the template file into the current scope, so a template gets access to all of the variables in scope inside of the rendering function (which, as the article explained, also included $this.
                    • Because the templates are plain PHP, there’s nothing a template can’t do, including accessing external resources, reading the file system, etc. Yes. you shouldn’t put business logic in your templates, but it happens and then you’re screwed a few years down the line.

                    People have invented template engines for reasons. Most of them were and still are valid reasons.

                    1. 2

                      Author here. You are mostly right, but in most cases I’d consider those a feature and not necessarily a problem. Those features help keeping things simple. Of course you can abuse those features, but you shouldn’t.

                      As for the escaping to avoid XSS: you are very right. This is the weakest point of this approach to doing templates and requires a certain amount of developer discipline when designing the templates…

                      1. 2

                        I’d consider those a feature and not necessarily a problem

                        so did I nearly 20 years ago and now I wish I hadn’t.

                        Of course you can abuse those features, but you shouldn’t.

                        people always think that and poof 10 years (if you’re lucky. probably sooner) later they drown in technical dept and the rewriting-effort starts to get going.

                        requires a certain amount of developer discipline when designing the templates

                        discipline doesn’t work. Never has. You only need to forget a single htmlspecialchars() to get the equivalent security of having none. A solution that requires the developer to take care of all instances of something when all an attacker needs is a single instance can’t scale.

                        Simplicity is nice, but not at this price.

                        1. 2

                          I appreciate where you are coming from. I’ve worked in “enterprise” software development where my approach would make people lose their shit. I’m trying to get that way of working out of my system and build neat simple solutions that are not perfect, but are worth considering, probably for many (smaller) projects. If nothing else, it introduces you to some nice long forgotten PHP features :-)

                        2. 1

                          It would be possible to write a template validator that inspects template PHP files and has a whitelist of acceptable PHP features. Things like <?=$var?> could be flagged, and the Tpl class could have an extra function so that you can do <?=$this->unescaped($var)?> if you really mean it. You only need to run the validator when you ship the template, in the same way you already run the rest of your code through Phan and/or Psalm.

                          About $this being in the scope, I do think that’s a feature, but for shorter templates, would it be possible to make in-scope functions, so that you can write <?=e($var)?> instead of <?=$this->e($var)?>?

                        3. 2

                          This is a bit contradictory as all PHP template engines suffer from these problems.

                          1. 2

                            That doesn’t mean they’re good. There are template engines that can give you at least default escaping. I’ve used to maintain PHPTAL that has context-sensitive escaping and even ensures HTML is well-formed.

                            Some template engines claim to be “universal” or “format-independent”, so that you can use the same syntax for HTML, e-mails, and even SQL if you want. But in practice it means they’re not good for anything: you get XSS, messed up e-mail encodings, and SQL injections.

                            “Just don’t write bugs” approach doesn’t work, so you really need a template engine where security vulnerability isn’t the default behavior.

                            1. 2

                              There is no default escaping. There is only code. If you rely on the template engine you’re relying on someone else to do the escaping for you.

                              All output is done using ‘echo’, ‘print’, etc. Make a custom “always escaping” function or method, and you have just as good “default escaping” as any template engine can provide.

                              1. 2

                                “There is only code” entirely misses the point of secure defaults. If you have to remember to use an escaping function, you will eventually forget it, and create an XSS vuln.

                                PHP templates == XSS, and this is a people problem, not a code problem.

                                PHP makes it particularly messy:

                                • Humans are bad at noticing absence of things, so a code review is more likely to miss <?=$foo than it would ${foo|unsafe} (both equally risky, but one looks more innocent).

                                • Escaping is technically required pretty much everywhere in HTML for syntax correctness, but there’s a commonly held belief that escaping is only for “untrusted” data or strings that “contain unsafe characters”. Or that strings can be “sanitized” by stripping tags. This creates disagreements about what even has to be escaped.

                                1. 2

                                  You’re missing the point. The people who make the engine also have to remember to escape and what not, they will eventually forget it too. It doesn’t matter.

                                  Besides, when you add a huge monster of a template engine the chance of errors, mistakes, and security issues increase exponential.

                                  Adding some engine doesn’t automatically solve these problems. Good coding solves these problems.

                                  1. 1

                                    There are far less boundaries/inputs in the template engine than there are in your own code, combined with the amortization of effort.

                                    “Good coding” doesn’t solve the massive safety issues we have with programming the same way that “good driving” makes seatbelts redundant.

                          2. 1

                            I think it’s fine for smaller projects; e.g. the type where everything is just in one or a few pages. Not having an external dependency is a pretty good advantage in those cases.

                          1. 4

                            This is the best approach no matter the size of the project. With PHP there is no need to add yet another layer of abstraction. Make sure you use a couple of inspection/escaping/validation functions or methods to eliminate any risk of forgetting to escape output, etc.

                            1. 2

                              Never filter input, validate input! And then escape output.

                              1. 3

                                Because many corporations need extensive control Mozilla has created a something called “policy support” which can be implemented using a JSON file called policies.json. This file is a cross-platform compatible file that makes it the preferred method for enterprise environments to control Firefox in different environments. By using the policies.json file you can control a great amount of how Firefox works, including the DNS over HTTPS feature.

                                Create the file before you start using Firefox in order to avoid initial data going through Cloudflare.

                                Find out where Firefox is installed. On Arch Linux Firefox gets installed in /usr/lib/firefox/. On FreeBSD it gets installed in /usr/local/lib/firefox/. If a subdirectory called distribution doesn’t exist you need to manually create it. Then create the policies.json file in that directory.

                                On the README for the policies templates you can find a list of options to control.

                                I have created a policies.json that looks like this:

                                  "policies": {
                                    "DisableAppUpdate": true,
                                    "DisableFirefoxAccounts": true,
                                    "DisableTelemetry": true,
                                    "DNSOverHTTPS": {
                                      "Enabled": false,
                                      "Locked": true        
                                    "DontCheckDefaultBrowser": true,
                                    "NetworkPrediction": false,
                                    "PromptForDownloadLocation": true,
                                    "SearchEngines": {
                                      "PreventInstalls": true
                                    "SearchSuggestEnabled": false

                                You can view your settings by typing about:policies in the address bar.

                                If you want to block Cloudflare and other known companies that supply DoH at good list with both domain names (for DNS blocking) and IP addresses (for firewall blocking) is available at: https://github.com/oneoffdallas/dohservers

                                1. 2

                                  An easier way to do this, if you’re already running your own custom DNS resolver, is to block the canary domain

                                  I’m figuring that we’re going to have ISPs blocking the canary domain, and the arms race will continue onward, but at least for now that’s how it is.

                                  1. 2

                                    I forgot to mention the canary domain.

                                    I’m blocking the canary domain on my DNS server already, but since we will most likely begin to see fallback options, like systemd did with systemd-resolved, I think it’s good to have a list that we can work on and add domains and IP addresses as they go public.