1. 29
    1. 9

      Original author here. I wrote this article because I have set up NFSv4 with Kerberos twice so far and, both times, I ended up with a functional system but… very frustrated by how difficult and fragile everything seems.

      That said, I still have a bunch of unanswered questions (all listed at the bottom of the article) and I suspect that some of you folks might have some insightful answers or corrections… hence why I’m submitting the article myself. If you do have any of those insights, please share here or there. And thanks!

      1. 6

        The fact that Fedora and FreeBSD show nobody for file ownership even when they seems to do the right thing when talking to the NFSv4 server sound like a bug either in the code or in my configuration. Which is it?

        This is presumably because the server and the client do not agree on their NFS-level “domain name”, which is used by the NFSv4 ID mapping daemon to map “user@domain” strings in the protocol messages back to local uids. On illumos systems it’s pretty easy to tell what the current NFSv4 domain is, but I recall it being more difficult on Linux. You can configure it though, obviously! Or you can create a special DNS TXT record that systems will use to discover it on their own.

        1. 3

          On Linux, this is set in idmapd.conf, which is usually /etc/idmapd.conf. If you don’t have a specific Domain set, Linux will try to work out a default one based on hostname and DNS information, which may explain some of the Fedora versus Debian differences. The manual pages say that nfsidmap -d is how you display the current domain name.

          1. 2

            Aha! Thanks. Indeed, nfs-idmapd.service wasn’t running on Fedora, and the hostname set in the client didn’t contain the domain name part. Fixed! Will have to update the text.

          2. 1

            Hm. So on FreeBSD, it looks like I’m missing running nfsuserd, although starting it doesn’t make a difference…

        2. 8

          Great post! A couple of minor notes and some answers:

          Kerberos is an authentication broker.

          Nitpick: I’d classify it as an authentication service. A broker to my mind is responsible for managing the coordination of messages between other (often distributed) components, but doesn’t itself perform processing of the messages. Kerberos in contrast is a client/server model where the Kerberos server (KDC) processes Kerberos client requests.

          Its goal is to detach authentication decisions between a client machine and a service running on a second machine, and move that responsibility to a third machine—the Kerberos Domain Controller (KDC).

          You mean the Key Distribution Center (KDC). Domain controllers are strictly a Windows concept as part of an Active Directory deployment. Kerberos itself has no concept of domain controllers.

          Kerberos claims to be an authentication system only, not an authorization system. However, the protocol I described above separates the TGT from the TGS, and this separation makes it sound like Kerberos could also implement authorization policies. Why doesn’t it do these?

          The issuance of a TGT which is used by the client to request service tickets to specific service principals has several benefits over the client directly requesting service tickets without an initial TGT:

          • The Kerberos client can discard the user’s password after obtaining the TGT. There’s no need for it to be retained in either memory or persistent storage, which has some clear security benefits. You sort of mentioned this in the article, by correctly noting that the TGT means the user doesn’t get repeatedly prompted for their password on subsequent interactions with the KDC, but a client could still handle this by caching the password in memory. The TGT makes even this (admittedly terrible idea) unnecessary.
          • The Kerberos server (KDC) can verify the authenticity of the TGT instead of the user itself. I.e. it doesn’t need to perform a far more expensive user authentication process when issuing service tickets (the vast majority of tickets). If the TGT is valid, the KDC can trust in the authenticity of its content and simply issue the requested service ticket for the user in the supplied TGT. A single KDC may be processing a very large volume of requests, so avoiding the need to authenticate users more often than necessary has material performance impact.
          • As a supplement to the last point, authenticating a user is much slower on the network as it will require multiple round-trip requests to the KDC. A service ticket request can be done in a single round-trip, literally in the case of UDP, or after the initial handshake in the case of TCP.

          These are all performance and security reasons, but the resulting design doesn’t enable authorisation decisions to be made at the Kerberos level. The information required to make that decision doesn’t typically exist in Kerberos, and even if it were, it would significantly complicate the service ticket issuance flow.

          Authorisation decisions instead tend to be made on user attributes which are stored in a directory service, usually based on a protocol like LDAP. Active Directory takes this approach where Kerberos is used for AuthN and LDAP is used for AuthZ. Technically, LDAP can be used for both, and some services use this approach for SSO, but then you lose the benefits Kerberos brings.

          Something that is desirable is to include user information in the Kerberos service ticket which can be used by the authenticating Kerberos services to make authorisation decisions. Kerberos doesn’t have a standardised facility to do this, so Microsoft extended Kerberos to include a structure called the PAC (Privilege Attribute Certificate) that includes information about the user (e.g. their group membership as retrieved from LDAP at the time the TGT was issued). Kerberos services can examine the PAC to make authorisation decisions instead of having to perform more expensive lookups to a network service to determine user authorisation if that information is not available locally.

          1. 2

            Thanks for the detailed response! I’ll try to incorporate some of this into the text when I find a moment this week.

          2. 2

            Very well written, this is very similar to my path to implementing nfsv4 with krb5p.

            One thing I would add is to use k5start instead of kinit on workstations in either .bashrc or a ~/.config/autostart job, to not only obtain a ticket but also keep it renewed. I had tried using kinit to get the ticket and then krenew to keep it active, however krenew doesn’t seem to be ticket cache aware and would always fail.

            Also worth noting that because of nfsv4 requiring locking, if I was unable to reach my nfs server my client would hang indefinitely, especially in nautilus. I chose setting retry=0 and retrans=1 in mount options to get it to fail fast, but that feels clunky.

            1. 2

              I guess I’ll keep on using sshfs

              1. 1

                i don’t know much about nfs but I am planning to get the same Synology device, what amount of setup work is required to have a robust nas system after acquiring the device? looking at this guide it seems like it’s a lot of assembly required. i thought Synology software would handle all this?

                1. 2

                  Setting the device up was trivial, and even the NFS bits could be easily configured through the UI. All of the complexity I wrote about comes from the KDC and the clients.

                  And if you stay away from NFS, it’s much easier. The device seems solid to me.

                  1. 1

                    I have a DS923+ — it is very hand-holdy and quite hard to get wrong. This post is going off the rails (good thing! but yes).

                  2. 1

                    Excellent article! I was wondering if you have evaluated how NFSv4 with krb5p encryption compares to Samba in terms of throughput, latency, and overall system resource usage. How do they perform in real-world situations relative to each other?

                    1. 1

                      I’ll include those measurements when I write the promised upcoming post comparing the DS923+ to FreeBSD :)

                      (I don’t have any numbers though yet. I tried to run bonnie++ earlier but it was infinitely slow over NFS…)