NFSv4: Interop, ACLs & Automount

NFSv4 has been around for a long time but it still seems a bit foreign to me. The following is a quick rundown of things I recent learned related to NFSv4 from limited experience in implementing it.

Interoperability

Is it possible to setup NFSv4 along side NFSv3 on the same server, serving the same volumes? Of course. However, it might not always work exactly as expected with legacy clients.

A normal /etc/exports for NFSv3/v4 interoperability might look like so:

/export                   10.0.0.0/8(rw,no_subtree_check,fsid=0)
/export/namespace         10.0.0.0/8(rw,no_subtree_check)
/export/namespace/share1  10.0.0.0/8(rw,no_subtree_check)
/export/namespace/share2  10.0.0.0/8(rw,no_subtree_check)

With this configuration, we have the “virtual root” export (fsid=0), the namespace export (for mounting the whole namespace with one mount) and the individual “share” exports (for mounting individual shares, most likely with automount). The NFSv4 clients can perform mounts using the servername:/namepace syntax and the NFSv3 clients can mount the whole root, namepace or individual “shares” with servername:/export, servername:/export/namespace or servername:/export/namespace/share1.

All is well in the NFS world… or so it seems at first. It turns out that an older SunOS does not entirely like how this RHEL 6 NFS server is exporting the file systems:

hostname% cd /namespace
hostname% ls
share1     share2     share3     share4
hostname% pwd
/namespace
ubcpetnxi% cd share1
ubcpetnxi% pwd
/share1

Notice the final line. I was just in /namespace then I changed into /namespace/share1. Now pwd tells me the path is only /share1. I was expecting /namespace/share1. It looks to me like the SunOS NFS client is not behaving well with how the NFS server exporting the file systems and/or how the bind mounts are setup locally on the server to map the storage into the NFSv4 “virtual root”.

Please leave a comment to if you know of a different /etc/exports and/or mount configuration that would alleviate the SunOS NFS client issues noted here!

Access Control Lists

NFSv4 defines a model for Access Control Lists (ACLs) that has similarities to that of Microsoft’s NTFS. But don’t worry about interoperability: NFSv4 translates your existing “POSIX” ACLs on ext3,ext4,xfs,etc. to NFSv4 ACLs automatically.

The main gotcha with exporting a filesystem with “POSIX” ACLs with the NFSv4 server is that the normal getfacl and setfacl tools don’t seem to work on the NFS client side! Because the NFSv4 server only presents the translated NFSv4 ACLs to the clients, the nfs4-progs package must be installed and the nfs4_getfacl and nfs4_setfacl commands used instead to view and manipulate the ACLs on NFSv4 clients.

Also, the little + at the end of the rwxrwxrwx permissions listing you can see with some variant of ls -l, the symbol that normally indicates the presence of an ACL, it simply doesn’t appear on a (Linux?) NFSv4 mount where ACLs exist. Sadness.

Automount

Automount on RHEL 6 (and clones) appears to have a bug related to bind mounts. NFSv4 exports cannot (trivially?) be mounted locally on the NFSv4 server on itself with bind mounts as is possible with NFSv3 (or lower) exports. I have read that this inability is due to the “virtual root” abstraction that NFSv4 employs. Instead, automount should be performing true NFSv4 mounts when operating locally on the server… but it doesn’t do that on CentOS 6 (and in my experience RHEL 6):

See: http://bugs.centos.org/view.php?id=6101

The workaround is to specify port=2049 in the NFS mount options of the automount map in use (where 2049 is the port the NFS server is listening on). This appears to cause automount to immediately attempt an NFS mount, bypassing the (failing) attempt at a bind mount.

 

File Access Auditing w/ Linux NFS Server

What is the accepted/best solution for auditing file access with GNU/Linux NFS server infrastructure?

I recently received a request for auditing the basic file access patterns (read/write) for an NFS export on a RHEL 6 server. While researching the problem, I discovered that there is no apparent method to accomplish this task without reliable and trusted client side auditing. It wasn’t a priority and we didn’t have admin on all the clients so I had to politely decline with “Sorry, we are unable to support that with the existing infrastructure”.

Linux has auditd and at first it looked promising for this problem. I later learned more about the architecture of auditd and found out that the method it uses for the directory watch feature is not capable of auditing reads and writes occurring over NFS because the syscalls for those operations are happening on the client side, not the server-side. What I still don’t quite understand yet is why the Linux NFS server doesn’t have built-in auditing functionality. If such functionality does exist, it’s either undocumented or I’m simply unable to find it.

Samba provides a logging directive that enables file access auditing without client side cooperation so it seems plausible that the GNU/Linux NFS server could implement similar functionality. Disclaimer: I don’t really understand NFS architecture so I don’t know if this is a difficult problem. NetApp appears to support server-side NFS auditing so it looks possible. Wouldn’t it be nice if “enterprise” GNU/Linux vendors would also support server-side auditing of file access over NFS?

References:

FreeIPA and Samba 3 Integration

FreeIPA makes a pretty excellent backend for Samba 3. While all the information one needs to set this up is available online, I wasn’t able to find it all  in one location so I’ve decided to try my best at filling that gap here on techslaves.org. Hopefully this short guide will aid those trying to piece together the various parts necessary to integrate FreeIPA v2 and Samba 3, at least until FreeIPA v3 where there is talk of enabling Samba integration with a simple command line argument to the “ipa-server-install” script.

Not for Domains

It’s important to keep in mind that these instructions are not for a integrating FreeIPA with a Samba domain controller but merely a Samba file server. My understanding is that FreeIPA will never conveniently/properly support the necessary bits to make it a suitable backend for a Samba 3 PDC. I believe FreeIPA will eventually look towards Samba 4 integration (using Domain trusts) for this kind of integration but don’t quote me on that. Either way, these instructions are not for Samba domain controllers, just Samba file servers.

The Assumptions

There are some basic assumptions that these instructions make.

  • FreeIPA is installed and functional
  • You have a general idea of how to use LDAP command line tools
  • If you have a nice GUI LDAP browser, you can use it to apply the example LDIFs and edit the tree instead of the ldap CLI tools
  • The LDAP commands are executed on the FreeIPA server
  • Samba and FreeIPA are installed on the same server (although it shouldn’t be difficult to use TLS encryption with separate servers)
  • Your LDAP suffix is “dc=domain,dc=tld”
  • You know the difference between the “admin” account and the directory manager and their passwords

The Goods

Let’s not beat around the bush any further.

  1. Determine your Samba server SID by executing the following command while smbd is running and jot it down:
root@ipaserver:~
# net getlocalsid
SID for IPASERVER domain  is: S-1-5-21-3180075094-3458813485-3821849995
  1. With the “admin” kerberos ticket, add two attributes to “cn=ipaConfig,dc=etc,dc=domain,dc=tld” that tell FreeIPA to setup each account as a Samba account and each group as a Samba group:
ldapmodify -Y GSSAPI <<EOF
dn: cn=ipaconfig,cn=etc,dc=domain,dc=tld
changetype: modify
add: ipaUserObjectClasses
ipaUserObjectClasses: sambaSAMAccount
-
add: ipaGroupObjectClasses
ipaGroupObjectClasses: sambaGroupMapping
EOF
  1. With the directory manager password and the Samba SID you jotted down from above, create an instance of the 389 DS DNA plugin that will automatically generate SIDs for your users and groups which are necessary for use with Samba:
ldapadd -x -D "cn=Directory Manager" -W <<EOF
dn: cn=SambaGroupSid,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config
objectClass: top
objectClass: extensibleObject
dnatype: sambaSID
dnaprefix: S-1-5-21-3180075094-3458813485-3821849995-
dnainterval: 1
dnamagicregen: assign
dnafilter: (|(objectclass=sambasamaccount)(objectclass=sambagroupmapping))
dnascope: dc=domain,dc=tld
cn: SambaSid
dnanextvalue: 15277
EOF

The thing to note here is that the “dnaprefix” is set to the SID your jotted down… PLUS a hyphen (“-“) appended to the end!

  1. Now we have to start modifying the FreeIPA API, CLI and WebUI to allow us to specify the “sambaGroupType” attribute at group creation time. We have to set “sambaGroupType” because it is a required attribute for the objectClass “sambaGroupMapping” which we are automatically adding to every group with the “ipaGroupObjectClasses” setting from earlier.

Although the value is going to be “4” for every conceivable case in this non-domain configuration, I was not able to figure out how to make the DNA plugin insert static values like it can set incrementing values so I decided to allow setting it through the CLI and WebUS with defaults enabled instead. If anyone knows how to setup 389 to automatically add an attribute with a static value upon DN creation of DNs with specific objectClasses, please tell me.

There are a few steps required to make this CLI/UI stuff happen but the FreeIPA developers have actually made this quite simple.

The rule is: Extend the FreeIPA schema first, then the CLI, then the WebUI.

4.1. Extend the FreeIPA schema with a custom field by adding the attribute “ipaCustomFields” with a value of “Samba Group Type,sambagrouptype,true” to “cn=ipaConfig,dc=etc,dc=domain,dc=tld” with an “admin” kerberos ticket:

ldapmodify -Y GSSAPI <<EOF
dn: cn=ipaconfig,cn=etc,dc=domain,dc=tld
changetype: add
add: ipaCustomFields
ipaCustomFields: "Samba Group Type,sambagrouptype,true"
EOF

As there can only be one “ipaCustomFields” attribute, if you have multiple custom fields you need to separate each definition with a “$” like so: “Samba Group Type,sambagrouptype,true$Description,attrname,isrequiredboolean”.

4.2. Extend the CLI for groups by editing the python file “/…/site-packages/ipalib/plugins/group.py” to define the custom field and specify a default if not implicitly defined (diff):

--- group.py.orig  2011-08-15 14:59:48.570715207 -0700
+++ group.py    2011-08-16 12:43:43.493236507 -0700
@@ -118,6 +118,13 @@
             label=_('GID'),
             doc=_('GID (use this option to set it manually)'),
         ),
+        Int('sambagrouptype',
+            cli_name='sgt',
+            label=_('Samba Group Type'),
+            doc=_('Samba Group Type (default is 4)'),
+            default=4,
+            autofill=True,
+        ),
     )
 
 api.register(group)

Important: Restart “httpd” at this point!

4.3. Test the CLI. With an “admin” (or equivalent priv) kerberos ticket, try creating a new group:

account@ipaserver:~
$ ipa group-add testgrp --desc="Testing the group.py CLI mods"
---------------------
Added group "testgrp"
---------------------
  Group name: testgrp
  Description: Testing the group.py CLI mods
  GID: 1234500010
  Samba Group Type: 4

4.4 With the CLI functioning properly, we can move on to extending the WebUI. To extend the WebUI for group attributes, edit “/usr/share/ipa/ui/group.js” like so (diff):

--- group.js.orig  2011-08-15 10:01:28.515209121 -0700
+++ group.js    2011-08-16 13:52:59.587352034 -0700
@@ -34,6 +34,7 @@
                 column({name: 'cn'}).
                 column({name: 'gidnumber'}).
                 column({name: 'description'}).
+                column({name: 'sambagrouptype'}).
                 dialog(
                     IPA.add_dialog({
                         'name': 'add',
@@ -41,6 +42,7 @@
                     }).
                         field(IPA.text_widget({name: 'cn', undo: false})).
                         field(IPA.text_widget({name: 'description', undo: false})).
+                        field(IPA.select_widget({name: 'sambagrouptype', undo: false, options: [{label: 'Local', value: 4}, {label: 'Domain', value: 2}]})).
                         field(IPA.checkbox_widget({
                             name: 'posix',
                             label: IPA.messages.objects.group.posix,
@@ -56,6 +58,7 @@
                     }).
                         input({name: 'cn' }).
                         input({name: 'description'}).
+                        input({name: 'sambagrouptype'}).
                         input({name: 'gidnumber' }))).
         facet(
             IPA.group_member_user_facet({

And then these the WebUI to ensure that you can both see the attribute in the group list, but also add it via the select widget added to the new/edit group dialog.

That should be it. Questions, comments, suggestions, correction and more… all are welcome!