Cannot write to Hadoop DFS directory mode 775 group permission UserGroupInformation

I’m running Hadoop 2.6.2 on a private cluster with file-system permissions enabled. The cluster has password files with only system users like hadoop, no personal accounts. I’m accessing DFS from a linux edge node that has personal accounts like mine (‘clott’).

The problem is that I cannot write to a DFS directory (‘shared’) that is mode 775 and group hadoop; the edge node shows me a member of hadoop group. I thought the user-group memberships on the edge node would be used, but no? Here’s what I see:

clott@edge$ id
uid=1003(clott) gid=1003(clott) groups=1003(clott),27(sudo),1001(hadoop)

clott@edge$ hdfs dfs -ls /user/hadoop
Found 3 items
drwxr-xr-x   - hadoop hadoop          0 2015-12-03 16:55 /user/hadoop/DistributedShell
drwxrwxr-x   - hadoop hadoop          0 2015-12-22 09:25 /user/hadoop/shared

clott@edge$ hdfs dfs -mkdir /user/hadoop/shared/foo
mkdir: Permission denied: user=clott, access=WRITE, inode="/user/hadoop/shared":hadoop:hadoop:drwxrwxr-x

I checked the name node and found this warning in its log:

2015-12-22 09:32:40,941 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user clott

That makes some sense – the name node doesn’t grant me access because it doesn’t recognize any groups for me.

So how does the name node find group memberships? Some googling seems to suggest I have to add my user name and various groups to the cluster but that didn’t sound right. Of course this works if I set the DFS directory mode to 777, but that’s not right.

Please help, thanks in advance.

Answer

The definition of hadoop.security.group.mapping (as written here) is:

Class for user to group mapping (get groups for a given user) for ACL. The default implementation, org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback, will determine if the Java Native Interface (JNI) is available. If JNI is available the implementation will use the API within hadoop to resolve a list of groups for a user. If JNI is not available then the shell implementation, ShellBasedUnixGroupsMapping, is used. This implementation shells out to the Linux/Unix environment with the bash -c groups command to resolve a list of groups for a user.

To accomplish your goal you’d need to add your user account (clott) on the NameNode machine and add it to hadoop group there.

If you are going to run MapReduce with your user, you’d need your user account to be configured on NodeManager hosts as well.