Hadoop Security – Hadoop HDFS File Permissions

  • Post author:
  • Post last modified:February 26, 2018
  • Post category:BigData
  • Reading time:3 mins read

Hadoop HDFS file permissions are almost similar to the POSIX file system. In a Linux system, we usually create OS level users and make them members of an existing operating system group. But in Hadoop, we create directory and associate it with an owner and a group.

Hadoop HDFS File Permissions

Hadoop HDFS File and Directory Permissions

The following sections show Hadoop HDFS file and directory permissions:

Just like Linux operating system, Hadoop uses notation (r,w) to denote read and write permissions. There is an execute (x) permission for files but you cannot execute the HDFS files hence it is ignore on files if you use. Execute (x) permission applies to the directories indicate that you can access the subdirectories.

Hadoop HDFS File Permission Example

You can use HDFS command hdfs dfs –ls /path/file.csv to check the file permissions. Below is the example of hdfs file permission:

$ hdfs dfs -ls /test
Found 2 items
drwxr-xr-x - hdpadmin hdfs 0 2017-11-02 10:45 /test/test1
-rw-r--r-- 3 hdpadmin hdfs 60 2017-10-26 11:43 /test/test_ext_tbl.txt

In the above example, on the far left, there is a string of letters. The first letter determines whether a file is a directory or not, and then there are three sets of three letters each. Those sets denote owner, group, and other user permissions, and the “rwx” are read, write, and execute permissions.

Hadoop HDFS Change File Permission

You can use the hdfs chmod command to change the file permission. Below is the command you can use:

hdfs dfs -chmod [-R] <mode | octal mode> <file or directory name>
  • -R modifies the files recursively. This is the only option currently supported.
  • <MODE> Mode is the same as mode used for the shell’s command. The only letters recognized are ‘rwxXt’, e.g. +t, a+r, g-w, +rwx, o=r.
  • <OCTALMODE> Mode specified in 3 or 4 digits. If 4 digits, the first may be 1 or 0 to turn the sticky bit on or off, respectively. E.g. 775 or 777.

Create and Change HDFS directory owner

Below is the command to create directory in the Hadoop ecosystem:

$ hdfs dfs -mkdir /test/test1

Verify the created directoy

$ hdfs dfs -ls /test
Found 2 items
drwxr-xr-x - hdpadmin hdfs 0 2017-11-02 10:45 /test/test1
-rw-r--r-- 3 hdpadmin hdfs 60 2017-10-26 11:43 /test/test_ext_tbl.txt
Below is the command to change the owner of the HDFS directory:
$hdfs dfs -chown -R hdfsadmin:hdfs /test/

Read:

Tags: , ,