Hadoop HDFS file permissions are almost similar to the POSIX file system. In a Linux system, we usually create OS level users and make them members of an existing operating system group. But in Hadoop, we create directory and associate it with an owner and a group.
Hadoop HDFS File and Directory Permissions
The following sections show Hadoop HDFS file and directory permissions:
Just like Linux operating system, Hadoop uses notation (r,w) to denote read and write permissions. There is an execute (x) permission for files but you cannot execute the HDFS files hence it is ignore on files if you use. Execute (x) permission applies to the directories indicate that you can access the subdirectories.
Hadoop HDFS File Permission Example
You can use HDFS command hdfs dfs –ls /path/file.csv to check the file permissions. Below is the example of hdfs file permission:
$ hdfs dfs -ls /test Found 2 items drwxr-xr-x - hdpadmin hdfs 0 2017-11-02 10:45 /test/test1 -rw-r--r-- 3 hdpadmin hdfs 60 2017-10-26 11:43 /test/test_ext_tbl.txt
In the above example, on the far left, there is a string of letters. The first letter determines whether a file is a directory or not, and then there are three sets of three letters each. Those sets denote owner, group, and other user permissions, and the “rwx” are read, write, and execute permissions.
Hadoop HDFS Change File Permission
You can use the hdfs chmod command to change the file permission. Below is the command you can use:
hdfs dfs -chmod [-R] <mode | octal mode> <file or directory name>
- -R modifies the files recursively. This is the only option currently supported.
- <MODE> Mode is the same as mode used for the shell’s command. The only letters recognized are ‘rwxXt’, e.g. +t, a+r, g-w, +rwx, o=r.
- <OCTALMODE> Mode specified in 3 or 4 digits. If 4 digits, the first may be 1 or 0 to turn the sticky bit on or off, respectively. E.g. 775 or 777.
Create and Change HDFS directory owner
Below is the command to create directory in the Hadoop ecosystem:
$ hdfs dfs -mkdir /test/test1
Verify the created directoy
$ hdfs dfs -ls /test Found 2 items drwxr-xr-x - hdpadmin hdfs 0 2017-11-02 10:45 /test/test1 -rw-r--r-- 3 hdpadmin hdfs 60 2017-10-26 11:43 /test/test_ext_tbl.txt Below is the command to change the owner of the HDFS directory: $hdfs dfs -chown -R hdfsadmin:hdfs /test/
Read:
- Hadoop HDFS Schema Design for ETL Process
- Hadoop Data Warehouse and Design Considerations
- Hadoop Single Node Cluster Setup on Ubuntu
- 7 Best Hadoop Books to Learn Bigdata Hadoop
- Impala Architecture
- Import using Apache Sqoop
- Export using Apache Sqoop
- Sqoop Command with Secure Password