Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 1060714 - chmod -R of deep directory raises NullPointerException
Summary: chmod -R of deep directory raises NullPointerException
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: rhs-hadoop
Version: unspecified
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: ---
: ---
Assignee: Bradley Childs
QA Contact: Martin Kudlej
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-02-03 12:50 UTC by Martin Kudlej
Modified: 2015-05-26 18:16 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-05-26 18:16:44 UTC


Attachments (Terms of Use)

Description Martin Kudlej 2014-02-03 12:50:41 UTC
Description of problem:
I've created directory which is 500 dirs deep:
$ d="$(seq -s "/d" 1 500)"
$ su test1 -c "hadoop fs -mkdir -p d$d"
Then I've tried to change their permissions:
$ su test1 -c "hadoop fs -chmod -R 000 $(find ./d1/ -name d500)"

and I've got this exception:
14/02/03 09:22:07 INFO glusterfs.GlusterVolume: Initializing gluster volume..
14/02/03 09:22:07 INFO glusterfs.GlusterFileSystem: Configuring GlusterFS
14/02/03 09:22:07 INFO glusterfs.GlusterFileSystem: Initializing GlusterFS,  CRC disabled.
14/02/03 09:22:07 INFO glusterfs.GlusterFileSystem: GIT INFO={git.commit.id.abbrev=51e5108, git.commit.user.email=bchilds@redhat.com, git.commit.message.full=2.1.5 branch/build
, git.commit.id=51e5108fbec0b50d921aeb00ba2489bbdbe3d6ff, git.commit.message.short=2.1.5 branch/build, git.commit.user.name=childsb, git.build.user.name=Unknown, git.commit.id.describe=2.1.4-21-g51e5108, git.build.user.email=Unknown, git.branch=master, git.commit.time=17.01.2014 @ 16:05:54 EST, git.build.time=21.01.2014 @ 02:19:28 EST}
14/02/03 09:22:07 INFO glusterfs.GlusterFileSystem: GIT_TAG=2.1.4
14/02/03 09:22:07 INFO glusterfs.GlusterFileSystem: Configuring GlusterFS
14/02/03 09:22:07 INFO glusterfs.GlusterVolume: Initializing gluster volume..
14/02/03 09:22:07 INFO glusterfs.GlusterVolume: Root of Gluster file system is /mnt/glusterfs
14/02/03 09:22:07 INFO glusterfs.GlusterVolume: mapreduce/superuser daemon : yarn
14/02/03 09:22:07 INFO glusterfs.GlusterVolume: Working directory is : glusterfs:/user/test1
14/02/03 09:22:07 INFO glusterfs.GlusterVolume: Write buffer size : 131072
-chmod: Fatal internal error
java.lang.NullPointerException
        at org.apache.hadoop.fs.shell.PathData.getDirectoryContents(PathData.java:269)
        at org.apache.hadoop.fs.shell.Command.recursePath(Command.java:347)
        at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:308)
        at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:278)
        at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:260)
        at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:244)
        at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190)
        at org.apache.hadoop.fs.shell.Command.run(Command.java:154)
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:255)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:305)

Inotify log does not help.


Version-Release number of selected component (if applicable):


How reproducible:
100% with octal premissions 00X and 111 and maybe another else
0% with octal permissions 777

Actual results:
It raises NullPointerException.

Expected results:
Recursive permission set will work without exception.

Additional info:
$ cat /var/log/hadoop-yarn/yarn/yarn-yarn-resourcemanager-dhcp-lab-124.englab.brq.redhat.com.out
ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 14880
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 32768
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 65536
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Comment 2 Scott Haines 2014-04-09 00:09:44 UTC
Per Apr-02 bug triage meeting, granting both devel and pm acks

Comment 6 Bradley Childs 2015-03-20 15:10:43 UTC
Martin,

the chmod in your problem description isn't correct, Could you supply the proper command?  

  su test1 -c "hadoop fs -chmod -R 000 $(find ./d1/ -name d500)"

returns:

  find: `./d1/': No such file or directory


If I run this:

hadoop fs -chmod -R 755 d1

it completes ok after a very long time.

If I go to the FUSE mount of the gluster volume and try to change a directory to mode 000 I get this:

[mapred@vm-1 mapred]$ chown -R 000 result/
chown: changing ownership of `result/_partition.lst': Operation not permitted
chown: changing ownership of `result/_SUCCESS': Operation not permitted
chown: changing ownership of `result/part-r-00000': Operation not permitted
chown: changing ownership of `result/': Operation not permitted


All of this makes me believe that 000 isn't a valid mode or there is a bug with gluster+FUSE.  Either way changing a directory to 000 would have to be fixed with gluster+FUSE before it will work in Hadoop.


Note You need to log in before you can comment on or make changes to this bug.