Project

General

Profile

Actions
Copy link

Bug #9260

closed

hadoop fs gets EINVAL

Added by Sage Weil over 9 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
John Spray
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Hadoop/Java
Labels (FS):
Java/Hadoop
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

This will fail on hadoop fs -put with EINVAL. No apparent problems in the libcephfs log.

roles:
- - mon.0
  - mds.0
  - osd.0
  - osd.1
  - osd.2
  - hadoop.master.0
- - mon.1
  - osd.3
  - osd.4
  - osd.5
  - hadoop.slave.0
- - mon.2
  - hadoop.slave.1
  - client.0
tasks:
- ssh_keys: null
- install: null
- ceph:
- hadoop: null
- workunit:
    branch: wip-hadoop
    clients:
      client.0:
      - hadoop/wordcount.sh
interactive-on-error: true

Actions
Copy link
 #1

Updated by John Spray over 9 years ago

Hmm, apparently there's more than one way this can fail:

2014-08-28 14:38:11,866.866 INFO:tasks.workunit.client.0.mira114.stderr:+ tar -xf /home/ubuntu/cephtest/hadoop_input/files.tar
2014-08-28 14:38:11,866.866 INFO:tasks.workunit.client.0.mira114.stderr:+ /home/ubuntu/cephtest/apache_hadoop/bin/hadoop fs -mkdir /wordcount_input
2014-08-28 14:38:12,094.094 INFO:tasks.workunit.client.0.mira114.stderr:Exception in thread "main" java.lang.NoClassDefFoundError: com/ceph/fs/CephFileAlreadyExistsException
2014-08-28 14:38:12,095.095 INFO:tasks.workunit.client.0.mira114.stderr:    at java.lang.Class.forName0(Native Method)
2014-08-28 14:38:12,095.095 INFO:tasks.workunit.client.0.mira114.stderr:    at java.lang.Class.forName(Class.java:270)
2014-08-28 14:38:12,095.095 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
2014-08-28 14:38:12,095.095 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
2014-08-28 14:38:12,095.095 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1380)
2014-08-28 14:38:12,095.095 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
2014-08-28 14:38:12,095.095 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404)
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.fs.FsShell.init(FsShell.java:82)
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.fs.FsShell.run(FsShell.java:1745)
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:    at org.apache.hadoop.fs.FsShell.main(FsShell.java:1895)
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:Caused by: java.lang.ClassNotFoundException: com.ceph.fs.CephFileAlreadyExistsException
2014-08-28 14:38:12,096.096 INFO:tasks.workunit.client.0.mira114.stderr:    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
2014-08-28 14:38:12,097.097 INFO:tasks.workunit.client.0.mira114.stderr:    at java.security.AccessController.doPrivileged(Native Method)
2014-08-28 14:38:12,097.097 INFO:tasks.workunit.client.0.mira114.stderr:    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
2014-08-28 14:38:12,097.097 INFO:tasks.workunit.client.0.mira114.stderr:    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
2014-08-28 14:38:12,097.097 INFO:tasks.workunit.client.0.mira114.stderr:    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
2014-08-28 14:38:12,097.097 INFO:tasks.workunit.client.0.mira114.stderr:    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
2014-08-28 14:38:12,097.097 INFO:tasks.workunit.client.0.mira114.stderr:    ... 14 more
2014-08-28 14:38:12,107.107 INFO:tasks.workunit:Stopping ['hadoop/wordcount.sh'] on client.0...

Actions
Copy link
 #2

Updated by John Spray over 9 years ago

Duh, that last exception was just libcephfs-java not being installed.

Actions
Copy link
 #3

Updated by John Spray over 9 years ago

Could be related to #9266, a recurrence of something trying to look up pool names before osdmap is loaded in client. Going to debug that one first.

Actions
Copy link
 #4

Updated by John Spray over 9 years ago

  • Status changed from New to Resolved
commit e0b19e30235653d9f19586bc442c143ccf2397e5
Author: John Spray <jspray@redhat.com>
Date:   Fri Aug 29 13:29:22 2014 +0100

    qa: fix+cleanup hadoop wordcount test

    The glob for the examples jar was wrong.

    Fixes: #9260

    Signed-off-by: John Spray <john.spray@redhat.com>
Actions
Copy link
 #5

Updated by John Spray over 9 years ago

  • Assignee set to John Spray
Actions
Copy link
 #6

Updated by Greg Farnum almost 8 years ago

  • Component(FS) Hadoop/Java added
Actions
Copy link
 #7

Updated by Patrick Donnelly about 5 years ago

  • Category deleted (48)
  • Labels (FS) Java/Hadoop added
Actions
Copy link

Also available in: Atom PDF