Bug #1656
closedHadoop client unit test failures
0%
Description
The Ceph Hadoop File System passes nearly all its tests except a few. I've included the test log below that shows the failed tests. I do not have time at the moment to work on fixing these but wanted to record them some where.
Testcase: testWorkingDirectory took 0.282 sec Testcase: testMkdirs took 0.048 sec Testcase: testMkdirsFailsForSubdirectoryOfExistingFile took 0.043 sec FAILED Should throw IOException. junit.framework.AssertionFailedError: Should throw IOException. at org.apache.hadoop.fs.FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile(FileSystemContractBaseTest.java:129) Testcase: testGetFileStatusThrowsExceptionForNonExistentFile took 0.023 sec Testcase: testListStatusReturnsNullForNonExistentFile took 0.02 sec Caused an ERROR null java.io.FileNotFoundException at org.apache.hadoop.fs.ceph.CephFileSystem.listStatus(CephFileSystem.java:462) at org.apache.hadoop.fs.FileSystemContractBaseTest.testListStatusReturnsNullForNonExistentFile(FileSystemContractBaseTest.java:157) Testcase: testListStatus took 0.045 sec Testcase: testWriteReadAndDeleteEmptyFile took 0.036 sec Testcase: testWriteReadAndDeleteHalfABlock took 0.038 sec Testcase: testWriteReadAndDeleteOneBlock took 0.04 sec Testcase: testWriteReadAndDeleteOneAndAHalfBlocks took 0.038 sec Testcase: testWriteReadAndDeleteTwoBlocks took 0.045 sec Testcase: testOverwrite took 0.035 sec Testcase: testWriteInNonExistentDirectory took 0.025 sec Testcase: testDeleteNonExistentFile took 0.014 sec Testcase: testDeleteRecursively took 0.019 sec Testcase: testDeleteEmptyDirectory took 0.014 sec Testcase: testRenameNonExistentPath took 0.012 sec Testcase: testRenameFileMoveToNonExistentDirectory took 0.015 sec Testcase: testRenameFileMoveToExistingDirectory took 0.014 sec Testcase: testRenameFileAsExistingFile took 0.019 sec Testcase: testRenameFileAsExistingDirectory took 0.015 sec Testcase: testRenameDirectoryMoveToNonExistentDirectory took 0.011 sec Testcase: testRenameDirectoryMoveToExistingDirectory took 0.02 sec Testcase: testRenameDirectoryAsExistingFile took 0.014 sec Testcase: testRenameDirectoryAsExistingDirectory took 0.018 sec Testcase: testInputStreamClosedTwice took 0.017 sec Testcase: testOutputStreamClosedTwice took 0.012 sec
Updated by Greg Farnum over 12 years ago
What versions of the systems were you running when these failed?
I don't remember how they're set up but they might just be invalid tests for the branch they were run on, or I might have broken something when switching back to the 0.20 branch, or some other similar problem.
Updated by Noah Watkins over 12 years ago
This was hadoop-0.20.205.0 with the latest Ceph master branch.
It looked like the patch in src/client/hadoop was out of date with respect to your recent updates, as well as some paths that have changed.
When you originally made the patch was it against a standard release tarball, or the hadoop-common source (e.g. from git or svn)? I am just about done making a patch that applies to the standard release tarball that should work well while other cleanup efforts continue.
Updated by Greg Farnum over 12 years ago
I believe the patch was made against the then-current svn 0.21 branch (which is now very dead). I pushed changes to the Java code into master yesterday afternoon that are the newest and most up-to-date code I've produced, which is against the 0.20 branch API (or at least my approximation of it).
You can also look at my personal git clone at https://github.com/gregsfortytwo/hadoop-common/tree/ceph-testing -- this contains the necessary Hadoop config changes (plus a couple site-specific ones on top). I don't know if this is an easier-to-digest format or not.
Updated by Noah Watkins over 12 years ago
Alright, so I think at this point I'd like to see two patches:
1) A patch against the downloadable tarball (much easier for people setup than from source, especially for students I'm working with this quarter)
2) A patch for the source tree (hadoop-common) for upstream.
I've used your updated code in master from today to produce (1). Producing (2) is also trivial, but maybe less important in the short term as cleanup continues? Do you want to keep (1) and an updated copy of (2) in the Ceph tree for easier distribution?
One thing I am unaware of at this point is if new features are even being excepted into 0.20 branch, or if anything new must be targetted at the to-be-released 0.23 trunk.
Updated by Greg Farnum over 12 years ago
Sounds good to me -- which patches we want to keep in the tree are probably a management decision but I'm happy to put them in. :)
If you're not going to use (2) let's not bother until we see what changes we make based on your feedback and our own plans going forward.
Updated by Sage Weil over 12 years ago
- Translation missing: en.field_position set to 29
Updated by Sage Weil over 12 years ago
- Translation missing: en.field_position deleted (
29) - Translation missing: en.field_position set to 14
Updated by Sage Weil over 12 years ago
- Translation missing: en.field_position deleted (
16) - Translation missing: en.field_position set to 1
Updated by Sage Weil over 12 years ago
- Translation missing: en.field_story_points set to 1
- Translation missing: en.field_position deleted (
1) - Translation missing: en.field_position set to 1
Updated by Sage Weil over 12 years ago
- Translation missing: en.field_position deleted (
1) - Translation missing: en.field_position set to 18
Updated by Sage Weil over 12 years ago
- Translation missing: en.field_position deleted (
14) - Translation missing: en.field_position set to 30
Updated by Sage Weil over 11 years ago
- Project changed from Ceph to CephFS
- Category deleted (
20)
Updated by John Spray about 7 years ago
- Status changed from New to Won't Fix
I'm told that there is a newer hdfs test suite that we would adopt if refreshing the hdfs support, so this ticket is no longer relevant.
Updated by Patrick Donnelly about 5 years ago
- Category deleted (
48) - Labels (FS) Java/Hadoop added