Project

General

Profile

Bug #17621

Hadoop does a bad job closing files and we end up holding too many caps

Added by Greg Farnum over 7 years ago. Updated about 5 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, Hadoop/Java
Labels (FS):
Java/Hadoop
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

(Just scribbling this in the tracker for a record.)

We've had several reports on the mailing list of Hadoop clients persistently holding too many caps. This seems to be related to the Hadoop infrastructure doing a bad job of closing files (possibly because it's relying on finalizers instead of explicitly calling close?). We should find a way to work around it in our bindings since Hadoop's job files are pretty safe for us to drop and re-open on-demand.

History

#1 Updated by Patrick Donnelly about 5 years ago

  • Status changed from New to Rejected

Java/Hadoop testing is no longer a priority.

#2 Updated by Patrick Donnelly about 5 years ago

  • Category deleted (48)
  • Labels (FS) Java/Hadoop added

Also available in: Atom PDF