Project

General

Profile

Bug #17621

Hadoop does a bad job closing files and we end up holding too many caps

Added by Greg Farnum almost 3 years ago. Updated 6 months ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
10/19/2016
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, Hadoop/Java
Labels (FS):
Java/Hadoop
Pull request ID:

Description

(Just scribbling this in the tracker for a record.)

We've had several reports on the mailing list of Hadoop clients persistently holding too many caps. This seems to be related to the Hadoop infrastructure doing a bad job of closing files (possibly because it's relying on finalizers instead of explicitly calling close?). We should find a way to work around it in our bindings since Hadoop's job files are pretty safe for us to drop and re-open on-demand.

History

#1 Updated by Patrick Donnelly 6 months ago

  • Status changed from New to Rejected

Java/Hadoop testing is no longer a priority.

#2 Updated by Patrick Donnelly 6 months ago

  • Category deleted (48)
  • Labels (FS) Java/Hadoop added

Also available in: Atom PDF