Project

General

Profile

Actions

Bug #14733

closed

Could not fetch updated apt files

Added by Loïc Dachary about 8 years ago. Updated about 8 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
Infrastructure Service
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

General failure of some suites today

because of

TASK: [testnode | Update apt cache.] ****************************************** 

2016-02-10T20:34:32.434 INFO:teuthology.task.ansible.out:[0;31mfailed: [mira077.front.sepia.ceph.com] => {"failed": true}[0m
[0;31mmsg: Could not fetch updated apt files[0m

2016-02-10T20:34:32.437 INFO:teuthology.task.ansible.out:[0;31m
FATAL: all hosts have already failed -- aborting[0m

2016-02-10T20:34:32.438 INFO:teuthology.task.ansible.out:
PLAY RECAP ******************************************************************** 
           to retry, use: --limit @/var/lib/teuthworker/cephlab.retry

[0;31mmira077.front.sepia.ceph.com[0m : [0;32mok[0m[0;32m=[0m[0;32m40[0m   [0;33mchanged[0m[0;33m=[0m[0;33m2[0m    unreachable=0    [0;31mfailed[0m[0;31m=[0m[0;31m1[0m   

2016-02-10T20:34:32.662 INFO:teuthology.task.ansible:Archiving ansible failure log at: /var/lib/teuthworker/archive/loic-2016-02-10_20:32:58-rados-infernalis-backports---basic-multi/4911/ansible_failures.yaml
2016-02-10T20:34:32.674 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 69, in run_tasks
    manager.__enter__()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/__init__.py", line 121, in __enter__
    self.begin()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/ansible.py", line 231, in begin
    self.execute_playbook()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/ansible.py", line 256, in execute_playbook
    self._handle_failure(command, status)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/ansible.py", line 279, in _handle_failure
    raise AnsibleFailedError(failures)
AnsibleFailedError: {'mira077.front.sepia.ceph.com': {'invocation': {'module_name': 'apt', 'module_args': ''}, 'failed': True, 'msg': 'Could not fetch updated apt files'}}
2016-02-10T20:34:32.722 ERROR:teuthology.run_tasks: Sentry event: http://sentry.ceph.com/sepia/teuthology/?q=40d1d1c46cb646ecbe3c13cdba9c2dba
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 69, in run_tasks
    manager.__enter__()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/__init__.py", line 121, in __enter__
    self.begin()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/ansible.py", line 231, in begin
    self.execute_playbook()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/ansible.py", line 256, in execute_playbook
    self._handle_failure(command, status)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/ansible.py", line 279, in _handle_failure
    raise AnsibleFailedError(failures)
AnsibleFailedError: {'mira077.front.sepia.ceph.com': {'invocation': {'module_name': 'apt', 'module_args': ''}, 'failed': True, 'msg': 'Could not fetch updated apt files'}}
2016-02-10T20:34:32.723 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab
Actions #2

Updated by David Galloway about 8 years ago

  • Category set to Infrastructure Service
  • Status changed from New to In Progress
  • Assignee set to David Galloway
W: Failed to fetch http://apt-mirror.front.sepia.ceph.com/archive.ubuntu.com/ubuntu/dists/trusty-updates/multiverse/i18n/Translation-en  Hash Sum mismatch

E: Some index files failed to download. They have been ignored, or old ones used instead.
status is: 100

Paused queue for now and running apt-mirror.sh

Actions #4

Updated by Dan Mick about 8 years ago

So...the upstream format of the i18n Index files has changed; I'm sure it's an error but we can put a hack into a local copy of apt-mirror.

The problem is that, if you look at, say, http://archive.ubuntu.com/ubuntu/dists/trusty-updates/main/i18n/Index, those paths should not contain "main/i18n/" because apt-mirror adds them to a base path that already contains "main/i18n". And it always has. (and across our mirrored repos the only ones that do are ones that were changed last night).

So I'm certain this is a format change upstream that is killing apt-mirror. I've put a hack into a fork of the apt-mirror source, derived from 0.5.1 (because that's the ubuntu version), and have installed that version of apt-mirror and it resolves the problem. See https://github.com/dmick/apt-mirror/commit/cb2f8d4f9effb994b63e7d613d6590c37697481a

Actions #6

Updated by Dan Mick about 8 years ago

  • Status changed from In Progress to Resolved
  • Assignee changed from David Galloway to Dan Mick
Actions

Also available in: Atom PDF