Project

General

Profile

Bug #45390

FreeBSD: osdmap decode and encode does not give the same OSDMap

Added by Willem Jan Withagen 7 months ago. Updated 7 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

The problems occurs both in Octopus and Master.

This is the simple version of a part of test_compression.cc:


#include <errno.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include "gtest/gtest.h" 
#include "common/ceph_context.h" 
#include "common/config.h" 
#include "osd/OSDMap.h" 

TEST(OSDMAP, Dencode)
{
#include "osdmaps/osdmap.2982809.h" 

  bufferlist orig;
  orig.append((char*)osdmap_a, sizeof(osdmap_a));
  cout << "orig length " << orig.length()
        << std::endl;
  orig.hexdump(cout);
  cout  << std::endl;
  uint32_t size = 128*1024;
  OSDMap *o = new OSDMap;
  o->decode(orig);
  bufferlist fbl;
  o->encode(fbl, o->get_encoding_features() | CEPH_FEATURE_RESERVED);
  cout << "Result: " << ((fbl.contents_equal(orig))?"True":"False") << std::endl;
  fbl.hexdump(cout);
  cout  << std::endl;
  ASSERT_TRUE(fbl.contents_equal(orig));
}

This fails in the assert test...
already in the third byte
This is the original read osdmap:

0x8,0x7,0xdf,0x56,0x9,0x0,0x7,0x1,0x91,0x47,0x5,0x0,0xb4,0xf4,0x63,0xa0,
0xc6,0x71,0x43,0xa8,0xbd,0x36,0xe4,0xa,0xb8,0xd2,0x33,0xd2,0x99,0x83,0x2d,0x0,
0x6d,0x34,0x16,0x52,0xe8,0x59,0x97,0x1c,0xa,0x4d,0x4e,0x5e,0x13,0x5c,0x3b,0x35,
0x4,0x0,0x0,0x0,0x4,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x1b,0x5,0x20,0xe,
0x0,0x0,0x1,0x3,0x0,0x2,0x0,0x20,0x0,0x0,0x0,0x20,0x0,0x0,0x0,0x0,
0x0,0x0,0x0,0x0,0x0,0x0,0x40,0x83,0x2d,0x0,0x76,0x4a,0x1,0x0,0x0,0x0,

First hexdump top:

00000000  08 07 df 56 09 00 07 01  91 47 05 00 b4 f4 63 a0  |...V.....G....c.|
00000010  c6 71 43 a8 bd 36 e4 0a  b8 d2 33 d2 99 83 2d 00  |.qC..6....3...-.|
00000020  6d 34 16 52 e8 59 97 1c  0a 4d 4e 5e 13 5c 3b 35  |m4.R.Y...MN^.\;5|
00000030  04 00 00 00 04 00 00 00  00 00 00 00 1b 05 20 0e  |.............. .|
00000040  00 00 01 03 00 02 00 20  00 00 00 20 00 00 00 00  |....... ... ....|
00000050  00 00 00 00 00 00 40 83  2d 00 76 4a 01 00 00 00  |......@.-.vJ....|
00000060  00 00 40 83 2d 00 00 00  00 00 d0 00 00 00 01 00  |..@.-...........|

Second hexdump top:

00000000  08 07 9f 6b 0a 00 07 01  c1 8c 05 00 b4 f4 63 a0  |...k..........c.|
00000010  c6 71 43 a8 bd 36 e4 0a  b8 d2 33 d2 99 83 2d 00  |.qC..6....3...-.|
00000020  6d 34 16 52 e8 59 97 1c  0a 4d 4e 5e 13 5c 3b 35  |m4.R.Y...MN^.\;5|
00000030  04 00 00 00 04 00 00 00  00 00 00 00 1b 05 20 0e  |.............. .|
00000040  00 00 01 03 00 02 00 20  00 00 00 20 00 00 00 00  |....... ... ....|
00000050  00 00 00 00 00 00 40 83  2d 00 76 4a 01 00 00 00  |......@.-.vJ....|
00000060  00 00 40 83 2d 00 00 00  00 00 d0 00 00 00 01 00  |..@.-...........|

So the

History

#1 Updated by Willem Jan Withagen 7 months ago

Added code to dump JSON tree for both cases, and then it seems both trees are equal.
So the serialized OSDMap contains byte that are essential in the actual output.

And could be dependant on correct initialisation of the object?

#2 Updated by Kefu Chai 7 months ago

  • Description updated (diff)

#3 Updated by Kefu Chai 7 months ago

  • Description updated (diff)

#4 Updated by Willem Jan Withagen 7 months ago

Willem Jan Withagen wrote:

Added code to dump JSON tree for both cases, and then it seems both trees are equal.
So the serialized OSDMap contains byte that are essential in the actual output.

And could be dependant on correct initialisation of the object?

Compared the JSON output to the JSON output from a Linux system.
And that shows:
- Listing of the socket adres does not work,
Which is correct, since I have an abondoned PR for fixing
the discepancy between the sockaddr on Linux and BSD
But in the end that'll probably not fix the problem, since we are trying
to translate a linux serialixzed structure into an object, and then back
into a serialisation, to then compare the 2. That is not going to work.
- The time is different, it is off by 1 hour.
Which is suspectly exact my offset to GMT.
Could very well be a representation difference between BSD and Linux again.

So best not to compare serialized structures?

*** /tmp/dump-orig.txt  Wed May  6 15:43:22 2020
--- /tmp/dump-linux-endec.txt   Thu May  7 23:31:06 2020
***************
*** 1,4 ****
! 2982809"b4f463a0-c671-43a8-bd36-e40ab8d233d2""2013-08-22T17:55:25.479681+0200""2020-02-20T10:10:34.893082+0100""0.000000""0.000000""sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit"5799936[
      "pglog_hardlimit",
      "purged_snapdirs",
      "recovery_deletes",
--- 1,4 ----
! 2982809"b4f463a0-c671-43a8-bd36-e40ab8d233d2""2013-08-22T15:55:25.479681+0000""2020-02-20T09:10:34.893082+0000""0.000000""0.000000""sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit"5799936[
      "pglog_hardlimit",
      "purged_snapdirs",
      "recovery_deletes",
***************
*** 314,320 ****
              "addrvec": [
                  {
                      "type": "v1",
!                     "addr": "(unrecognized address family \u0000)",
                      "nonce": 1972920
                  }
              ]
--- 314,320 ----
              "addrvec": [
                  {
                      "type": "v1",
!                     "addr": "128.142.161.208:6827",
                      "nonce": 1972920
                  }
              ]

Also available in: Atom PDF