Project

General

Profile

Support #58629

Sepia Lab Access Request

Added by Mer Xuanyi about 1 year ago. Updated about 1 year ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
User access
Target version:
-
% Done:

0%

Tags:
Reviewed:
Affected Versions:

Description

1) Do you just need VPN access or will you also be running teuthology jobs?

just access

2) Desired Username:

xuanyi.meng

3) Alternate e-mail address(es) we can reach you at:

4) If you don't already have an established history of code contributions to Ceph, is there an existing community or core developer you've worked with who has reviewed your work and can vouch for your access request?

If you answered "No" to # 4, please answer the following (paste directly below the question to keep indentation):

4a) Paste a link to a Blueprint or planning doc of yours that was reviewed at a Ceph Developer Monthly.

4b) Paste a link to an accepted pull request for a major patch or feature.

https://github.com/ceph/ceph/pull/46750

4c) If applicable, include a link to the current project (planning doc, dev branch, or pull request) that you are looking to test.

https://github.com/ceph/ceph/pull/46050

5) Paste your SSH public key(s) between the pre tags

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTjkMeRhDE3X4BS6huupNCe5M+g3v1RtB9wjDmGQdgJKgQgg0vSnEXqMEYY7r88jTrQPvbot66VIyRht6MrFRW5dGpC7SfYZC+cSBoN2ZESq/m4HOLx8sI2rYxEPIj4uzaWE6qEsPjQ8Z7NPP9RNRW9fWQgkFn8FRaPU2oI4vxGN+WabfiBdlbPL8lcMAz0I0SgqPHWUm192o4fFlGblnCFWVRRYEsuQYk4ySIyr/HviINa2nsibZnt01aES7u1ButUf475LmsZrIthTL77c3qkCNCkLb55F35y2GrLH6mdRz9IqD2o8y0AGUmo3gPC/Wpu7Fz6Ca90y8O+kev/bT9R2KEbzDMoOBpBY18DfMN0De0k0scWDYaSwzQ7ohRIe3cCFth3LGBo9+B+fbLCEsRhiBtXGgHGHUG8Vx10vpWROm4Ba7+luxigVjsqry6jIuyIoN48yBdXFSYIoj2HBvWz9VGSOK7kaVZeK9ruN9zu+otKsIAjiu8e40qxosoVMM= root@DESKTOP-VVPDIRP

6) Paste your hashed VPN credentials between the pre tags (Format: user@hostname 22CharacterSalt 65CharacterHashedPassword)

xuanyi.meng@xtaotech bQUR+Xs5prB2HrTLuZ73oQ 78e94831d7df0205a1efbabc9d991427cd677cb5e34b5fa2ccda00b1d58ce27d

History

#1 Updated by Mer Xuanyi about 1 year ago

Does anyone handle this request?

#2 Updated by adam kraitman about 1 year ago

  • Category set to User access
  • Status changed from New to In Progress
  • Assignee set to adam kraitman

#3 Updated by adam kraitman about 1 year ago

Mer Xuanyi is there an existing community or core developer you've worked with who has reviewed your work and can vouch for your access request?

#4 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

Mer Xuanyi is there an existing community or core developer you've worked with who has reviewed your work and can vouch for your access request?

temporarily no, I've been worked with @vshankar and @lxbsz, but I didn't talk to them before this request, so no vouch.

#5 Updated by adam kraitman about 1 year ago

Okay I will add your username but can you please choose a username without a dot?

#6 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

Okay I will add your username but can you please choose a username without a dot?

Thank you, is "mer" ok?

#7 Updated by adam kraitman about 1 year ago

Yes, I am adding it now

#8 Updated by adam kraitman about 1 year ago

Hey Mer Xuanyi,

You should have access to the Sepia lab now. Please verify you're able to connect to the vpn and ssh using the private key matching the pubkey you provided.

Be sure to check out the following links for final workstation setup steps:
https://wiki.sepia.ceph.com/doku.php?id=vpnaccess#vpn_client_access
https://wiki.sepia.ceph.com/doku.php?id=testnodeaccess#ssh_config

Most developers choose to schedule runs from the shared teuthology VM. For information on that, see http://docs.ceph.com/teuthology/docs/intro_testers.html

If you plan on scheduling tests, one of the options you'll need to set with teuthology-suite is -p, --priority. Please refrain from using a priority lower than 101 (lower number = higher priority). When a high priority is used, it locks up too many testnodes at once and prevents other developers from testing changes.

Thanks.

#9 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

Hey Mer Xuanyi,

You should have access to the Sepia lab now. Please verify you're able to connect to the vpn and ssh using the private key matching the pubkey you provided.

Be sure to check out the following links for final workstation setup steps:
https://wiki.sepia.ceph.com/doku.php?id=vpnaccess#vpn_client_access
https://wiki.sepia.ceph.com/doku.php?id=testnodeaccess#ssh_config

Most developers choose to schedule runs from the shared teuthology VM. For information on that, see http://docs.ceph.com/teuthology/docs/intro_testers.html

If you plan on scheduling tests, one of the options you'll need to set with teuthology-suite is -p, --priority. Please refrain from using a priority lower than 101 (lower number = higher priority). When a high priority is used, it locks up too many testnodes at once and prevents other developers from testing changes.

Thanks.

I can't start openvpn-client@sepia serivce, maybe it's because the user changed before?

➜  ~ systemctl status openvpn-client@sepia.service
× openvpn-client@sepia.service - OpenVPN tunnel for sepia
     Loaded: loaded (/usr/lib/systemd/system/openvpn-client@.service; disabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Mon 2023-02-13 14:15:23 CST; 9s ago
       Docs: man:openvpn(8)
             https://community.openvpn.net/openvpn/wiki/Openvpn24ManPage
             https://community.openvpn.net/openvpn/wiki/HOWTO
    Process: 255672 ExecStart=/usr/sbin/openvpn --suppress-timestamps --nobind --config sepia.conf (code=exited,>
   Main PID: 255672 (code=exited, status=1/FAILURE)

Feb 13 14:15:23 DESKTOP-VVPDIRP systemd[1]: Starting OpenVPN tunnel for sepia...
Feb 13 14:15:23 DESKTOP-VVPDIRP openvpn[255672]: Options error: In [CMD-LINE]:1: Error opening configuration fil>
Feb 13 14:15:23 DESKTOP-VVPDIRP openvpn[255672]: Use --help for more information.
Feb 13 14:15:23 DESKTOP-VVPDIRP systemd[1]: openvpn-client@sepia.service: Main process exited, code=exited, stat>
Feb 13 14:15:23 DESKTOP-VVPDIRP systemd[1]: openvpn-client@sepia.service: Failed with result 'exit-code'.
Feb 13 14:15:23 DESKTOP-VVPDIRP systemd[1]: Failed to start OpenVPN tunnel for sepia.

the sepia secret is xuanyi.meng (I've tried change that to mer but still can't boot)

➜  ~ cat /etc/openvpn/sepia/secret
xuanyi.meng@xtaotech
...

#10 Updated by adam kraitman about 1 year ago

Yes maybe it's the user change, please re-run the new-client script with the "mer" user and paste the output

#11 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

Yes maybe it's the user change, please re-run the new-client script with the "mer" user and paste the output

Here is:

➜  openvpn sepia/new-client mer@xtaotech

!!!!! DO NOT RUN THIS SCRIPT MORE THAN ONCE !!!!!

Please paste the following line in your Sepia Lab Access Request tracker ticket:

mer@xtaotech VUjwFfr6OGjDlF0L1Lrvcw 839b438aaeb36eddce541ccdc8a1a0d9934953b83425aaddfc15ec77f773af1e

#12 Updated by adam kraitman about 1 year ago

Run this before you restart the OpenVPN service
sudo sed -i 's/nogroup/openvpn/g' /etc/openvpn/client/sepia/client.conf
and then try to ssh

#13 Updated by Mer Xuanyi about 1 year ago

I realized I've the different dir struct than you, like this:

➜  ~ tree /etc/openvpn
/etc/openvpn
├── client
├── sepia
│   ├── ca.crt
│   ├── client.conf
│   ├── new-client
│   ├── secret
│   └── tlsauth
├── sepia.conf -> sepia/client.conf
├── sepia-vpn-client.tar.gz
└── server
3 directories, 7 files

So I made a symbol link for sepia and now in /etc/openvpn:

➜  openvpn tree
.
├── client
│   ├── sepia -> ../sepia
│   └── sepia.conf -> ../sepia.conf
├── sepia
│   ├── ca.crt
│   ├── client.conf
│   ├── new-client
│   ├── secret
│   └── tlsauth
├── sepia.conf -> sepia/client.conf
├── sepia-vpn-client.tar.gz
└── server

The client.conf after execute sed:

script-security 1
client
remote vpn.sepia.ceph.com 1194
dev tun
remote-random
resolv-retry infinite
nobind
user nobody
group openvpn
persist-tun
persist-key
comp-lzo
verb 2
mute 10
remote-cert-tls server
tls-auth sepia/tlsauth 1
ca sepia/ca.crt
auth-user-pass sepia/secret

Now the openvpn service can boot successful (the manual on this page should be update https://ceph.github.io/sepia/adding_users/)

➜  ~ systemctl status openvpn-client@sepia
● openvpn-client@sepia.service - OpenVPN tunnel for sepia
     Loaded: loaded (/usr/lib/systemd/system/openvpn-client@.service; disabled; vendor preset: disabled)
     Active: active (running) since Wed 2023-02-15 11:05:28 CST; 15min ago
       Docs: man:openvpn(8)
             https://community.openvpn.net/openvpn/wiki/Openvpn24ManPage
             https://community.openvpn.net/openvpn/wiki/HOWTO
   Main PID: 347497 (openvpn)
     Status: "Pre-connection initialization successful" 
      Tasks: 1 (limit: 308305)
     Memory: 2.6M
     CGroup: /system.slice/system-openvpn\x2dclient.slice/openvpn-client@sepia.service
             └─347497 /usr/sbin/openvpn --suppress-timestamps --nobind --config sepia.conf

But ssh still failed:

➜  client ssh mer@teuthology.front.sepia.ceph.com
ssh: connect to host teuthology.front.sepia.ceph.com port 22: Connection timed out

journal of openvpn:

➜  ~ journalctl -u openvpn-client@sepia.service
Feb 15 11:05:28 DESKTOP-VVPDIRP systemd[1]: Started OpenVPN tunnel for sepia.
Feb 15 11:05:28 DESKTOP-VVPDIRP openvpn[347497]: Outgoing Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Feb 15 11:05:28 DESKTOP-VVPDIRP openvpn[347497]: Incoming Control Channel Authentication: Using 160 bit message hash 'SHA1' for HMAC authentication
Feb 15 11:05:29 DESKTOP-VVPDIRP openvpn[347497]: TCP/UDP: Preserving recently used remote address: [AF_INET]8.43.84.129:1194
Feb 15 11:05:29 DESKTOP-VVPDIRP openvpn[347497]: UDP link local: (not bound)
Feb 15 11:05:29 DESKTOP-VVPDIRP openvpn[347497]: UDP link remote: [AF_INET]8.43.84.129:1194
Feb 15 11:05:29 DESKTOP-VVPDIRP openvpn[347497]: NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
Feb 15 11:06:30 DESKTOP-VVPDIRP openvpn[347497]: TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Feb 15 11:06:30 DESKTOP-VVPDIRP openvpn[347497]: TLS Error: TLS handshake failed
Feb 15 11:06:30 DESKTOP-VVPDIRP openvpn[347497]: SIGUSR1[soft,tls-error] received, process restarting

#14 Updated by adam kraitman about 1 year ago

Please change the "group openvpn" to "group nobody" in the client.conf and try to restart the service

#15 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

Please change the "group openvpn" to "group nobody" in the client.conf and try to restart the service

still timeout :(

#16 Updated by adam kraitman about 1 year ago

Can you please paste the output of "ls -l" of both directories where the files exist? I want to see the permissions

#17 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

Can you please paste the output of "ls -l" of both directories where the files exist? I want to see the permissions

➜  openvpn ls -lhart
total 24K
-rw-r--r--   1 root root    2.8K Oct 28  2020 sepia-vpn-client.tar.gz
drwxr-x---   2 root openvpn 4.0K Nov  2 00:00 server
lrwxrwxrwx   1 root root      17 Feb 14 11:27 sepia.conf -> sepia/client.conf
drwxr-xr-x   5 root root    4.0K Feb 14 11:27 .
drwxr-xr-x 103 root root    4.0K Feb 15 10:54 ..
drwxr-x---   2 root openvpn 4.0K Feb 15 11:05 client
drwxrwxr-x   2 1106    1106 4.0K Feb 15 14:39 sepia
➜  openvpn ls -lhart sepia
total 28K
-r--r----- 1 1106 1106  636 Mar 23  2015 tlsauth
-rw-r--r-- 1 1106 1106 1.3K Mar 23  2015 ca.crt
-rwxr-xr-x 1 1106 1106 2.0K Oct 28  2020 new-client
drwxr-xr-x 5 root root 4.0K Feb 14 11:27 ..
drwxrwxr-x 2 1106 1106 4.0K Feb 15 14:39 .
-rw------- 1 root root  100 Feb 15 14:39 secret
-rw-r--r-- 1 1106 1106  273 Feb 15 18:03 client.conf

#18 Updated by adam kraitman about 1 year ago

Change the top folder to look like this

ls l /etc/openvpn/client/
total 8
drwxr-xr-x 2 akraitma akraitma 4096 Jan 25 18:51 sepia
lrwxrwxrwx 1 root root 17 Jan 8 18:34 sepia.conf -> sepia/client.conf
-rw-r--r-
1 root root 2737 Jan 8 12:04 sepia-vpn-client.tar.gz

#19 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

Change the top folder to look like this

ls l /etc/openvpn/client/
total 8
drwxr-xr-x 2 akraitma akraitma 4096 Jan 25 18:51 sepia
lrwxrwxrwx 1 root root 17 Jan 8 18:34 sepia.conf -> sepia/client.conf
-rw-r--r-
1 root root 2737 Jan 8 12:04 sepia-vpn-client.tar.gz

Looks like no different, my config dir now looks like this:

➜  ~openvpn
➜  ~openvpn tree
.
├── client
│   ├── sepia
│   │   ├── ca.crt
│   │   ├── client.conf
│   │   ├── new-client
│   │   ├── secret
│   │   └── tlsauth
│   └── sepia.conf -> sepia/client.conf
├── sepia-vpn-client.tar.gz
└── server

3 directories, 7 files
➜  ~openvpn ls -lhart client
total 12K
lrwxrwxrwx 1 root root      17 Feb 14 11:27 sepia.conf -> sepia/client.conf
drwxrwxr-x 2 root root    4.0K Feb 15 14:39 sepia
drwxr-xr-x 4 root root    4.0K Feb 20 11:11 ..
drwxr-x--- 3 root openvpn 4.0K Feb 20 11:11 .

And I've tried to modify the uid&gid of sepia/ to some other combinations like root:root, root:openvpn, openvpn:openvpn, even mer:mer (I don't have this user before, created by useradd).

In all of these scenes I can start openvpn-client@sepia service, but the ssh is all failed.

So maybe the problem is not with the owner, could you please re-confirm if I use the correct user or command:

ssh  mer@teuthology.front.sepia.ceph.com

Thanks

#20 Updated by Mer Xuanyi about 1 year ago

Or maybe I should delete the sepia dir and try again.

#21 Updated by adam kraitman about 1 year ago

Can you ping teuthology.front.sepia.ceph.com ?

#22 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

Can you ping teuthology.front.sepia.ceph.com ?

I can't, the DNS looks like works, but all packs were dropped.

➜  ~ ping teuthology.front.sepia.ceph.com
PING teuthology.front.sepia.ceph.com (172.21.0.51) 56(84) bytes of data.
^C
--- teuthology.front.sepia.ceph.com ping statistics ---
15 packets transmitted, 0 received, 100% packet loss, time 14582ms

#23 Updated by adam kraitman about 1 year ago

I know there is a firewall in china maybe this is what's blocking it?

#24 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

I know there is a firewall in china maybe this is what's blocking it?

I'm already configured http-proxy in sepia.conf

proto tcp-client
http-proxy proxy-ip proxy-port

And I can ping the vpn.sepia.ceph.com instead of the teuthology.front.sepia.ceph.com

➜  ~ ping vpn.sepia.ceph.com -c5
PING gw.sepia.ceph.com (8.43.84.129) 56(84) bytes of data.
64 bytes from gw.sepia.ceph.com (8.43.84.129): icmp_seq=1 ttl=45 time=266 ms
64 bytes from gw.sepia.ceph.com (8.43.84.129): icmp_seq=2 ttl=45 time=283 ms
64 bytes from gw.sepia.ceph.com (8.43.84.129): icmp_seq=3 ttl=45 time=278 ms
64 bytes from gw.sepia.ceph.com (8.43.84.129): icmp_seq=4 ttl=45 time=285 ms
64 bytes from gw.sepia.ceph.com (8.43.84.129): icmp_seq=5 ttl=45 time=290 ms

--- gw.sepia.ceph.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4003ms
rtt min/avg/max/mdev = 265.614/280.241/290.022/8.301 ms
➜  ~ ping teuthology.front.sepia.ceph.com -c5
PING teuthology.front.sepia.ceph.com (172.21.0.51) 56(84) bytes of data.

--- teuthology.front.sepia.ceph.com ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4184ms

#25 Updated by adam kraitman about 1 year ago

I know that Yingxin Cheng also had this issue with the firewall and configured a proxy maybe you should ask him how he did it ?

#26 Updated by Mer Xuanyi about 1 year ago

adam kraitman wrote:

I know that Yingxin Cheng also had this issue with the firewall and configured a proxy maybe you should ask him how he did it ?

Thanks, I don't have his contact information, maybe you could give me an address.

Also available in: Atom PDF