CloudEndure

From Dikapedia
Jump to: navigation, search

CloudEndure log files

  • Installation log file: ./cloudendure.log
    • Generated during the agent installation process.
    • Located in the directory from which the installation process was initiated.
  • Agent Logs: /var/lib/cloudendure/agent.log.0


How to start/stop the CloudEndure Agent

sudo /var/lib/cloudendure/runAgent.sh 
sudo /var/lib/cloudendure/stopAgent.sh


  • Note: stop / starting of the agent causes re-replication (always), Stopping replication from the console will cause restart of replication (from 0, restart from scratch). Reinstalling of agent (such as re-installing the agent while the old agent is still running) causes rescan (in most cases like migration project) and rebooting the source machine will cause a rescan. However, from me and cheehons test, if its a DR project, and we re-install the agent, it will go through the initial sync process again.


  • A rescan will occur upon Agent reinstallation
  • A rescan will occur upon the addition, removal, or resize of a disk
  • A rescan will occur once the Source Machine is rebooted for the first time after Agent installation.
  • A volume that was written to outside the system will not be consistent. (Ex. If a user shuts down a machine, attaches one or more of its volumes to a different machine, writes to the volumes, and then re-attached the volumes.) You must manually restart the Agent for the volume to be rescanned.
  • Fast startup (hiberboot) is not supported. A rescan will occur if hiberboot is enabled.
  • Restarting the CloudEndure service will cause a rescan.
  • Boot from VHD is not supported – a rescan will occur.
  • Bitlocker on the boot partition is not supported – a rescan will occur.
  • Mirrored system partition is not supported – a rescan will occur.

Ref: https://docs.cloudendure.com/#Installing_the_CloudEndure_Agents/Installing_the_Agents/Installing_the_Agents.htm?Highlight=no%20rescan


  • Note: If an Agent in re-installed, then a new initial sync process will be started, and all of your saved Point in Time snapshots (in case of Disaster Recovery) will be deleted.

How to check the status of CloudEndure Agent

On the Source machine, enter the following command: 

ps -ef | grep cloudendure | grep -v grep | grep -v bash | wc -l

The results are as follows:

5 = The Agent is fully running

Less than 5 = Some of the components may not be running.


Uninstalling an Agent from a Linux Target machine

Run as root or with sudo the following commands: 

/var/lib/cloudendure/stopAgent.sh

/var/lib/cloudendure/install_agent --remove

Then remove the machine from the console.


How to specify specific volumes to be replication

sudo python ./installer_linux.py -t <token> --force-volumes --devices=/dev/xvda --no-prompt
  • One caveat though, this flag isn't supported if they're doing AWS->AWS replication setup. But if they are doing AWS->AWS, they can work around that by changing it to OTHER->AWS.
  • --force-volumes must be used with the --no-prompt argument for cancelling the automatic Agent Installer detection of physical disks to replicate

Example: 

sudo python3 aws-replication-installer-init.py --aws-access-key-id <accesskeyId> --aws-secret-access-key <secretAccessKey> --force-volumes --devices="/dev/nvme0n1,/dev/mapper/mpathb,/dev/mapper/mpathc" --no-prompt

CloudEndure Networking and Communication

To know more about CloudEndure's SSL encryption and network communication, please check your internal CloudEndure notes in quip.

Troubleshooting Communication Errors

https://docs.cloudendure.com/#FAQ/Troubleshooting_Communication_Errors/Troubleshooting_Communication_Errors.htm%3FTocPath%3DNavigation%7CFAQ%25C2%25A0and%25C2%25A0Troubleshooting%7CTroubleshooting%2520Communication%2520Errors%7C_____0



Post Launch Scripts

https://docs.cloudendure.com/#FAQ/FAQ/Post_Launch_Scripts.htm?Highlight=post%20boot

Example of one I used before: 

mkdir -p /boot/post_launch
touch /boot/post_launch/clear_eth0.sh
echo "#!/bin/bash" > /boot/post_launch/clear_eth0.sh
echo "rm -f /etc/sysconfig/network/ifcfg-eth0" >> /boot/post_launch/clear_eth0.sh
chmod +x /boot/post_launch/clear_eth0.sh

This will utilize the post_launch functionality of the CloudEndure service, to automatically remove the ifcfg-eth0 file as the last step in the conversion process, thus removing the manual efforts needed after launch (When source machine has ens5, but conversion process generates an ifcfg-eth0 file which causes issues).


How to install CE agent on RHEL6

1. Downloaded RHEL6.10 ISO from: https://archive.org/details/rhel-server-6.10-x86_64-dvd 2. Uploaded to my vCenter environment, and created a new VM. 3. Followed steps in https://www.tecmint.com/installation-of-rhel-6/ to install rhel6 on LVM partitions. Tried to configure the LVMs as similar as the customer's.

1. Installed the exact same kernel as customers on RHEL6 (2.6.32-754.35.1): 1A. http://rpm.pbone.net/resultsb_dist_79_size_30318452_name_kernel-firmware-2.6.32-754.35.1.el6.noarch.rpm.html 

# wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.10/updates/x86_64/Packages/kernel-firmware-2.6.32-754.35.1.el6.noarch.rpm
# rpm -i kernel-firmware-2.6.32-754.35.1.el6.noarch.rpm

1B. http://rpm.pbone.net/info_idpl_73482398_distro_centos6_com_kernel-devel-2.6.32-754.35.1.el6.x86_64.rpm.html 

# wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.10/updates/x86_64/Packages/kernel-devel-2.6.32-754.35.1.el6.x86_64.rpm
# rpm -i kernel-devel-2.6.32-754.35.1.el6.x86_64.rpm

1C. http://rpm.pbone.net/resultsb_dist_79_size_33985172_name_kernel-2.6.32-754.35.1.el6.x86_64.rpm.html 

# wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.10/updates/x86_64/Packages/kernel-2.6.32-754.35.1.el6.x86_64.rpm
# rpm -i kernel-2.6.32-754.35.1.el6.x86_64.rpm

1D. Run dracut: 

dracut -v -f /boot/initramfs-2.6.32-754.35.1.el6.x86_64.img 2.6.32-754.35.1.el6.x86_64

1E. Verified grub, default is 0, and is pointing to the kernel 2.6.32-754.35: 

cat /etc/grub.conf

5. Reboot and confirm the kernel version is correct 

# uname -r
2.6.32-754.35.1.el6.x86_64


6. Attempted to install the CE agent, however, it failed due to gcc not found. Had to install gcc manually, which required some dependencies to be installed as well. See the next section below:


How to install gcc manually on RHEL6

(This is needed for CloudEndure Agent installation) Had to install gcc manually, which required some dependencies to be installed as well:

Download and install dependencies, and then install gcc-4.4.7-23: 

# wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.3/os/x86_64/Packages/ppl-0.10.2-11.el6.x86_64.rpm
# rpm -i ppl-0.10.2-11.el6.x86_64.rpm

# wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.6/os/x86_64/Packages/mpfr-2.4.1-6.el6.x86_64.rpm
# rpm -i mpfr-2.4.1-6.el6.x86_64.rpm

# wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.3/os/x86_64/Packages/cloog-ppl-0.15.7-1.2.el6.x86_64.rpm
# rpm -i cloog-ppl-0.15.7-1.2.el6.x86_64.rpm

# wget  ftp://ftp.pbone.net/mirror/vault.centos.org/6.10/os/x86_64/Packages/cpp-4.4.7-23.el6.x86_64.rpm
# rpm -i cpp-4.4.7-23.el6.x86_64.rpm

# wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.10/os/x86_64/Packages/gcc-4.4.7-23.el6.x86_64.rpm
# rpm -i gcc-4.4.7-23.el6.x86_64.rpm

You can run it as a script: 

#!/bin/bash 
wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.3/os/x86_64/Packages/ppl-0.10.2-11.el6.x86_64.rpm
rpm -i ppl-0.10.2-11.el6.x86_64.rpm

wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.6/os/x86_64/Packages/mpfr-2.4.1-6.el6.x86_64.rpm
rpm -i mpfr-2.4.1-6.el6.x86_64.rpm

wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.3/os/x86_64/Packages/cloog-ppl-0.15.7-1.2.el6.x86_64.rpm
rpm -i cloog-ppl-0.15.7-1.2.el6.x86_64.rpm

wget  ftp://ftp.pbone.net/mirror/vault.centos.org/6.10/os/x86_64/Packages/cpp-4.4.7-23.el6.x86_64.rpm
rpm -i cpp-4.4.7-23.el6.x86_64.rpm

wget ftp://ftp.pbone.net/mirror/vault.centos.org/6.10/os/x86_64/Packages/gcc-4.4.7-23.el6.x86_64.rpm
rpm -i gcc-4.4.7-23.el6.x86_64.rpm


How to install the Replication Agent on a secured network for MGN (step-by-step guide)

1) Set up the MGN Replication template to use the private subnet (subnet-78c77246) and use private IP for replication.

2) Create the following endpoints:

  • s3 gateway endpoint (full access policy) - This is needed to wget the installer
  • s3 interface endpoint (full access policy) - This will be used upon agent installation --s3-endpoint
  • ec2 interface endpoint (full access policy)
  • mgn interface endpoint (full access policy) - This will be used upon agent installation --endpoint

3) Create the following:

  • Bastion AL2 in public subnet
  • Bastion Windows in public subnet
  • Private AL2 in private subnet (subnet-78c77246)
  • Private Windows in private subnet

4) SSH to the private AL2 from the bastion AL2 and download the installer. Or use the Bastion Windows to RDP to the Private windows instance.

5) Install the agent like so: 

sudo python3 aws-replication-installer-init.py --region us-east-1 --aws-access-key-id <access_key> --aws-secret-access-key <secret_access_key> --no-prompt --endpoint vpce-00d948d85cc80da2e-uz02i5zz-us-east-1b.mgn.us-east-1.vpce.amazonaws.com --s3-endpoint vpce-07bda406c858874d6-16mb55iu-us-east-1e.s3.us-east-1.vpce.amazonaws.com

Replication started Successfully. Letting it reach CDP.


SSL Speed test for Linux

See: https://docs.aws.amazon.com/drs/latest/userguide/perform-connectivity-bandwidth-test.html

There are a couple options. You can use Lynx - A terminal web browser - https://www.geeksforgeeks.org/using-lynx-to-browse-the-web-from-the-linux-terminal/

or speedtest-cli package. this works to do a speedtest against the CE Speedtest AMI without a browser in Linux: https://github.com/librespeed/speedtest-cli

1. Launch a test EC2 instance using the publish AMI CE-ssl-speedtest in your target region. The EC2 instance must be the same instance type as the replication server.

2. Select the same subnet as the subnet used in the replication settings of your source server.

3. Make sure that the security group allows TCP Port 1500 inbound access.

4. On the source server, configure the SpeedTest CLI as shown in the following example: 

(Run as root)
# cd /tmp
# git clone https://github.com/librespeed/speedtest-cli.git
# cd speedtest-cli/
# ls -l
# ./build.sh
  • NOTE: You may need to install Go if you get "Go: command not found". Here is how you can do it on AL2:
# wget --no-verbose https://go.dev/dl/go1.20.5.linux-amd64.tar.gz
# tar -C /usr/local -xzf go1.20.5.linux-amd64.tar.gz
# export PATH=$PATH:/usr/local/go/bin
# go version

Continue on... 

# ./build.sh
# cat << EOF >> ./servers.json
[
  {
    "id": 1,
    "name": "PHP Backend",
    "server": "https://<test server private IP>:1500/speedtest/",
    "dlURL": "/garbage.php",
    "ulURL": "/empty.php",
    "pingURL": "/empty.php"
  }
 ]
EOF
  • make sure to replace the test server's IP address. If you're using the test server's public IP for a speed test, include "getIpURL": "/getIP.php" after the "pingURL" line.

5. Run the LibreSpeed CLI as shown in the following example to test the replication speed: 

# ./out/librespeed-cli-linux-amd64 —local-json ./servers.json —server 1 —no-icmp —skip-cert-verify —simple
Retrieving server list from https://librespeed.org/backend-servers/servers.php
Selecting the fastest server based on ping
Selected server: New York, United States (2) (Clouvider) [nyc.speedtest.clouvider.net]
Sponsored by: Clouvider @ https://www.clouvider.co.uk/
You're testing from: 34.229.93.121 - Unknown ISP
Ping: 12.55 ms	Jitter: 15.69 ms
Download rate:	640.79 Mbps
Upload rate:	661.86 Mbps


https://www.tecmint.com/check-internet-speed-from-command-line-in-linux/

Retrieved from "https://dikapedia.com/wiki/index.php?title=CloudEndure&oldid=2969"