EBS
Contents
- 1 How to list and delete EBS Snapshots in Bulk
- 2 How can I identify if my EBS volume is micro-bursting and how can I prevent this from happening?
- 3 How can I obtain metrics from Amazon CloudWatch that show the EBS throughput of my EC2 instance, and then set up an alarm that notifies me when the EC2 instance reaches the throughput limit?
- 4 EBS Benchmarking
- 5 Sharing Encrypted Snapshots (KMS permissions)
- 6 Extend the partition and file system after resizing an EBS volume
- 7 Other way to grow partition and extend fs
- 8 Why df -h seem to show less than the actual size of the volume
How to list and delete EBS Snapshots in Bulk
How to list all EBS snapshots: https://aws.amazon.com/premiumsupport/knowledge-center/ebs-volume-snapshot-ec2-instance/
How to delete EBS snapshots in bulk:
- This deletes all EBS snapshots created before 2020-07-28:
$ for i in $(aws ec2 describe-snapshots --owner-ids self --query 'Snapshots[?StartTime<=`2020-07-28`].SnapshotId' --output text); do aws ec2 delete-snapshot --snapshot-id $i; sleep 5; done;
- Another option: This script does delete the oldest number of snapshots based on the snapshot complete time which is specified in the variable "count".
#!/bin/sh
# this script is to delete snapshots in bulk
#retrieving snapshot IDs under an account
accountID=331851643035 #your aws account ID
count=4 #how many of the oldest snapshots you would like to delete
file=snapshotIds.txt # a file is created under current directory and holds snapshotIds
#following filters the oldest snapshot and writes into the $file
aws ec2 describe-snapshots --owner-ids $accountID | grep SnapshotId | awk '{print $2}' | sed 's/"//g' | tail -$count >$file
#Following code snippet reads snapshot IDs from the $file and delete one by one, Any error occurs logged in error.log file
while IFS= read -r line || -n $line ; do
echo "Deleting snapshot --> $line"
aws ec2 delete-snapshot --snapshot-id $line >>error.log 2>&1
done < $file
[1] Instance types - Instances built on the Nitro System
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#ec2-nitro-instances
[2] Amazon CloudWatch Metrics for Amazon EBS - Graphs in the Amazon EC2 Console
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using_cloudwatch_ebs.html#graphs-in-the-aws-management-console-2
How can I identify if my EBS volume is micro-bursting and how can I prevent this from happening?
https://aws.amazon.com/premiumsupport/knowledge-center/ebs-identify-micro-bursting/
How can I obtain metrics from Amazon CloudWatch that show the EBS throughput of my EC2 instance, and then set up an alarm that notifies me when the EC2 instance reaches the throughput limit?
https://aws.amazon.com/premiumsupport/knowledge-center/ec2-aggregate-ebs-throughput/
EBS Benchmarking
Run Fio and iostat command at the same time!!
iostat -xdmzt 1 600
Sharing Encrypted Snapshots (KMS permissions)
Extend the partition and file system after resizing an EBS volume
https://docs.aws.amazon.com/ebs/latest/userguide/recognize-expanded-volume-linux.html
Other way to grow partition and extend fs
$ sudo /usr/bin/cloud-init -d single -n growpart $ sudo /usr/bin/cloud-init -d single -n resizefs
Why df -h seem to show less than the actual size of the volume
the system actually takes up a small amount of disk space. You would see that the storage device showing the 2.8T storage device in "lsblk" is showing as 2.7T under "df -h."
This is because some of the storage is hidden in the different units of measurement used to report disk capacity. How this space is reported and used varies by filesystem and by tools, as well. I've found a great article breaking down essentially what happens to the "missing" storage, and why it is "missing" (This is an article from Red Hat, but is also relevant to most of, if not all Linux operating systems); [1]