AWS- clean up

to keep costs under control for AWS we will clean up below :

  • clean up idle EC2 Instance
  • clean up EBS Volumes which are unattached / all the volumes available to remove
  • clean up snapshots orphans / delete all Snapshots which are not in use
  • all Snapshots not in use by any AMI’s
  • Release elastic IPs when they’re not in use

 

 

#aws ec2 describe-regions | grep -i regionname
“RegionName”: “ap-south-1”
“RegionName”: “eu-west-1”
“RegionName”: “ap-northeast-2”
“RegionName”: “ap-northeast-1”
“RegionName”: “sa-east-1”
“RegionName”: “ap-southeast-1”
“RegionName”: “ap-southeast-2”
“RegionName”: “eu-central-1”
“RegionName”: “us-east-1”
“RegionName”: “us-east-2”
“RegionName”: “us-west-1”
“RegionName”: “us-west-2”

 

# aws ec2 describe-volumes > describe_volumes.txt

# aws ec2 describe-volumes –region us-west-1 > describe_volumes_us-west1.txt

# aws ec2 describe-snapshots > describe_snapshots

 

 

➜ # more describe_snapshots | grep -i SNAPSHOT | awk ‘{print $2}’ | sort | uniq | wc -l
15445

# aws ec2 describe-volumes –region us-west-1 | grep -i available
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,
“State”: “available”,

 

 

 

Reference :

aws clean up

http://www.robertsindall.co.uk/blog/how-to-clean-up-amazon-ebs-volumes-and-snapshots/

Detect useless Snapshots and Volumes in the Amazon EC2 Cloud

http://cloudacademy.com/blog/how-to-manage-ebs-volumes-snapshots-in-aws/

aws – Storage Gateway

gateway-cached volumes

Gateway-cached volumes allow you to utilize Amazon S3 for your primarily data, while retaining some portion of it locally in a cache for

frequently accessed data

 

gateway-stored volumes

gateway -stored volumes store your primary data locally, while asynchronously backing up data to AWS.

 

gateway-virtual tape Library

aws – metadata

Instance metadata is data about your instance tah you can use to configure or manage the running instance

[root@ip-10-192- ]# curl http://169.254.169.254/latest/meta-data/

ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
hostname
instance-action
instance-id
instance-type
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-keys/
reservation-id
security-groups

services/

[root@ip-10-] curl http://169.254.169.254/latest/meta-data/ami-id ; echo
ami-de347abc

 

[root@ip-10-192-10]# curl http://169.254.169.254/
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04
2011-01-01
2011-05-01
2012-01-12
2014-02-25
2014-11-05
2015-10-20
2016-04-19
2016-06-30

root@ip-10-192]# curl http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key

Check public IP behind NAT

[root@ip-10-192- ]# wget -qO- http://ipecho.net/plain ; echo
50.18.yyy.yy

EBS – volume in linux

After you attach an Amazon EBS volume to your instance, it is exposed as a block device. You can format the volume with any file system and then mount it

[root@ip-172-30… //]# lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda    202:0    0   8G  0 disk
└─xvda1 202:1    0   8G  0 part /
xvdf    202:80   0   2G  0 disk

[root@ip-172-30 //]# file -s /dev/xvdf
/dev/xvdf: data

If the output of the previous command shows simply data for the device, then there is no file system on the device and you need to create

[root@ip-172-30-0-59 //]# mkfs -t ext4 /dev/xvdf
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 524288 4k blocks and 131072 inodes
Filesystem UUID: 33193f80-886e-41ad-858e-6be5a4dde19e
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

after format, check again

[root@ip-172-30-//]# file -s /dev/xvdf
/dev/xvdf: Linux rev 1.0 ext4 filesystem data, UUID=33193f80-886e-41ad-858e-6be5a4dde19e (extents) (large files) (huge files)

 

[root@ip-172-30- /]# ls -al /dev/disk/by-uuid/
total 0
drwxr-xr-x 2 root root  80 Oct  4 14:16 .
drwxr-xr-x 7 root root 140 Oct  4 14:16 ..
lrwxrwxrwx 1 root root  10 Oct  4 14:16 33193f80-886e-41ad-858e-6be5a4dde19e -> ../../xvdf
lrwxrwxrwx 1 root root  11 Oct  4 14:17 43c07df6-e944-4b25-8fd1-5ff848b584b2 -> ../../xvda1

edit /etc/fstab

[root@ip-172-30-0-235 /]# cat /etc/fstab
#
LABEL=/     /           ext4    defaults,noatime  1   1
tmpfs       /dev/shm    tmpfs   defaults        0   0
devpts      /dev/pts    devpts  gid=5,mode=620  0   0
sysfs       /sys        sysfs   defaults        0   0
proc        /proc       proc    defaults        0   0
/dev/xvdf   /apps       ext4    defaults        0   0

 

create a directory apps

# mkdir  apps

#mount -a

test

 

[root@ip-172-30- /]# df
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/xvda1       8123812 3819192   4204372  48% /
devtmpfs          498816      60    498756   1% /dev
tmpfs             509664       0    509664   0% /dev/shm
/dev/xvdf        1998672    3076   1874356   1% /apps

 

With Amazon EBS encryption, you can now create an encrypted EBS volume and attach it to a supported instance type. Data on the volume, disk I/O,
and snapshots created from the volume are then all encrypted. The encryption occurs on the servers that host the EC2 instances, providing
encryption of data as it moves between EC2 instances and EBS storage. EBS encryption is based on the industry standard AES-256
cryptographic algorithm.
** Snapshots that are taken from encrypted volumes are automatically encrypted.
** Volumes that are created from encrypted snapshots are also automatically encrypted.

Public snapshots of encrypted volumes are not supported, but you can share an encrypted snapshot with specific accounts if you
take the following steps:

– Use a custom CMK, not your default CMK, to encrypt your volume.
– Give the specific accounts access to the custom CMK.
– Create the snapshot.
– Give the specific accounts access to the snapshot.

– You cannot snapshot an EC2 instance store volume.