HDFS Snapshots

Make your HDFS directory snapshotable , in our case test4

[gpadmin@sphdmst01 tmp]$ hdfs dfsadmin -allowSnapshot /test4
Allowing snaphot on /test4 succeeded

[gpadmin@sphdmst01 tmp]$ hdfs dfsadmin -disallowSnapshot /test4
Disallowing snaphot on /test4 succeeded

Create a snapshot

[gpadmin@sphdmst01 ]$ hdfs dfs -createSnapshot /test4 first-snapshot
Created snapshot /test4/.snapshot/first-snapshot

[gpadmin@sphdmst01 tmp]$ hdfs dfs -ls -R /test4/.snapshot
drwxr-xr-x   – gpadmin hadoop          0 2015-08-11 15:28 /test4/.snapshot/first-snapshot
-rw-r–r–   2 gpadmin hadoop      14515 2015-01-12 10:05 /test4/.snapshot/first-snapshot/Hadoop Servers.xlsx
-rw-r–r–   2 gpadmin hadoop          0 2015-01-12 10:04 /test4/.snapshot/first-snapshot/Hadoop_prod.xlsx
-rw-r–r–   2 gpadmin hadoop       4322 2015-01-12 10:08 /test4/.snapshot/first-snapshot/check_hadoop-dfs.sh
-rw-r–r–   2 gpadmin hadoop          0 2015-01-12 10:04 /test4/.snapshot/first-snapshot/pgadmin.log

You can read the content of the file

[gpadmin@sphdmst01 tmp]$ hdfs dfs -cat /test4/.snapshot/first-snapshot/check_hadoop-dfs.sh

Recover the file from the snapshot

[gpadmin@sphdmst01 /]$ hdfs dfs -cp /test4/.snapshot/first-snapshot/check_hadoop-dfs.sh /ovitest

[gpadmin@cmtolsphdmst01 /]$ hdfs dfs -ls /ovitest
Found 6 items
-rw-r–r–   2 gpadmin hadoop       4322 2015-08-11 15:37 /ovitest/check_hadoop-dfs.sh
-rw-r–r–   2 gpadmin hadoop         66 2015-01-13 16:31 /ovitest/test.txt
-rw-r–r–   2 gpadmin hadoop         66 2015-01-13 17:09 /ovitest/test2.txt
-rw-r–r–   2 gpadmin hadoop         66 2015-01-13 17:10 /ovitest/test3.txt
-rw-r–r–   2 gpadmin hadoop         66 2015-01-14 10:52 /ovitest/test4.txt
-rw-r–r–   2 gpadmin hadoop         66 2015-01-14 10:53 /ovitest/test5.txt

Another example :

gpadmin@sphdmst01 ~]$ hdfs dfs -mkdir /test_snapshot

[gpadmin@sphdmst01 ~]$ hdfs dfs -put dfs-old-lsr-1.log /test_snapshot
[gpadmin@sphdmst01 ~]$ hdfs dfs -put dfs-old-fsck-1.log /test_snapshot

[gpadmin@sphdmst01 ~]$ hdfs dfs -ls /test_snapshot
Found 2 items
-rw-r–r–   2 gpadmin hadoop      45341 2015-09-14 09:39 /test_snapshot/dfs-old-fsck-1.log
-rw-r–r–   2 gpadmin hadoop      83862 2015-09-14 09:38 /test_snapshot/dfs-old-lsr-1.log

[gpadmin@sphdmst01 ~]$ hdfs dfs -createSnapshot /test_snapshot snapshot_dir
createSnapshot: Directory is not a snapshottable directory: /test_snapshot

[gpadmin@sphdmst01 ~]$ hdfs dfsadmin  -allowSnapshot /test_snapshot
Allowing snaphot on /test_snapshot succeeded

[gpadmin@sphdmst01 ~]$ hdfs dfs -createSnapshot /test_snapshot snapshot_dir
Created snapshot /test_snapshot/.snapshot/snapshot_dir

Snapshot is read-only, HDFS will protect against user or application deletion of the snapshot

Leave a comment