Einladung OpenLab #2: Verschlüsselung

Ok, my last version was a pure Bash solution: working, but slow. There were some comments how to improve the performance and I decided finally to reimplement the second script as Python script.
The Bash script didn’t finish within a day. The Python script ends after 1-2 hours in my test scenario. So, here are the scripts again:

make-lsLR.sh – call this regularly (cron) to create the needed files that are stored in /root/. Of course you can alter the location easily and exclude other directories from being scanned. [download make-lsLR.sh]
# Usage: ./make-lsLR.sh
# Purpose:
# to make a file that is parseable for recovering
# a filled /lost+found directory by parsing
# filesize, md5sum, permisions and path+filename
# Author: Ingo Juergensman – http://blog.windfluechter.net
# License: GPL v2, see http://gnu.org for details.
# first: get all directories
nice -15 find / -path /sys -prune -o
-path /proc -prune -o
-path /var/lib/backuppc -prune -o
-path /var/spool/squid -prune -o
-type d -printf “%U:%G %#m ” -print > /root/ls-md5sum-dirs.txt

exit 0

# next: get all relevant information
nice -15 find / -path /sys -prune -o
-path /proc -prune -o
-path /var/lib/backuppc -prune -o
-path /var/spool/squid -prune -o
-type f -printf “%s %U:%G %#m ”
-exec nice -15 md5sum {} ; | tr -s ” ” > /root/ls-md5sum-files.txt

check_lost+found.py – The second script is to be run when your fsck managed to mess up with your files and stored them into lost+found directory. It takes 3 arguments: 1) the source directory where your messed up lost+found directory is, 2) the target directory to which the data will be saved and 3) a switch to actually make it happen instead of a dry-run. [download check_lostfound.py.sh]
# usage: check_lostfound.py [make_it_so]
# Purpose: to find files in lost+found and trying to restore
# original files by comparing ls-md5sum-files.txt (generated by
# make-lsLR.sh
# Option make_it_so cause the data actually being written/moved
# whereas the script runs in dry mode per default.
# Author: Ingo Juergensman – http://blog.windfluechter.net
# License: GPL v2, see http://gnu.org for details.

from string import *
import sys, os
from os.path import *
import string

if len(sys.argv)<>4:
print “usage: “+sys.argv[0]+”
mnt = sys.argv[1]
target = sys.argv[2]
mode = sys.argv[3]

print “Creating list of files in %s/lost+found” % mnt
cmd = ‘find %s/lost+found -type f -printf “%%s %%U:%%G %%#m ” -exec nice -15 md5sum {} ; > /root/lostfound-files.txt’ % mnt

d = open(dirs, ‘r+’)
f = open(source, ‘r+’)
l = open(lost, ‘r+’)


# create the missing directories first
print “Creating missing directories in %s” % target
for entry in d:
ugid=string.split(entry, ” “, 2)[0]
perm=string.split(entry, ” “, 2)[1]
pfad=”%s%s” % (target,string.split(replace(entry, ”
“, “”), ” “, 2)[2:][0])
res = isdir(pfad)
if (os.path.exists(pfad) and os.path.isdir(pfad)):
print “%s exists… ” % pfad
cmd = “%s %s – mkdir %s” % (ugid, perm, pfad)
print cmd
if mode == “make_it_so”:
uid, gid = string.split(ugid, “:”)
os.chown(pfad, int(uid), int(gid))
os.chmod(pfad, int(perm))
print “%s exists… ” % pfad

# now parse /root/ls-md5sum-files.txt to get
# md5sum and /path/to/filename pairs
for line in f:
line=replace(line, ”
“, “”)
#size1 = split(line)[0]
#ugid1 = split(line)[1]
#perm1 = split(line)[2]
md5s1 = string.split(line)[3]
#path1 = split(line)[4]
sfiles[md5s1] = strip(str(string.split(line,” “, 4)[4:][0]))

# next: do the same to the files in lost+found
for line in l:
size2 = string.split(line)[0]
ugid2 = string.split(line)[1]
perm2 = string.split(line)[2]
md5s2 = string.split(line)[3]
path2 = strip(str(string.split(replace(line, ”
“, “”),” “, 4)[4:][0]))
s = “%s %s %s %s %s” % (md5s2, ugid2, perm2, size2, path2)

# finally look at lost+found and copy the files
# to the appropriate place. Instead of copying
# the files can be moved as well, but to copy is
# safer in case of mistakes or errors
for lf in lfiles:
md5s = string.split(lf)[0]
lfile = string.split(lf, ” “, 4)[4:][0]
if sfiles.has_key(md5s):
if os.path.exists(target+sfiles[md5s]):
targetfile = sfiles[md5s]
print “restoring %s%s ” % (target, targetfile) #lfile
cmd = ‘cp -p “%s” “%s%s”‘ % (lfile, target, targetfile)
if mode == “make_it_so”:
del sfiles[md5s]


I’ve chosen to copy the files to a different place instead of moving them within the same filesystem to their original place for safety reasons. Primary goal is to retrieve the files from lost+found, not to replace a full featured backup and restore application. Because of this the script doesn’t handle hard- nor symlinks correctly. It just copy files.

Of course there’s still room for improvements, like handling hard-/symlinks correctly or using inode number instead of md5sums to move data back to its prior location. But it works for me[tm] well enough in this way, so I’m satisfied so far. You’re welcome, though, to improve this piece of ugliness if you like.

Maybe someone else finds this usefull as well. Use it on your own risk, of course. 🙂


2 thoughts on “Einladung OpenLab #2: Verschlüsselung

  1. Email Verschlüsselung wird in unserem sich immer mehr für die Dinger der Bürger interessierenden Staat immer wichtiger. Ich nutze GnuPG schon seit 5 jahren.

  2. Hmmm, *gruebel* das erste mal hab ich PGP vor ueber 10 Jahren benutzt… noch auf dem Amiga.
    Das Problem mit der Verschluesselung ist, dass man nicht einfach so seine Mails verschluesseln kann, da die meisten Empfaenger nichts damit anzufangen wissen. Da muss noch viel Ueberzeugungsarbeit geleistet werden.

    Aber ich denke, das wird auch alles beim OpenLab thematisiert… 😉

Comments are closed.