My Media project

I have had a digital camera for about 5 years now, and we've accumulated a lot of pictures from it. The biggest problem we've run into is that we have a tendency to not ever print any of the pics, nor really to organize them, or really do anything else with them. In the face of my fourth hard drive "data event", I'm looking for The Better Way to deal with the volume of digital media I'm saving.

What am I trying to do:
1. Never lose an original piece of digital media (photo, video, CD, .m4p, etc)
2. Have my digital media accessible so we can use it, in reasonable format (web-thumbnail, .mp3, etc)
3. Not keep a buncha copies of everything
4. Do it automatically as much as possible

So far what I have is about 350GB of potentially-original pictures and video on our home computer and on a set of DVDs. (m4p's are not included yet, and CDs are still on the shelf). I've got Apache sharing them out via WebDAV onto our local lan. I'd like to turn on SSL client authentication (distribute self-signed certs to our laptops) and be able to mount that drive up from any net connection. But there's a couple of minor issues (I'll talk about them later). I'm seeing the limits of my Unix-oriented thought processes when I'm running on a Windows XP Home platform.

For example, #3 above is easily accomplished -- "find . | xargs md5sum | sort" piped to a short awk script... if the md5sums match, diff the files, if they're identical, remove the second, and hardlink a second "copy" in its place. But Windows doesn't have find, xargs, sort, md5sum, awk, diff, or ln. (without cygwin)

#2 I think could be solved with a tool like Subversion... Check in the original media (take a CD for example...) titleoftrack1.wav and tag that version "primary". Then check it out, oggenc it, "svn rename .wav .ogg" and tag that version ogg-hq. Similarly, encode a mp3-hq, mp3-lq, etc. (I only have a 2GB ipod so I like 64kbps vbr mp3's there, but can afford higher-quality on my laptop or home stereo)

#1... I burn two copies of the original to DVD+R, md5sum the original (store the md5sums on the DVD too) and compare them after writing. I need 2 verified copies. One should be offsite, the other in a storage container onsite. (offsite will be implemented someday)

And that just leaves 4. How the **** do I automate all this?

--Joe