{"id":2380,"date":"2008-01-18T15:22:45","date_gmt":"2008-01-18T20:22:45","guid":{"rendered":"http:\/\/localhost\/wordpress\/2008\/01\/18\/cheap-and-dirty-windows-log-file-archiving\/"},"modified":"2008-01-18T15:22:45","modified_gmt":"2008-01-18T20:22:45","slug":"cheap-and-dirty-windows-log-file-archiving","status":"publish","type":"post","link":"http:\/\/10.10.2.102\/creevacom\/index.php\/2008\/01\/18\/cheap-and-dirty-windows-log-file-archiving\/","title":{"rendered":"Cheap and Dirty Windows Log File Archiving"},"content":{"rendered":"

\"\"<\/p>\n

So your company has a requirement to maintain log files for a year?<\/p>\n

You don’t know how to go about it and you need to implement it now?<\/p>\n

I have a solution for you and best of all it’s free. This solution however is not supported by me, there will be no bug fixes by me, and any damage you cause to your own servers is your own fault. That is my one-sentence disclaimer to tell you that you truly are on your own.<\/p>\n

For this solution or temporary fix (depending on your organization) you are going to need the following helper programs:<\/p>\n

Info-zip<\/a> – we’ll use this to compress down files and save space specifically we need the zip.exe file<\/p>\n

MD5SUMS<\/a> – this allows us to generate MD5 checksums to verify if any file tampering has taken place after the fact. Specifically, we need the md5sums.exe file.<\/p>\n

Dump Event Log (Dumpel.exe)<\/a> – this is a tool offered by Microsoft to dump your event logs into a text file. Though this link is part of the Windows 2000 Resource Kit I have tested and it does work with the Windows XP and Windows Server 2003 log files.<\/p>\n

We take these 3 programs and wrap them to work together via a batch file. All 3 of these programs MUST be in the same directory as the batch file for it to work as designed. Here is the batch file:<\/p>\n

@echo off
\nREM Sets date variables for file name
\nfor \/F “tokens=1,2” %%d in (‘date \/T’) do set day=%%d & set date=%%e
\nset yyyy=%DATE:~6,4%
\nset dd=%DATE:~3,2%
\nset mm=%DATE:~0,2%
\nset startDate=%yyyy%-%mm%-%dd%
\nREM Adds Computer to prefix the date
\nset outputname=%computername%-%startdate%
\nREM Cleans out previous zip files from a bad run
\ndel %outputname%.zip
\nREM Dumps each of the log files going back for 2 days
\nREM allowing for overlaps we may miss due to time changes
\ndumpel -f %outputname%.sec -l security -d 2
\ndumpel -f %outputname%.app -l application -d 2
\ndumpel -f %outputname%.sys -l system -d 2
\nREM creates an MD5 hash for verification checking
\nmd5sums %outputname%.sec >%outputname%.md5
\nmd5sums %outputname%.app >>%outputname%.md5
\nmd5sums %outputname%.sys >>%outputname%.md5
\nREM Compresses the 4 files
\nzip %outputname%.zip %outputname%.*
\nREM Cleans up the unneeded files to save
\ndel %outputname%.sec
\ndel %outputname%.app
\ndel %outputname%.sys
\ndel %outputname%.md5<\/p><\/blockquote>\n

I’ve included my comments in the batch file – but let’s go through it a section at a time so you can fully understand it.<\/p>\n

@echo off<\/p><\/blockquote>\n

If you don’t know @echo off suppresses everything from your screen in a batch, I wouldn’t suggest modifying my script since this is batch file programming 101.<\/p>\n

for \/F “tokens=1,2” %%d in (‘date \/T’) do set day=%%d & set date=%%e
\nset yyyy=%DATE:~6,4%
\nset dd=%DATE:~3,2%
\nset mm=%DATE:~0,2%
\nset startDate=%yyyy%-%mm%-%dd%
\nREM Adds Computer to prefix the date
\nset outputname=%computername%-%startdate%<\/p><\/blockquote>\n

This section adds the prefix to the files we are going to be using on all of your files – these allow us to work with files that include the computer’s name you are running this on and the date on which it was run.<\/p>\n

del %outputname%.zip<\/p><\/blockquote>\n

This verification actually cleans up the zip file if this script has already been run during the current day. Mine is modified to delete the zip file completely since at the end of my script I move my files to a remote location and don’t need archived logs filling up my hard drive quickly.<\/p>\n

dumpel -f %outputname%.sec -l security -d 2
\ndumpel -f %outputname%.app -l application -d 2
\ndumpel -f %outputname%.sys -l system -d 2<\/p><\/blockquote>\n

This area does the physical dumping of the log file. Dumpel is the command. The -f switch allows us to specify a file name. If you notice I used the %outputname% as the first part of the file with the file type of which log file it is as the suffix. The -l switch lets us specify which logfile we are dumping from the event log (security, application, or system). the -d switch allows us to specify how many days we wish to save. I chose 2 days to allow some overlap on the log files which is good for security reasons since we shouldn’t miss any events if you change the time of the day the script is run. It also gives us two more logfiles to verify the authenticity of the log data we are looking at.<\/p>\n

When this section is done running you should have three files. If your computer’s name was BOB-SERVER and the date you ran this was on January 3, 2007, the file names would read like this; BOB-SERVER-2007-01-03.sec for your security log, BOB-SERVER-2007-01-03.app for your application log, and BOB-SERVER-2007-01-03.sys for your system log.<\/p>\n

md5sums %outputname%.sec >%outputname%.md5
\nmd5sums %outputname%.app >>%outputname%.md5
\nmd5sums %outputname%.sys >>%outputname%.md5<\/p><\/blockquote>\n

This section generates an MD5 Hash<\/a> of the logfile data allows you to see if the data was tampered with when it was originally generated. It is next to impossible to edit a file and maintain the same hash data. This allows you some security that your log files are authentic. For those wondering “well can’t I just rerun the md5 program and generate a new hash and save that after modification?” – I have your answer. I didn’t include how to store these files after they are generated and we will touch upon that question under “What do you do now?” at the bottom. The command outputs your three BOB-SERVER-2007-01-03 files and outputs it to a single BOB-SERVER-2007-01-03.md5 file that includes a section with each of the above files. I decided personally that I didn’t need an md5 file for each of them – feel free to modify this if your needs differ.<\/p>\n

zip %outputname%.zip %outputname%.*<\/p><\/blockquote>\n

This compresses the dumped events logs down to a manageable size. I managed to get a 60 MB log file that I generated during varying testing phases down to just over 6 MB. I also managed to get 480 KB of log files down to 14kb. At this point, you should have a BOB-SERVER-2007-01-03.zip file which includes your three event logs and your md5 file.<\/p>\n

del %outputname%.sec
\ndel %outputname%.app
\ndel %outputname%.sys
\ndel %outputname%.md5<\/p><\/blockquote>\n

This section cleans up the files outside of the zip. I manage to get these files down 90% in size I don’t need these to eat up extra space.<\/p>\n

What do you do now?<\/p>\n

From here I would add a line at the end to move the logs to another server where you can store them for the length your organization deems necessary. To help combat the MD5 re-engineering I mentioned above I would copy the compressed archived to two locations on your network. This will help make having an MD5 meaningful. Another option is adding a script that e-mails you the MD5 hash so you have it saved for reference. Having the MD5 and collecting 2 days of information from the logs would mean an attacker may have to edit 2-4 archives and regenerate MD5s for them – double that if you store a second set of archives in another location.<\/p>\n

While this may fulfill your needs for log file capturing and an easy way to store them, it does not address the fact of easy log file auditing and tracking down events. There are all-in-one solutions out there for you to use and I don’t say in any terms this is a solution to those. You need to address your own needs and decide what works for you. This is to give you some time until you decide what you are going to do.<\/p>\n","protected":false},"excerpt":{"rendered":"

So your company has a requirement to maintain log files for a year? You don’t know how to go about it and you need to implement it now? I have a solution for you and …<\/p>\n","protected":false},"author":1,"featured_media":97594,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/posts\/2380"}],"collection":[{"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/comments?post=2380"}],"version-history":[{"count":0,"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/posts\/2380\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/media\/97594"}],"wp:attachment":[{"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/media?parent=2380"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/categories?post=2380"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/10.10.2.102\/creevacom\/index.php\/wp-json\/wp\/v2\/tags?post=2380"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}