使用kfsmd监控文件系统的变化

http://www.linux.com/feature/124903

 

Applications can ask the Linux kernel to report changes to selected files and directories. I created the Kernel Filesystem Monitoring Daemon (kfsmd) to make monitoring filesystem changes simple.

There are packages available for both 32- and 64-bit Fedora 7 and 8 and Ubuntu 7.10 Gutsy, as well as 32-bit packages for openSUSE 10.3. You can also download a repo file, which can be used with Fedora 8 and yum. Placing the repo file into /etc/yum.repos.d allows you to install kfsmd and its dependencies with yum install kfsmd on a Fedora 8 machine. You can also compile directly from source if that is your preference.

Command-line clients for kfsmd come in two categories: monitoring and logging. The monitoring client produces output on the console whenever something happens to a filesystem you are watching. You can log to either a Berkeley DB4 file or a PostgreSQL database.

The following session shows a simple directory monitoring session using kfsmd. It creates and populates a temporary directory, then starts kfsmd-cat to watch /tmp/k for any filesystem changes. The main command-line parameter is the watch command, which takes the directory or file to watch as a single argument. While kfsmd-cat was running, I opened a second terminal and created the df5.txt file and then removed it. These actions were reported to the console by kfsmd.

What's that command again?

Although some of the command names shown in this article are quite long, the commands themselves are simple to use. In the article text I shorten command names from "kernel-filesystem-monitor-daemon-postgresql" to simply kfsmd-postgresql, but I show the full command names in code blocks. The command-line clients all have long names in order to be as descriptive as possible, and because common usage of kfsmd is generally through scripts and not directly on the command line.

$ mkdir /tmp/k $ cd /tmp/k $ date > df1.txt $ date > df2.txt $ kernel-filesystem-monitor-daemon-cat -v watch . setting up watch for:. setting up watches calling run event on wd:1 . filename:df5.txt CLOSE URL:./df5.txt event on wd:1 . filename:df5.txt DELETE_FILE URL:./df5.txt

If you specify a directory to monitor with a full filesystem path, then kfsmd also monitors existing and newly created subdirectories by default. You can use the ignorepfx argument to limit these recursive monitors by explicitly telling kfsmd not to monitor some subdirectories. In the next example, which uses ignorepfx, I created two subdirectories inside /tmp/k: junk1 and subdir1. Both of the directory create and delete events were reported by kfsmd, and because of the ignorepfx argument, kfsmd did not monitor the /tmp/k/junk1 subdirectory itself, so files I created in that directory were not monitored and reported by kfsmd. Note that as ignorepfx is the prefix of a path, using just "junk" means that the subdirectory junk1 is not monitored.

$ kernel-filesystem-monitor-daemon-cat -v \ watch /tmp/k ignorepfx /tmp/k/junk event on wd:1 /tmp/k filename:junk1 CREATE URL:/tmp/k/junk1 event on wd:1 /tmp/k filename:subdir1 CREATE URL:/tmp/k/subdir1 should adding monitor for:subdir1 event on wd:2 /tmp/k/subdir1 filename:subfileA.txt CREATE URL:/tmp/k/subdir1/subfileA.txt event on wd:2 /tmp/k/subdir1 filename:subfileA.txt CLOSE URL:/tmp/k/subdir1/subfileA.txt event on wd:2 /tmp/k/subdir1 filename:subfileA.txt DELETE URL:/tmp/k/subdir1/subfileA.txt event on wd:1 /tmp/k filename:subdir1 DELETE URL:/tmp/k/subdir1 event on wd:1 /tmp/k filename:junk1 DELETE URL:/tmp/k/junk1

You can see that filesystem changes reported by kfsmd have a regular style. The primary report has the prefix "EVENT_TYPE URL:" where the event type is what happened to the file and the URL: string is used as a direct prefix to the file path being reported. This structure makes it convenient to use the kfsmd-cat command and pipe the output into a script that will perform some action for you automatically when files change.

The following script uses Perl to print the paths of files that are deleted in any monitored directory. It uses the pwd command on the first line of the command to make the paths reported by kfsmd absolute. The kfsmd-cat command will produce output similar to that shown above, and Perl code massages the output into a particular format, or can execute a command whenever a deletion happens. The script ignores all lines that do not start with DELET. Lines which report file deletion then have the prefix string "anything...URL:" stripped off so only the file path is printed to the console.

$ kernel-filesystem-monitor-daemon-cat watch `pwd` \ | perl -ne '{ if( /^DELET/ ) { s/.*URL://g; print; } }' /tmp/k/df5.txt

A second invocation of kfsmd-cat, shown below, sends an email message whenever a file is deleted in the current directory.

$ kernel-filesystem-monitor-daemon-cat watch `pwd` \ | kfsmd-sendemail.pl

The Perl script kfsmd-sendemail.pl is shown below. The three lines which you might have to change to use this yourself are listed at the top of the script; the FromAddress and ToAddress should be modified to suit your local environment.

#!/usr/bin/perl -n $Mailer = "| /usr/sbin/sendmail -t"; $FromAddress = 'ben@localhost'; $ToAddress = 'ben@localhost'; if( /^DELETE_/ ) { s/.*URL://g; chomp; $url=$_; $now=`date`; open MAIL,"$Mailer"; print MAIL <<THE_EMAIL; From: $FromAddress To: $ToAddress Subject: KFSMD: A file was deleted The file: $url Was deleted at $now THE_EMAIL close MAIL; }

Logging with kfsmd

To log filesystem events into a PostgreSQL database, use the kfsmd-postgresql command. Before using this command you must set up the database with the postgresql-schema.sql script. This can be done using the first command shown below. Note that the the database setup command only needs to be run once. The kfsmd-postgresql daemon will run in the background by default. You can set up watches with it in the same manner as for the kfsmd-cat command, though you must specify the database host and database name.

$ cat postgresql-schema.sql | psql -h myPostgreSQLServer $ kernel-filesystem-monitor-daemon-postgresql \ -h myPostgreSQLServer \ -d kernel_filesystem_monitor_daemon_postgresql watch `pwd`

With the command above, filesystem changes are logged to the PostgreSQL database called kernel_filesystem_monitor_daemon_postgresql on the server myPostgreSQLServer. You can query the database using SQL as shown below.

$ psql -h myPostgreSQLServer # \c kernel_filesystem_monitor_daemon_postgresql # select * from dirs d, events e where d.pwd = e.pwd order by time; pwd | url | id | mask | pwd | time | name -----+--------+----+------+-----+----------------------------+---------- 1 | /tmp/k | 2 | 512 | 1 | 2008-01-03 13:26:39.913456 | df11.txt 1 | /tmp/k | 1 | 8 | 1 | 2008-01-03 13:26:39.913456 | df11.txt 1 | /tmp/k | 3 | 8 | 1 | 2008-01-03 13:26:41.917512 | df21.txt

There are many situations where kfsmd is the right tool for the job. For example, if you are editing a file and wish to automatically publish it to a remote server each time it is saved, you can use kfsmd-cat and when a CLOSED event is detected execute a little script to rsync the file to the server. If you have a long-running task and wish to know when it is completed, just monitor for a filesystem change that occurs at the end of the process, such as when a file download or a build completes.

The kfsmd-postgresql and kfsmd-stldb4 commands allow you to easily record filesystem changes in a database, which is great for auditing what happened and when.