PERL
Apache Log Parser
The goal of this script was to make a simple way to parse Apache HTTP server logs. The script is capable of understanding both the common log and the combined log formats. The program will display on screen instructions to guide the user through the process. The script is capable of pulling out four important pieces of information:
- List IP addresses of connections to HTTP server
- Statistics on number of HTTP requests
- IP addresses of successful connections to HTTP server
- List of not found requests and IP addresses of users who made them
(Sample apache log file borrowed from honeynet.org)
USAGE: perl apache.pl
< Download Source | View Source >
Config File Manager
In an enterprise environment there are many routers and switches each with a configuration. When you modify configurations you are left with a long history of configurations that must be managed. This script aims to help an administrator keep track of router and switch configuration files. One down side to keeping all configuration files is space. The more files you save the more space it will take up. This script also takes that issue into account. The script will keep track of configuration files stored in a special directory. The directory will have no subdirectories, just files. The files in the directory will each be compressed, when they are logged in and decompressed when they are checked out. There must be the ability to keep multiple versions of configuration files. When a configuration file is added to the collection, or checked back in, its name has the current date and time appended to it:
Main-router.2006.02.06.17.12.03
This will indicate that the version was created on February 6, 2006 at 5:12:03pm. Naming the files this way will make finding the most recent version easier. The complete functionality of the script is as follows:
- Add a new config file to the directory, compressing it on the way in. This is only allowed if there are no other versions of this file already present. At this time, the user will be prompted for a 1-line description of the purpose of the config file, which will be stored in the compressed directory with it.
- List all config files. This will list just the file names, not the version number and not all versions. For the above example, the listing will be Main-router. Next to each name will be the 1-line purpose entered when it was added.
- List all versions of a config file that is in the directory.
- Delete an existing config file from the directory. This deletes all versions of the config file and the 1-line purpose. There is no way to delete just one version.
- Retrieve an existing config file. This creates a copy of the (uncompressed) file in the user's directory. By default, this retrieves the latest version, but your system should allow the user to specify which version he/she wishes to receive.
- Check out an existing config file. This is like retrieve, but a record of who has checked the file out is kept. Also, only the latest version can be checked out. Nobody is allowed to delete, check out, retrieve, or check in this file until the person who checked it out checks it back in. Checking a file out implies an intent to change it and all of the above operations are denied until the new version in checked in.
- Check in a new version of an existing config file. The new version is compressed and is named for the current date and time.
USAGE: perl configManager.pl
< Download Source | View Source >
Integrity Checker
A key component to a security and intrusion detection system is data integrity. One approach to data integrity is to check key files for consistency with their known contents. This can be done by storing a digest of each file's hash (MD5, CRC32, SHA1, etc) and comparing it to the digest of the actual file to that of the stored value. If there is a discrepancy, report it so it may be looked in to by administrators.
This script will check all files in any set of directories against a record of their digests (digest = MD5 checksum). The digests themselves will be kept in a directory owned and managed by your script. The script will take the inifilepath as an argument. The INI file defines important values for the script to function. The format of the INI file is as follows:
DigestDir=path-to-directory-storing-digests
TimeOut=30:00:00
Directory=/home/HOMEUSERSDIRECTORY
Directory=/usr/include
The DigestDir is the absolute path of a directory in which digests are stored. There is to be one digest file in this directory for each Directory entry that follows. The TimeOut is used to warn the systems administrator when the digests are getting old. This is an amount of time after which the digests should be built. It is expressed as DDD:HH:MM where DDD is the number of days, HH is a number of hours, and MM is a number of minutes (seconds not included). E.G. A TimeOut of 23:12:30 means that digests should be rebuild after 23 days, 12 hours, and 30 minutes. Each Directory entry contains the absolute path of a directory hierarchy to be checked. All files in this directory and its subdirectories recursively should be summarized in a digest and checked whenever the script is run.
The digest of a directory hierarchy is a file containing the time that it was created and a listing of each file found below that directory and its MD5 checksum. The name of each directory to be checked will be the absolute pathname where all '/'s are replaced with '_'s. If any of the digest directories do not exist the script will make them on the fly. If any files that are needed do not exist an error message will be displayed and the script will terminate if it is unable to continue.
Timestamps: The script will maintain a timestamp for each directory that it is watching. This is simply the time in seconds since the start of the epoch. The timestamp will be stored in the digest file. When the script starts processing a directory, the script will check its timestamp and if it is out of date a warning will be displayed that the digest should be rebuilt. A timestamp is out of date if it is older than the amount of time specified in the INI file.
INVOCATION:
perl integrity.pl /home/.digests/integrity.ini
The script will read the INI file and determine where digests are kept for each target directory. It will then check all files stored in the target directories recursively against their digests in the digest directory. It will alert the administrator with warnings for the following:
- A new file that is in the actual directory but was not there when the digest was built
- A missing file; i.e. one that was in the directory when the digest was built but not there now
- A digest mismatch; i.e. a file whose current MD5 checksum differs from that stored in the digest
perl integrity.pl -r /home/.digests/integrity.ini
This will invoke the script to rebuild the digests for each directory. No checking is to be done when the digests are being rebuilt.
perl integrity.pl -v /home/digests/integrity.ini
This will invoke the script to run in verbose mode. The script will output a lot of information about what it is doing and when.
USAGE: perl integrity.pl [-r] [-v] inifilepath
< Download Source | View Source >
Disk Cataloger
A common problem systems administrators face is knowing where disk space is being used. It is useful to know for a subtree of the directory hierarchy, how much storage that subtree occupies on disk and what percentage of that total each subdirectory occupies. This information can make managing file space much more effective. This script will take one parameter: A root directory to search. The script will then recursively search through all files beginning at that root directory and gather information about disk usage. For the root directory, the script will computer the total number of KB stored underneath that directory. The script will use a menu interface to allow the administrator to navigate the tree and see statistics for each subtree. As the user enters a directory the statistics will be printed for that directory. The statistics collected and displayed are as follows:
- The total KB stored in that directory and all subdirectories (recursively)
- The subtotal KB stored under each immediate subdirectory
- The percentage that each immediate subdirectory represents of the total for that directory
USAGE: perl dirsize.pl RootDirectoryPath
< Download Source | View Source >
Account Maker
One common task a systems administrator is faced with is account management. One aspect of account management is the addition of new accounts on a system. To accomplish this, the administrator must place the new account in the proper group and ensure that there are no duplicate accounts on the system. This script will make the addition of new accounts easier. The script is used by system administrators at a university where there will be a large number of new accounts that need to be created each year. It will prompt the administrator for the full name of the user (e.g. John H Doe), the student's SSN (e.g. 111-11-1111), and the group the new account belongs to (e.g. student, faculty, staff, adjunct). Based on this information the script will formulate and execute a correctly formatted useradd command.
- Create username in the format of ABC1234 where ABC is the user's initials and 1234 is the last four digits of the SSN for that user. This is for students and adjuncts.
- Create username in the format of ABC where ABC is the user's initials for faculty.
- Create username using the user's first name for staff.
- Set path for students to /home/student/USERNAME
- Set path for faculty to /home/faculty/USERNAME
- Set path for staff to /home/staff/USERNAME
- Set path for adjuncts to /home/adjunct/USERNAME
- Ensure there will be no duplicate usernames
USAGE: perl addusers.pl
< Download Source | View Source >