ABSTRACT
Perl is a powerful text processing language that can be used to search file structures, identify data patterns of interest to the user/analyst, and extract associated records or strings for subsequent processing. Based largely on the C language, Perl runs on both Unix and Windows platforms. It is particularly useful in the Intelligence Community and Department of Defense, where analytical work often involves searching files for search patterns of interest. Perl scripts automate that search, writing records containing such terms, which are then imported into a database for subsequent analysis.
The current chapter illustrates how to write search patterns of various levels of complexity and includes a number of sample scripts to identify and capture records containing those data patterns. As an example of how Perl can be used to monitor log records, it describes how to set up scripts that run 24 × 7 searching new log files for records with search terms of interest, writing records with matches into a work directory. These scripts were used to capture and load Structured Query Language Server log records into the SQLSvrLogs database created in Chapter 7. Finally, Chapter 11 shows how to develop a web-based front end to this database for analysts to query the database tables to identify records for further analysis or action, updating tracking flags as appropriate.
