Monitoring using log files

Log files can be useful in catching project issues before they cause cluster issues. Logs are also useful when issues arise and need to be investigated. To get the most out of the log files it is crucial to ensure that all of the required log files are available. On very busy clusters, this may require regular log file archival, sometimes as frequently as every two hours. As the cluster load increases, check the log file overwrite time interval and setup an archive process to ensure that no log files are lost. Log files do not need to be stored for a long period of time, but will need to be available when an issue is reported. Logs should be kept for the largest interval between when an issue is reported and the investigation of that issue. A week's worth of log files is typically sufficient.

The Log DSC is very useful when investigating log files as it reduces the parsing requirements and allows the investigator to use SQL queries for investigation. The Log DSC can be used with DM Query to manually view logs, or it can be used programmatically to automatically monitor log files. For more information, see Log DSC and DM Query.

The UNICOM Intelligence Interviewer - Server Admin Manage Logs feature is often used to gather logs. The feature zips logs in server specific sub-folders and is useful in structuring log files for investigation. The feature provides options for querying log files across all servers or focusing on a particular server. Select a directory rather than a specific file and set the MR Init Custom setting to SearchSubFolders on the connection string to view logs across all folders.

It is a good practice to filter the log set when the set is large. For example, use the following to filter logs by type:

The log set can also be filtered by time if the original create and modified times are available in the log files. If the original create and modified times in the log files are available, the following will also filter by time: