Parsing Unix log files

Feb 12, 2013 at 7:57 PM
Edited Feb 12, 2013 at 7:59 PM
Using Powershell to parse Unix log files for text/errors.
  1. Import the SSH-Sessions module and SSH into the Unix host
    a. You'll need the SSH-Sessions folder in your default Powershell modules folder.
    b. Usually C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\
    c. Follow this guide: How to connect to SSH from Powershell
    d. This script depends on that module to work. I suggest you follow the guide, and once you can successfully connect to SSH sessions from Powershell, then resume here at this point.
  2. Execute a Unix shell script to parse the log, and return the desired result
  3. Evaluate the data/text that the Unix script returns
  4. Assemble the final results and either display them in Powershell or pass them to Polymon
Here's the powershell that will be in the Powershell monitor in Polymon:
###########"Display Data" function#####################
function display_data{ 
if ($errlvl -gt 3){$errlvl = 3}     
    switch($errlvl){
        3   {if ($verbose -gt 0){$final_displaytext = "Monitor failure: "+($monitor_alarm_text| select -uniq)+" "+($monitor_info_text| select -uniq);break} 
                else {$final_displaytext = "Monitor failure: "+($monitor_alarm_text| select -uniq);break}}
        2   {if ($verbose -gt 0){$final_displaytext = "Monitor warning: "+($monitor_alarm_text| select -uniq)+" "+($monitor_info_text| select -uniq);break} 
                else {$final_displaytext = "Monitor warning: "+($monitor_alarm_text| select -uniq);break}}
        1   {if ($verbose -gt 0){$final_displaytext = "Monitor OK: "+($monitor_info_text);break}
                else {$final_displaytext = "Monitor OK";break}}
        default {$final_displaytext = "FAIL: Invalid script result: "+($monitor_info_text| select -uniq);break}}
    $displaymessage = ("   "+$final_displaytext) -replace "   ,", ""
    Switch($polymon)
      {     1{$Status.StatusText=$displaymessage
              $Status.StatusID=$errlvl
              if (($counter.length) -gt 1)
                {$counter = $counter | select -uniq |sort-object
                 foreach ($count in $counter){$Counters.Add($count[0],$count[1])  }}}
            0{switch($errlvl){
                3{write-host  "Err LVL = FAIL: "$displaymessage}
                2{write-host  "Err LVL = WARN: "$displaymessage}
                1{write-host  "Err LVL = OK: "$displaymessage}}} }
    Remove-SshSession -ComputerName $HostName
    break}      
    
##########Script Settings######################
    $HostName = "XXXXXX" ; #                            Enter the Host Name                                        ****************Host name redacted.  ****************
    $Integchk_log_instances =@("prd")  #             Enter the instance name
    $unix_script = /home/user1/scripts/integchk_log_review_.sh ";#      What Unix script should be run?              ****************Script details redacted.  ****************
    $verbose = 1;#                              What level of messages/counters do you wish to display?
    
    
    
#####################################Script Contents Below######################################
$debug=0
if($Counters){[DOUBLE]$polymon = 1;$Status.StatusText = ""} else {[DOUBLE]$polymon = 0}
[DOUBLE]$ERRLVL = 1;$monitor_info_text= @();$monitor_alarm_text = @();$Singular_list = @() 
$FIELDS = @()
Import-Module SSH-Sessions
$newsession = New-SshSession -ComputerName $HostName  2>&1 #    *******************connection details redacted  See SSH-Sessions.psm for username/pw/key authentication options*******************
If ($newsession -notlike 'Successfully connected*' -and $newsession -notlike 'You are already*')  {[DOUBLE]$ERRLVL += 2 ; $monitor_alarm_text = "New SSH Session Failed ("+$hostname+")" ;  display_data}




############Run the Monitor script##############
Foreach($inst in $Integchk_log_instances)
        {
             
$SSH_command = $unix_script+" "+$inst
if ($debug -gt 0){write-host $SSH_command }
$Integchk_Log_Parse_Result = Invoke-SshCommand -ComputerName $HostName -Command $SSH_command 2>&1
If ($Integchk_Log_Parse_Result -notlike '*No integrity errors were detected*') {[DOUBLE]$ERRLVL += 2 ; $monitor_alarm_text += ", "+$inst+": "+$Integchk_Log_Parse_Result}#;  display_data}


########Parse out the various fields and assign error level value###############
        $FIELDS = $Integchk_Log_Parse_Result -split "&&"
        $Timestamp = $FIELDS[0]
        $Integchk_summary = $FIELDS[1] 
            if ($debug -gt 0){write-host 'timestamp field: ($timestamp)'+$Timestamp;write-host 'Summary field: ($Integchk_summary)'+$Integchk_summary}
        $monitor_info_text+= ", "+$inst+":("+$Timestamp+")"
            if ($verbose -gt 1){$monitor_info_text+= ""+$Integchk_summary}
        if (($Timestamp -like '*2013*') -and ($Integchk_summary -eq "No integrity errors were detected." ))
            {if ($debug -gt 0 ) {write-host '($monitor_info_text): '$monitor_info_text}   }
        else {
           if ($Timestamp -like '*2013*')
            {[DOUBLE]$ERRLVL += 2;
                    if ($debug -gt 0 ) {write-host "($monitor_info_text): "+$monitor_info_text}           
                    }
         else            
            {[DOUBLE]$ERRLVL += 2;$monitor_alarm_text += ", "+$inst+": "+"No Valid Result from Script run "  ;$Counter = ,@(('No Log File Found'), 1)}}
            
        }
        display_data        
Here's the Unix shell script. I have mine saved as /home/user1/scripts/integchk_log_review_.sh Remember to the set the proper path to this file in the Powershell script: ($unix_script = /home/user1/scripts/integchk_log_review_.sh ) Remember that whatever user ID you use to open the SSH session, that ID has to have read and execute rights on this Unix file. I use Notepad++ to edit my Unix shell scripts, via WinSCP. Remember that, whatever editor you use, that the Unix shell script has to have Unix style End-Of-Lines. In Notepad++ do "Edit : EOL Conversion : Unix Format"
date '+%m %d %y' |  
{  
read MONTH DAY YEAR 
typeset -Z2 DAY=`expr "$DAY" - 1`  
case "$DAY" in  
        00)  
          typeset -Z2 MONTH=`expr "$MONTH" - 1`  
                case "$MONTH" in  
                        00)  
                           MONTH=12  
                          typeset -Z2 YEAR=`expr "$YEAR" - 1`  
                        ;;  
                esac  
        DAY=`cal $MONTH $YEAR | grep . | fmt -1 | tail -1`  
esac  
}
OUTPUTLINE=`tail -2 /epic/$1/logs/integchk.log.$MONTH$DAY$YEAR | grep "error" |awk 'BEGIN { FS = ":" };{ print $1":"$2":"$3 " &&" $6}'`
if [ -z "$OUTPUTLINE" ] 
then
unset OUTPUTLINE
ERRORLINE=`tail -48 /epic/$1/logs/integchk.log.$MONTH$DAY$YEAR | grep "reported errors in the" |awk 'BEGIN { FS = ":" };{ print $1":"$2":"$3 " &&" $6}'|awk 'BEGIN { FS = "These" };{ print $1}'`
CORRECTIONLINE1=`tail -48 /epic/$1/logs/integchk.log.$MONTH$DAY$YEAR | grep "reported integrity errors" |awk 'BEGIN { FS = "reported" };{ print $1}'`
CORRECTIONLINE2=`tail -48 /epic/$1/logs/integchk.log.$MONTH$DAY$YEAR |grep errors |tr '\n' '\000'|tr '\r' '\000'|sed 's/Integrity check reported errors in the/\/epic\/'$1'\/bin\/integchk --db/g' | sed 's/ These globals should be rechecked to verify if the errors are accurate/1/g'|sed 's/database\.//g'|sed 's/1\.[^e]*Global/--global/g'|sed 's/reported integrity errors\.[^e]*Details follow\://g' |sed 's/[^g]*\/epic/\/epic/'`
OUTPUTLINE=$ERRORLINE$CORRECTIONLINE1' ***TO Recheck, run the following:*** '$CORRECTIONLINE2
fi
echo $OUTPUTLINE