string logic – Alan’s Notes

Powershell – love it / hate it

Sometimes it’s hard for me to wrap my head around things.

Powershell makes so many things easier than before it existed. At least for me. I’m not a programmer but simple commands piped one to another, like bash in Linux, I can get a lot done.

One of the “things” I need to get done is checking how many computers got a program installed. Because of the environment I’m in and the program itself, there’s no GPO based install for an MSI and there’s no third party tool based install. This stumped me for a while until I came up with the idea of using a startup script for the install.

Another challenge, powershell scripting is disabled. However I learned from “PowerShell Security” by Michael Pietroforte and Wolfgang Sommergut that powershell can be called within a .bat using Powershell Invoke-Command -ScriptBlock {} even if powershell is disabled by policy. So I wrote a start up script that relied on .bat files that had Powershell Invoke-Command -ScriptBlock {} in them to run the program install. The -ScriptBlock {} checked first if the dependencies were installed, installed them if not, then checked if the desired program was installed and installed it if not. It also wrote a log file for each pc named as “progname_<hostname>.txt” and appended to the file with each restart.

The startup script wasn’t running reliably every time a pc booted. Seemed to be NIC initialization or network initialization related. In any case, the pcs that were to be installed were listed in an AD group. The pcs that had run the startup script output that info into a file named “progname_<hostname>.txt”. One way to see which of the pcs had not gotten the install was by comparing the members of the AD group, the computer names, to the <hostname> portion of the log file names that were being created. Computers from the group without a corresponding file hadn’t gotten installed.

Easy, right? Get the list of computers to install with Get-ADGroupMember and compare that list to the <hostname> portion of the log files. How to get only the <hostname> portion? Get-ChildItem makes it easy to get the list of file names. But then need to parse it to get only the <hostname> part. Simple in a spreadsheet but I really wanted to get a listing of only the <hostname> without having to take any other steps.

I knew I needed to look at the Name portion of the file name, handle it as a string, chop off the “progname_”, and drop the “.txt” portion. But how to do that? After what seemed like way to much searching and experimenting I finally came up with…

$( Foreach ( $name in $(Get-ChildItem progname* -Name) ) { $name.split('_')[1].split('.')[0] } ) | Sort

The first .split('_')[1] lops off the common part of the filename that’s there only to identify the program the log is being created for, “progname_”, and keeps the rest for the second split(). The next split(), .split('.')[0], cuts off the file extension, .txt, and keeps only the part that precedes it. And so the output is only the hostname portion of the filename that the startup script has created.

Compare that list to the list from Get-ADGroupMember and, voila, I know which of the targeted pcs have and have not had the program installed without doing any extra processing to trim the file names. Simple enough, but for some reason it took me a while to see how to handle the file names as strings and parse them the way I needed.

Get-WinEvent, read carefully to filter by date

Get-WinEvent hashtable date filtering is different.

Widows event logs have lots of useful information. Getting it can be a slow process. Microsoft even says so in a number of posts and recommends using a hashtable to speed up filtering.

Many powershell Get-… commands include a method to limit the objects collected. A -Filter, -Include, or -Exclude parameter may be available to do this. They are generally implemented along the lines of <Get-command> -Filter/-Include/-Exclude 'Noun <comparison_operator> <value>'. Objects with <DateTime> type attributes like files, directories, users, services, and many more can all be filtered relative to some fixed <DateTime> value like yesterday, noon three days ago, etc. As a result the answer to the question, show every user who has logged in since yesterday afternoon, can be known.

In the case of the Get-WinEvent cmdlet none of these parameters are available. However the cmdlet’s output can be piped to a Where-Object and the event’s TimeCreated can be filtered relative to another time. In that way filtering is similar to how it works for other cmdlets that include a -Filter parameter.

All that goes to say I’d become very complacent about how to filter <DateTime> in powershell.

Now I needed to filter events in the log and, as claimed in many Microsoft posts, log filtering can be slooow. The posts also say filtering speed can be increased significantly by using a hashtable for filtering. And wouldn’t you know it, Get-WinEvent has a -FilterHashtable parameter. Great! Let’s use that to speed up my slow log filtering.

Well, guess what? Unlike any other <DateTime> filtering I’ve done there is no way to filter for StartTime or EndTime being greater than or less than some other time. And the fact that the hashtable key names StartTime and EndTime were being used instead of TimeCreated should have been my first clue that I wasn’t doing the usual filtering on TimeCreated.

The only option in a hashtable is to assign some value to a key name. So how to filter for, say, events that happened yesterday? There is no one <DateTime> value that represents all of yesterday. <DateTime> isn’t a range or array of values it is a fixed point in time value.

And for me this is where things get strange with Get-WinEvent. It can be used to extract events from a log, and those events can be piped to Where-Object and TimeCreated can be filtered by comparing to a <DateTime> just like using the -Filter parameter of other Get-… cmdlets.

After posting about this on PowerShell Forums, I found out I misunderstood the use of StartTime and EndTime in a -FilterHashtable used with Get-WinEvent. The post is, Hashtable comparison operator, less than, greater than, etc?.

Turns out whatever <DateTime> StartTime is set to in a hashtable filters for events that occurred on or after the time it is set to. And EndTime, in a hashtable, filters for events that occurred at or before the <DateTime> it has been set to!

As an example, I extract events from the Application log that occurred 24 hours ago or less. This is run on a test system that doesn’t get used often so there’s not many events in that time span.

The first example does not use the StartTime key in the hashtable. It pipes Get-WinEvent to a Where-Object and filters for TimeCreated being on or after one day ago.

The second example includes the StartTime key in the hashtable and sets it to one day ago.

Both return the same results, but there is no comparison operator used for the StartTime key in the hashtable. The hash table’s assigned StartTime value is used internally by Get-WinEvent to compare each event’s TimeCreated against the assigned StartTime value and check that it is on or after StartTime. Similarly, when EndTime is assigned a value, each event’s TimeCreated is evaluated if it is on or before the assigned value. I really feel that could have been made much clearer in the Get-WinEvent documentation.

Below, the hashtable does not include StartTime or EndTime. Where-Object filters against TimeCreated.

$FilterHashtable = @{ LogName = 'Application'
   ID = 301, 302, 304, 308, 101, 103, 108 
}
Get-WinEvent -FilterHashtable $FilterHashtable | 
   Where-Object { $_.TimeCreated -ge (Get-Date).Date.AddDays(-1) } | 
   Format-Table -AutoSize -Wrap TimeCreated, Id, TaskDisplayName

TimeCreated          Id TaskDisplayName
-----------          -- ---------------
3/1/2022 5:54:54 PM 302 Logging/Recovery
3/1/2022 5:54:54 PM 301 Logging/Recovery
3/1/2022 5:39:01 PM 302 Logging/Recovery
3/1/2022 5:39:01 PM 301 Logging/Recovery
3/1/2022 5:34:59 PM 103 General
3/1/2022 5:34:39 PM 103 General

Below, the hashtable includes StartTime. No Where-Object to filter on TimeCreated.

$FilterHashtable = @{ LogName = 'Application'
   ID = 301, 302, 304, 308, 101, 103, 108
   StartTime = (Get-Date).Date.AddDays(-1) 
}
Get-WinEvent -FilterHashtable $FilterHashtable | 
   Format-Table -AutoSize -Wrap TimeCreated, Id, TaskDisplayName

TimeCreated          Id TaskDisplayName
-----------          -- ---------------
3/1/2022 5:54:54 PM 302 Logging/Recovery
3/1/2022 5:54:54 PM 301 Logging/Recovery
3/1/2022 5:39:01 PM 302 Logging/Recovery
3/1/2022 5:39:01 PM 301 Logging/Recovery
3/1/2022 5:34:59 PM 103 General
3/1/2022 5:34:39 PM 103 General

regex – headaches and solutions

Regex could save your bacon. Try it.

Need to parse a string that doesn’t have a consistent layout but does contain regular patterns that need to be keyed on to extract information (or change it)? Want to extract values from a string without complex nested IN(…, MID(…, SUBSTR(…, LEN(… functions?

Regex seems to be the tool to use, if you can wrap your mind around the syntax for patterns that need to be keyed on. There’s plenty of examples to be found that illustrate searching for phone numbers, social security numbers, ZIP codes, ZIP+4, IPv4 addresses and so on. Add in features like look ahead, look behind, greedy or non greedy searching and I get the feeling there’s more than enough material to develop an entire graduate level course to formulate and understand regex expressions.

My own forays into regular expressions have focused mostly on mundane simple patterns. I did however find a couple uses for the look behind to extract a substring from an irregular length string. And the substring to extract was not fixed length. To me the function based way of parsing the string would have been much more complex than working out the regex syntax.

In one case there was a substring for the size of a hard drive. That substring varied in length depending on the size of the drive. In another case strings for monitor make and model, which could each be different lengths because more than one manufacturer was in the dataset and model names varied.

In both cases the target string was preceded by a fixed string that could be used to locate the sought for value within the larger string. And the position of the target string within the larger string varied because the details before it changed from pc to pc. Rescued by the regex look behind!

I posted about it on LinkedIn if you’re interested to look. Samanage and Powershell – two tools to produce helpful reports.

The significant portions of the statements are below. They allowed looking into the string for a pattern, `Fixed hard disk media”~size: “` or `name: “` or `manufacturer: “`, and then extracting the portion of the string that immediately followed.

$regexdrv = '(?<=Fixed hard disk media"~size: ")([^"]+)'
Select-String -Pattern $regexdrv | %{$_.Matches}
$regexdisp = '(?<=name: ")([^"]+)|(?<=manufacturer: ")([^"]+)'
Select-String -Pattern $regexdisp -AllMatches | %{$_.Matches}

If you find yourself struggling to parse out some information from a string, take a look at regex. You could find the solution to your problem.