Month of PowerShell - Embracing the Pipeline

#monthofpowershell

Understanding how the PowerShell pipeline works is essential to making PowerShell work for you. The pipeline is syntactically similar to CMD, Bash, and other shells, but works very differently in PowerShell. Let's take a look.

First, let's run a standard PowerShell command to interrogate Windows services. From your PowerShell session, run [code]Get-Service[/code]:

PS C:\Users\Sec504> Get-Service

Status   Name               DisplayName
------   ----               -----------
Running  AarSvc_1ebce       AarSvc_1ebce
Stopped  AJRouter           AllJoyn Router Service
Stopped  ALG                Application Layer Gateway Service
Stopped  AppIDSvc           Application Identity
Running  Appinfo            Application Information
Stopped  AppMgmt            Application Management
...

The [code]Get-Service[/code] cmdlet allow us to interrogate the services running on the Windows system. There are a lot of services though, so it can be difficult to catch all of the output.

Let's look at building a simple pipeline. Press the up arrow to recall the [code]Get-Service[/code] command. At the end of the line add a space followed by [code]| more[/code], then press Enter. Scroll through the output by pressing the spacebar multiple times until you return to the prompt.

PS C:\Users\Sec504> Get-Service | more

Status   Name               DisplayName
------   ----               -----------
Running  AarSvc_1ebce       AarSvc_1ebce
Stopped  AJRouter           AllJoyn Router Service
Stopped  ALG                Application Layer Gateway Service
Stopped  AppIDSvc           Application Identity
Running  Appinfo            Application Information
Stopped  AppMgmt            Application Management
...
Running  wscsvc             Security Center
Running  WSearch            Windows Search
Running  wuauserv           Windows Update
Stopped  WwanSvc            WWAN AutoConfig
Stopped  XblAuthManager     Xbox Live Auth Manager
Stopped  XblGameSave        Xbox Live Game Save
Stopped  XboxGipSvc         Xbox Accessory Management Service
Stopped  XboxNetApiSvc      Xbox Live Networking Service
PS C:\Users\Sec504>

In the previous command you created a pipeline – the [code]Get-Service[/code] cmdlet sent data through the pipe [code]|[/code] to the [code]more[/code] command. The [code]more[/code] command shows the output one screenful at a time, allowing us to display the output of [code]Get-Service[/code] without scrolling.

The pipeline is valuable for more than just [code]Get-Service[/code]. We can create pipelines to process the output of other commands as well. Create a pipeline to sort the output of [code]Get-Process[/code] by name. Type [code]Get-Process | Sort-Object -Property Name[/code], then press Enter.

PS C:\Users\Sec504> Get-Process | Sort-Object -Property Name

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    102       7     6232      10820              5260   0 conhost
    242      14     5516      32376       2.11   2548   1 conhost
    635      48    25180      67152       0.47   6660   1 Cortana
    378      21     1840       5276               528   1 csrss
    554      21     1816       5384               424   0 csrss
    415      16     3880      20188       0.38   3720   1 ctfmon
    127       7     1240       5796              3244   0 dasHost
    264      14     3920      13984              2120   0 dllhost
...

In this output we see that [code]Sort-Object[/code] has sorted the output by the [code]Name[/code] property. You can also sort by process ID using the [code]Id[/code] property. Press the up arrow to recall the previous command. Retrieve the process information, sorted by process ID.

PS C:\Users\Sec504> Get-Process | Sort-Object -Property Id

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
      0       0       60          8                 0   0 Idle
   3964       0      192        152                 4   0 System
    225      12     2288       9944                60   0 svchost
      0       9     4852      77120                92   0 Registry
    236      17     3656      12232       0.03    180   1 dllhost
     53       3     1056       1196               320   0 smss
    250      10     2048      12064               380   0 svchost
    561      21     1816       5392               424   0 csrss
    114       8     1428       5748               428   0 svchost
...

PowerShell has another really useful cmdlet: [code]Export-Csv[/code]. It takes the output from the previous command, and converts the data into CSV format, writing it out to a file. Use it now in the pipeline, converting the output of [code]Get-Process[/code]into a CSV file named [code]processes.csv[/code]:

PS C:\Users\Sec504> Get-Process | Export-Csv -Path processes.csv
PS C:\Users\Sec504>

Fantastic! Now, take a look at the contents of [code]processes.csv[/code]using [code]Get-Content[/code]:

PS C:\Users\Sec504> Get-Content .\processes.csv
#TYPE System.Diagnostics.Process
"Name","SI","Handles","VM","WS","PM","NPM","Path","Company","CPU","FileVersion","ProductVersion","Description","Product","__NounName","BasePriority","ExitCode","HasExited","ExitTime","Handle","SafeHandle","HandleCount","Id","MachineName","MainWindowHandle","MainWindowTitle","MainModule","MaxWorkingSet","MinWorkingSet","Modules","NonpagedSystemMemorySize","NonpagedSystemMemorySize64","PagedMemorySize","PagedMemorySize64","PagedSystemMemorySize","PagedSystemMemorySize64","PeakPagedMemorySize","PeakPagedMemorySize64","PeakWorkingSet","PeakWorkingSet64","PeakVirtualMemorySize","PeakVirtualMemorySize64","PriorityBoostEnabled","PriorityClass","PrivateMemorySize","PrivateMemorySize64","PrivilegedProcessorTime","ProcessName","ProcessorAffinity","Responding","SessionId","StartInfo","StartTime","SynchronizingObject","Threads","TotalProcessorTime","UserProcessorTime","VirtualMemorySize","VirtualMemorySize64","EnableRaisingEvents","StandardInput","StandardOutput","StandardError","WorkingSet","WorkingSet64","Site","Container"
"conhost","1","242","2203505201152","33071104","5648384","14400","C:\WINDOWS\system32\conhost.exe","Microsoft Corporation","2.71875","10.0.19041.1 (WinBuild.160101.0800)","10.0.19041.1","Console Window Host","Microsoft? Windows? Operating System","Process","8",,"False",,"2856","Microsoft.Win32.SafeHandles.SafeProcessHandle","242","2548",".","0","","System.Diagnostics.ProcessModule (conhost.exe)","1413120","204800","System.Diagnostics.ProcessModuleCollection","14400","14400","5648384","5648384","276024","276024","5939200","5939200","33181696","33181696","190709760","2203508932608","True","Normal","5648384","5648384","00:00:02.4531250","conhost","3","True","1","System.Diagnostics.ProcessStartInfo","6/30/2022 7:43:03 PM",,"System.Diagnostics.ProcessThreadCollection","00:00:02.7187500","00:00:00.2656250","186978304","2203505201152","False",,,,"33071104","33071104",,
...

This is a lot more information than what we saw when we last ran [code]Get-Process[/code]. Earlier we saw 8 columns of information but the CSV file has over 60 columns!

This highlights an important concept about the PowerShell pipeline:

This is an important concept to understand about PowerShell. When you run a PowerShell command and send the output in a pipeline, you are sending an object or a collection of objects to the next step in the pipeline. These objects often include more information than what you see in the default output.

Using PowerShell, we can retrieve the information that is important to us using different commands. Let's try it now. Type [code]Get-Process -Name lsass | Select-Object -Property *[/code], then press Enter.

PS C:\Users\Sec504> Get-Process -Name lsass | Select-Object -Property *


Name                       : lsass
Id                         : 672
PriorityClass              :
FileVersion                :
HandleCount                : 1147
WorkingSet                 : 18718720
PagedMemorySize            : 5947392
PrivateMemorySize          : 5947392
VirtualMemorySize          : 99164160
TotalProcessorTime         :
SI                         : 0
Handles                    : 1147
VM                         : 2203417387008
WS                         : 18718720
PM                         : 5947392
NPM                        : 25248
Path                       :
Company                    :
CPU                        :
ProductVersion             :
Description                :
Product                    :
__NounName                 : Process
BasePriority               : 9
ExitCode                   :
HasExited                  :
ExitTime                   :
Handle                     :
SafeHandle                 :
MachineName                : .
MainWindowHandle           : 0
MainWindowTitle            :
MainModule                 :
MaxWorkingSet              :
MinWorkingSet              :
Modules                    :
NonpagedSystemMemorySize   : 25248
NonpagedSystemMemorySize64 : 25248
PagedMemorySize64          : 5947392
PagedSystemMemorySize      : 149616
PagedSystemMemorySize64    : 149616
PeakPagedMemorySize        : 6225920
PeakPagedMemorySize64      : 6225920
PeakWorkingSet             : 18927616
PeakWorkingSet64           : 18927616
PeakVirtualMemorySize      : 100229120
PeakVirtualMemorySize64    : 2203418451968
PriorityBoostEnabled       :
PrivateMemorySize64        : 5947392
PrivilegedProcessorTime    :
ProcessName                : lsass
ProcessorAffinity          :
Responding                 : True
SessionId                  : 0
StartInfo                  : System.Diagnostics.ProcessStartInfo
StartTime                  :
SynchronizingObject        :
Threads                    : {696, 712, 716, 728...}
UserProcessorTime          :
VirtualMemorySize64        : 2203417387008
EnableRaisingEvents        : False
StandardInput              :
StandardOutput             :
StandardError              :
WorkingSet64               : 18718720
Site                       :
Container                  :

In this command we used [code]Get-Process[/code] to get information about the LSASS process, adding a new command to the pipeline: [code]Select-Object[/code]. [code]Select-Object[/code] is used to select objects or the properties of objects in the pipeline.

When we use [code]Select-Object -Property *[/code] we see a lot of extra properties that we didn't see when we ran [code]Get-Process[/code] by itself. This is the PowerShell pipeline in action: the object sent by [code]Get-Process[/code] has all of this data; we decide what to do with it using commands on the right of the pipeline.

Since PowerShell is designed around the concept of using a pipeline, there are lots of cmdlets that allow us to leverage this functionality. We can take the output of [code]Get-Process[/code] and export the results to an HTML report using [code]ConvertTo-HTML[/code], for example. Run [code]Get-Process | Select-Object -Property Name, Id, Path, CPU, WorkingSet64 | ConvertTo-Html | Out-File processes.html[/code] to create an HTML report of all running processes, saving the output as [code]processes.html[/code]:

PS C:\Users\Sec504> Get-Process | Select-Object -Property Name, Id, Path, CPU, WorkingSet64 | ConvertTo-Html | Out-File processes.html
PS C:\Users\Sec504>

Open the HTML report using your default browser by running [code]Start-Process processes.html[/code]:

PS C:\Users\Sec504> Start-Process processes.html
PS C:\Users\Sec504>

This report won't win any awards for beautiful style, but it gets the job done! Go ahead and close the report when you are done taking a look.

Once you understand the concepts around PowerShell pipelines, it can be applied to lots of different functionality. For example, we can get basic information about the Event Log service by running [code]Get-Service -Name eventlog[/code]:

PS C:\Users\Sec504> Get-Service -Name eventlog

Status   Name               DisplayName
------   ----               -----------
Running  eventlog           Windows Event Log

Press the up arrow to recall the previous command. Use [code]Select-Object[/code] in a pipeline to retrieve the following properties about the Event Log service: [code]Status[/code], [code]Name[/code], [code]DisplayName[/code], and [code]StartType[/code]:

PS C:\Users\Sec504> Get-Service -Name eventlog | Select-Object -Property Status, Name, DisplayName, StartType

 Status Name     DisplayName       StartType
 ------ ----     -----------       ---------
Running eventlog Windows Event Log Automatic

Like we did with [code]Get-Process[/code], you added an additional field of information ([code]StartType[/code]) to the output. Run that command again, this time exporting the 4 properties into a CSV file named [code]processes2.csv[/code]:

PS C:\Users\Sec504> Get-Service -Name eventlog | Select-Object -Property Status, Name, DisplayName, StartType | Export-Csv -Path processes2.csv
PS C:\Users\Sec504>

Now, [code]processes2.csv[/code]will only have the four selected properties listed, since you filtered the [code]Get-Service[/code]output with [code]Select-Objet[/code]. Take a look with [code]Get-Content processes2.csv[/code]:

 PS C:\Users\Sec504> Get-Content .\processes2.csv
#TYPE Selected.System.ServiceProcess.ServiceController
"Status","Name","DisplayName","StartType"
"Running","eventlog","Windows Event Log","Automatic"

So far we've started out pipeline examples with [code]Get-Process[/code] and [code]Get-Service[/code], but that doesn't have to be the start of the pipeline. Examine the status of the WinRM service: type [code]'winrm' | Get-Service[/code] then press Enter:

PS C:\Users\Sec504> 'winrm' | Get-Service

Status   Name               DisplayName
------   ----               -----------
Stopped  winrm              Windows Remote Management (WS-Manag...

In the previous command we created a string object [code]'winrm'[/code] as the input to the [code]Get-Service[/code] cmdlet. Using a pipeline in this manner is known as parameter binding, where a PowerShell command matches the input you supply in the pipeline to a designated parameter. [code]'winrm'[/code] is a string, but in the pipeline it becomes an object. Type [code]'winrm' | Get-Member[/code] to see all of the string properties.

PS C:\Users\Sec504> 'winrm' | Get-Member


   TypeName: System.String

Name             MemberType            Definition
----             ----------            ----------
Clone            Method                System.Object Clone(), System.Object ICloneable.C...
CompareTo        Method                int CompareTo(System.Object value), int CompareTo...
Contains         Method                bool Contains(string value)
CopyTo           Method                void CopyTo(int sourceIndex, char[] destination, ...
EndsWith         Method                bool EndsWith(string value), bool EndsWith(string...
Equals           Method                bool Equals(System.Object obj), bool Equals(strin...
...
Trim             Method                string Trim(Params char[] trimChars), string Trim()
TrimEnd          Method                string TrimEnd(Params char[] trimChars)
TrimStart        Method                string TrimStart(Params char[] trimChars)
Chars            ParameterizedProperty char Chars(int index) {get;}
Length           Property              int Length {get;}

Most of these elements are methods that perform an action on the string. This starts to exceed the focus of this article, but the important point is this:

One of the most useful elements in PowerShell parameter binding and the pipeline is integrating content from files. Run the [code]Set-Content[/code]command shown below to create a file called [code]services.txt[/code]with four service names (separated by [code][/code][code]n [code][/code]for new line characters):

PS C:\Users\Sec504> Set-Content -Path services.txt -Value "wuauserv`nw32time`nBITS`nAppidsvc"

We can quickly interrogate all of the services by leveraging PowerShell parameter binding in the pipeline. Use [code]Get-Content[/code]to retrieve the contents of the file, then use the pipeline to send the output to [code]Get-Service[/code]:

PS C:\Users\Sec504> Get-Content -Path .\services.txt | Get-Service

Status   Name               DisplayName
------   ----               -----------
Running  wuauserv           Windows Update
Running  w32time            Windows Time
Running  BITS               Background Intelligent Transfer Ser...
Stopped  Appidsvc           Application Identity

Here we can integrate a separate data source (a file with multiple service names, one per line) into PowerShell to query them all using the pipeline. Neat!

Let's clean up some of the temporary files using PowerShell and the pipeline:

PS C:\Users\Sec504> "processes.csv" , "processes.html", "processes2.csv", "services.txt" | foreach { Remove-Item -Path $_ }
PS C:\Users\Sec504>

This example is a little more complicated than absolutely necessary to delete four files, but it demonstrates another example of the pipeline: sending a list of objects (strings for four files) to a [code]foreach[/code]loop, referencing the current file name with [code]$_[/code]in the loop, and deleting the file with [code]Remove-Item[/code].

Summary

In this article we looked at several examples of working with the pipeline as a mechanism to leverage a big feature in PowerShell, that all pipeline elements are objects not just text. We saw this firsthand by looking at the output of [code]Get-Service[/code], and the output of [code]Get-Service | Export-Csv[/code]. Once we recognize the properties (data elements) available to use through the PowerShell pipeline, we can access new data resources and even embrace useful functionality like parameter binding to use text file lists for task automation.

I hope you followed along with this article and found this useful! Thanks for making it all the way to the end!

-Joshua Wright

Return to Getting Started With PowerShell

Joshua Wright is the author of SANS SEC504: Hacker Tools, Techniques, and Incident Handling, a faculty fellow for the SANS Institute, and a senior technical director at Counter Hack.