Pavleck.Net

Monitoring, Scripting, and other Technologies

Archive for the 'Command Shell' Category


System Center VUG today & The new Google

Posted by Jeremy D. Pavleck on 21st November 2008

Scott Moss shot me an IM a little bit ago to remind me about the System Center Virtual User Group that’s happening today (November 21st, 2008) at 1:00 PM Central time. Here’s the ClickToAttend.Com Link to the event - feel free to add this to your Google Calendar.

Speaking of Google!

Google is slowly rolling out new features for a select group of users. It’s a wiki-fied version of search with promote/demote buttons and a comment area. This will help you pick the more authorative links among a list - unless someone has hired an army to promote a malware site, which I wouldn’t put past the modern web.

Anyway, here’s a screen shot of what the new interface looks like, and a short video afterwords.

The new wiki-like features of Google search

The new wiki-like features of Google search

And a screencast!

Posted in Command Shell, Community, WMI | 3 Comments »

Creating an audible alert notification in SCOM

Posted by Jeremy D. Pavleck on 12th September 2008

Edit: 09/13/2008 - On the advice of Pete Zerger, updated script to include a throttling mechanism to prevent an overload if an alert storm occurs. Also changed things around to make it a more generic ‘run remote executable’ instead of run remote sound.

A question was recently asked on the MOM Mailing List over at myITforum.com.

That question was, quite to the point:
How to create a audible alert? I like to create one for the critical alerts..

I’ve been working earlier with a script that would go out and disable the run time tracing, stop it, then delete the log files. So I already had knew what would work - a simple PowerShell script that uses WMI’s process create method on a remote machine.

A caveat lector before I continue; while this solution will technically work, I haven’t tested it formally. Additionally, you’ll need to contend with permission issues that arise as well. If you’re running the OpsMgr services under a named account, you’ll need to give that same account local administrator access on whichever machine you plan to run this call against. If you’re using ‘Local System’ you’ll have to either add the RMS\Local System account to the remote machine’s admin group or embed credentials inside the WMI call((Be careful when doing this. I haven’t included directions for that because it’s just a nightmare waiting to happen. I can give you a jumping off point though.))

First, the script. It’s small and basic. It wants to know the machine you want to run the command on, the command, and because this is a a sound player, the path to the WAV file. It then creates the process via WMI, and decodes the return code. If it’s 0, everything is fine. If it’s anything else, the process creation failed and it writes an event to the Operations Manager event log, which you can create an additional rule to look for.

Download SCOM-RunRemoteExecutable.ps1

# ==============================================================================================
#
# Microsoft PowerShell Source File — Created with SAPIEN Technologies PrimalScript 2007
#
# NAME: SCOM-RunRemoteExecutable.ps1
#
# AUTHOR: Jeremy D. Pavleck , Pavleck.NET
# DATE  : 9/13/2008
#
# COMMENT: This is a Proof Of Concept script written in response to a mailing list request to
#               enable OpsMgr to sound an audible alert on a remote admin PC, such as a console in a
#               NOC.
#               This is to be run as a Notification Command Channel.
#
# NOTES/WARNING: This script uses a remote WMI call to spawn a process on a named server. As
#               such, there are security issues to keep in mind. I haven’t added the code to allow you
#               to use alternate credentials, but use http://poshcode.org/501 as a jumping off point.
#               If you’re running the OpsMgr services under a domain account, add that user to the local
#               administrators group on the machine you want to run this command. If you’re using local
#               system, add RMS\Local System to the admin group.
#               RUN AT YOUR OWN RISK!
#
# VERSIONS:
#               v1.0 - 09/12/2008 - Initial version
#               v1.1 - 09/13/2008 - On the advice of Pete Zerger, added a throttling routine to prevent
#                                                       to many executions during an alert storm
#               v1.2 - 09/13/2008 - Changed variable names to make it a more ‘run remote executable’ script
#
# ==============================================================================================

# User Settings
$remoteMachine = "adminconsole.pavleck.net"
# The location to the executable. This is the path on the REMOTE machine.
$myExe = "C:\Program Files\Real Alternative\Media Player Classic\mplayerc.exe"
$myExeParams = "C:\Windows\Media\tada.exe" # Paramters to pass to the executable, such as the location of the sound file, etc.
                                                                                   # Leave blank if none are needed
# Registry & throttling settings
$myKey = "SCOM_PowerShell_Scripts" # Reg key name to use
$myValueName = "LastRunTime" # Data value
$interval = 5 # Wait at least this long, in minutes
# Initialize a couple things
$firstRun = $False
$throttle = $False

# We use the OpsMgr API only because it’s a very quick and simple way to log to the eventviewer
$momAPI = New-Object -comObject "MOM.ScriptAPI"
# LogScriptEvent Severities
$momErr = 1
$momWarn = 2
$momInfo = 4
# Setup some event ids to use
$errID = 11000
$warnID = 11001
$infoID = 11002

$myName = $MyInvocation.MyCommand.Name # Grab script name

### Registry throttling settings
# First see if our key exists, if not, create it and populate it with the current date/time
# and set $firstRun to $True
If(!$(Test-Path HKLM:\SOFTWARE\$myKey)) {
        New-Item -Path HKLM:\Software\$myKey
        New-ItemProperty -Path HKLM:\SOFTWARE\$myKey -Name $myValueName -Value (Get-Date)
        $firstRun = $True
        }

# If this isn’t the first run, compare previous time with current time - if last run is $interval
# minutes ago or higher, carry on, otherwise exit
If(!($firstRun)) {
        $lastRun = (Get-ItemProperty -Path HKLM:\SOFTWARE\$myKey).$myValueName
        Set-ItemProperty -Path HKLM:\SOFTWARE\$myKey -Name $myValueName -Value (Get-Date)
                If(((Get-Date) - [DateTime]$lastRun).TotalMinutes -ge $interval) {
                $throttle = $False
                } else {
                $throttle = $True
                }
        }

# Function DecipherRetCode accepts an integer, and returns the failure assigned to that code.
# This only returns the most common failures, such as permissioning and the like
Function DecipherRetCode([int]$retCode) {
        switch ($retCode) {
                0 {return "Success"}
                2 {return "Access Denied"}
                3 {return "Insufficient Privilege"}
                8 {return "Unknown Failure"}
                21 {return "Invalid Parameter"}
                default {return "$($retCode) is uncommon, and will need to be researched manually. "}
        }
}

# This line is actually the entire script.
If($throttle) {
        # Throttling - cancel response
        $momAPI.LogScriptEvent($myName, $warnID, $momWarn, "Notification Workflow requested that $($myname) run, but last response ran less then $($interval) minutes ago. Exiting.")
        $momAPI = $null
        exit
} else {
$retCode = ([WMICLASS]"\\$remoteMachine\root\cimv2:win32_process").Create("$myExe $myExeParams")
        # If $retCode = 0 ($false) exit the If, anything else is $True, and will log it
        If($retCode) {
                $momAPI.LogScriptEvent($myName, $errID, $momErr, "Error creating process. Error Code: $($retcode) Error Message: $(DecipherRetCode $retCode)")
        }
$momAPI = $null
exit
}

To implement this, open the Operations Console and go to Administration > Settings > Notification
Click on the Command tab, then click on add. Fill it out as you normally would:

Then click on OK, and you’ll see it with the rest of your commands:

Now to finish it up you’ll need to create a new notification recipient. Right-click on Notifications and select new recipient.

Make the display something to designate that it runs a command, I used “Sound Audible Alert”. And because the NOC isn’t manned 24/7, I limited the notification time to weekdays from 8am to 6pm. You can also adjust this from the devices tab, but I’m not going to include an emailing or other devices, so I prefer to set it in the general tab, this way it’s obvious even with a casual glance what the settings are.

After that, click on the “Notification Devices” tab, then click “Add”.
In the resulting popup, select our new notification command and enter anything for the delivery address - I used NA, because for this particular command we don’t require any additional information - but OpsMgr still needs something in there. Hit next, keep the schedule at always unless you’re adding additional channels, next again, name the device - I used “Send Audible Alert”

Click OK, and your set. Treat it like any other notification recipient - either create a new rule just for this, or edit an existing subscription and add our new recipient to it.

As you can see, using PowerShell inside of Operations Manager makes it very flexible and powerful. We can run all manner of things in response to alerts; From running a simple sound file all the way up to initiating disaster recovery scenarios and intense system diagnostics - both things which I’ll be showing you later on as we explore the Notification Command Channel together.

Posted in Command Shell, Notifications, SCOM | No Comments »

SCOM Snippet: The hidden nag mode

Posted by Jeremy D. Pavleck on 10th September 2008

When it comes to notifications, we have many options - except one that people have asked about, a nag mode. Something that will re-send an email after a certain amount of time to make sure it’s taken care of.

Well, it does exist in OpsMgr.

Either intentionally or unintentionally as a bug, if you call the Update method on an alert without changing any criteria, the notification bound to the alert will re-fire. This will happen whether you add a comment with the update (Update(”Updating the alert”)) or not (Update(”")).

To enable this secret nag-mode, it’s as simple as writing a Powershell script that runs every X hours. In that script you’ll just need to do a Get-Alert with the criteria you’re looking for - in the example I’m just going to have it return all alerts older then 4 hours, and update them.

It’s very simple though - how simple? Like this:

$oldAlerts = Get-Alert | Where-Object {($_.LastModified -ge [DateTime]::Now.AddHours(-4)) -and ($_.ResolutionState -eq 0)}
ForEach($alert in $oldAlerts) {
$alert.Update("")
}

You can expand this as much as you’d like. Match against NetBiosComputerName to only nag for those critical core servers, match it against the monitoring object to ensure critical monitors are being addressed. Multiple management groups? Match against that. You see where I’m going with this. In fact, you can find out everything you can match against by just running Get-Alert | select -first 1 - there’s all the fields available.

Posted in Command Shell, Powershell, SCOM, SCOM Snippets | 1 Comment »

Operations Manager: From Start to Finish

Posted by Jeremy D. Pavleck on 5th September 2008

I’ve never had a ‘proper’ test/development environment for System Center products. I’ve used both client systems and VMs I’d spin up through VirtualBox. That will be changing.

I placed an order a few days ago for a new server - featuring a 2.8Ghz Quad core Xeon and 12GB of ram, it will begin a brand new environment - and I’ll be screencasting all of the most relevant parts of it.

We’ll cover sizing, install, deployment, security - and move on to extending OpsMgr by utilizing the SDK service, designing custom reports - and a lot more.

In the mean time, while I painfully await the arrival of a shiny box from DHL, I’m working on the framework for 2 new side projects - one of which I think you’ll all be quite happy with; the much awaited SCOPE - System Center OpsMgr Powershell Extentions - a collaborative operation between me, Marco Shaw, Cameron Fuller, Pete Zerger and - you, possibly. We’re at the very early stages of SCOPE, and could definitely use people now and down the road - especially C# programmers and those familiar with creating PowerShell snapins. If you’d like to help, send me an email (jeremy@pavleck.net) and let me know what you can do.

Posted in Command Shell, Community, SCOM | 2 Comments »

Setting agent failover servers & Switching SNMP device proxies

Posted by Jeremy D. Pavleck on 7th July 2008

By default, you can’t really specify a failover management server in OpsMgr. Why? Not really sure, though I think it’s a ploy to ensure you setup the OpsMgr Active Directory Integration, which will handle this for you.

No fret though, we can still do it - it’ll just take a little bit of actual effort.

First, we need to define our Primary and Failover management servers. This isn’t something you can just progmatically grab, so you’ll need to know the name yourself.

In my $PROFILE, I’ve set them to be defined to 2 variables with the following:

# Set Primary Management Server
$Primary_MS = Get-ManagementServer | ? {$_.Name -like "SERVERNAME*"}
# Set Failover Management Server
$Failover_MS = Get-ManagementServer | ? {$_.Name -like "BACKUPSERVER*"}

Now that we have that set, it’s simple to do the rest. First, lets grab all of the servers that don’t have a failover management server set.

$noFailoverSpecified = Get-Agent | ? {(!$_.GetFailoverManagementServers())}

What the above does is call the GetFailoverManagementServers() method on each agent. If they have a failover, it will return data and thus $True. If there aren’t any failovers, it will return nothing - which is the same as $False. So we look for all the ones that don’t return anything.

If you’re curious, you can see just how many servers are missing failovers with

$noFailoverSpecified.Count

- in my case it was 63.

Now, we just run a quick snippet that adds the failover server to the agent:

ForEach ($agent in $noFailoverSpecified) {
Set-ManagementServer -PrimaryManagementServer $Primary_MS -AgentManagedComputer $agent -FailoverServer $Failover_MS | Out-Null
}

That will crunch away as it’s doing it’s thing, we’re redirecting output to $null so we don’t have to see agents scrolling over and over. When it returns you to a prompt, you’re done. If you’d like to verify that you did indeed set all of the agents to have a failover, we can check real quick:

If ((Get-Agent |? {(!$_.GetFailoverManagementServers())}).Count -eq $null)
{
Write-Host "Every agent has a failover server, great job!" -ForeGroundColor Green
} else {
Write-Host "Looks like we missed some, try again!" -ForeGroundColor Magenta
}

And that’s that. All of your agents have a primary and failover server.

Screen shot of SCOM Command Shell showing steps to setup failover agents

But wait, you have a lot of remotely managed devices too? Monitoring SNMP on a bunch of different servers - what happens for that?

Well, we can’t setup a failover (From what I’ve seen, if I’m wrong please let me know) agent. But we can proactively write a script that will change the proxy agent on the devices, and run it as needed.

This was written in a response to this query on the newsgroups, and is only a cursory look into it. There may be other ways of doing this - and I’d love to hear it. As it stands, I’m not sure how to set them back to a management server as the monitor.

Firstly, we’ll have to pick an agent managed computer to use as the new proxy agent. You can’t use a management server for this, because they aren’t “Agent Managed” and you can’t use Set-ManagementServer because the devices aren’t “Remote Managed Computers”.  I have a seperate agent-managed server on my network I call “Timex” because it acts like a watcher node. So I’ll go ahead and use him.

$proxyAgent = Get-Agent |? {$_.Name -eq "timex.pavleck.net"}

Then gather a list of our current remotely managed devices

$remDevices = Get-RemotelyManagedDevice

Now just loop through it, setting the device to use the proxy agent we just instantiated:

ForEach($device in $remDevices) {
Set-ProxyAgent -ProxyAgent $proxyAgent -Device $device | Out-Null
}

That will loop through things changing the proxy server that it uses. When it’s done, we can verify it by running:

Get-RemotelyManagedDevice |? {$_.ProxyAgentPrincipalName -ne $proxyAgent.Name}

If it outputs nothing, then they’ve all been changed. Simple as that!

SCOM: Setting the proxy agent for a device via command shell

Posted in Command Shell, Powershell, SCOM, SCOM Snippets, SNMP | 3 Comments »