Monday, February 21, 2011

Multiple Monitoring Performance Data Collections

Ah yes – that code from the last couple of posts has a bug…

The retrieval of data from SCOM has proven very useful in the last few days but it seems most commonly I’m getting multiple monitoring performance data collections - so the code I wrote in the last couple of posts on how to retrieve that data using F# and Sho needs a little adjusting to concatenate the collections into one sequence – yield! works great in F# eg

let perfData (start:DateTime) (finish:DateTime) : seq<DateTime * float> = 
seq {
for mpdsItem in mpds do
yield! (mpdsItem.GetValues(start, finish)
|> Seq.map (fun(mpdv) -> (mpdv.TimeAdded, mpdv.SampleValue))
|> Seq.filter ( fun(d,v) -> v.HasValue)
|> Seq.map ( fun(d,v) -> (d,box v) )
|> Seq.map ( fun(d,v) -> (d, unbox v) )
)
}




Not sure about python but it must be similar.

Monday, February 14, 2011

SCOM data into F#

Having just discovered how to get SCOM data from Microsoft Sho - a dynamic language analysis environment - I thought I’d try from F#. Pretty much the same but you need to think about what to do with the graphing, and you need to handle the Nullable<float> values coming back from SCOM.

I used fschart to do the plotting and a sequence filter and boxing/unboxing to handle the Nullable data.

The histogram function wouldn’t be hard to make but while stumbling across fschart I also stumbled across this.

So my contribution was do this:

#r @"System.Windows.Forms.DataVisualization.dll"
#r @"C:\extras\FSChart10\FSChart\bin\debug\FSChart.dll"

open FSChart

open System.IO
open System.Drawing
open System.Windows.Forms
open System.Windows.Forms.DataVisualization.Charting

#r "c:\Program Files\System Center Operations Manager 2007\SDK Binaries\Microsoft.EnterpriseManagement.OperationsManager.dll"
open Microsoft.EnterpriseManagement.Monitoring

let mg = Microsoft.EnterpriseManagement.ManagementGroup("someManagementServer")
let mpdc = MonitoringPerformanceDataCriteria("ObjectName = 'ASP.NET' and CounterName like 'Request Execution%' and MonitoringObjectPath like 'someMonitoringObjectPathFilter%'")
let mpds = mg.GetMonitoringPerformanceData(mpdc)

open System

let (perfData:seq<DateTime * float>) =
mpds.[0].GetValues(DateTime.Today.AddDays(-1.), DateTime.Today)
|> Seq.map (fun(mpdv) -> (mpdv.TimeAdded, mpdv.SampleValue))
|> Seq.filter ( fun(d,v) -> v.HasValue)
|> Seq.map ( fun(d,v) -> (d,box v) )
|> Seq.map ( fun(d,v) -> (d, unbox v) )


hist 0.0 200. 50 (perfData |> Seq.map (fun (d,v) -> v) )


The result looks like this.


image


So the question is… what was the more efficient approach? Well when I did the sho/python code I actually had some c# code open in the background to help me through the classes - without that it would have been really hard. The F# approach made me scratch my head a few times wondering how to deal with charting and nullables but all the way through (much like the c#) I had the advantage of the richer type information to help me figure out what to do. 

SCOM and Microsoft Sho–A better histogram example…

The last blog post had a naff histogram as a few exceptionally long queries squashed the remainder of the data in the first bin, also I wasn’t checking for None so here’s a better, filtered example.

ShoLoadAssembly("c:\Program Files\System Center Operations Manager 2007\SDK Binaries\Microsoft.EnterpriseManagement.OperationsManager.dll")
ShoLoadAssembly("c:\Program Files\System Center Operations Manager 2007\SDK Binaries\Microsoft.EnterpriseManagement.OperationsManager.dll")

from Microsoft.EnterpriseManagement.Monitoring import *
from System import *

mg = Microsoft.EnterpriseManagement.ManagementGroup("someManagementServer")

mpdc = MonitoringPerformanceDataCriteria("ObjectName = 'ASP.NET' and CounterName like 'Request Execution%' and MonitoringObjectPath like 'someMonitoringPathFilter%'")

mpds = mg.GetMonitoringPerformanceData(mpdc)
mpds.Count

hist([mpdv.SampleValue for mpdv in mpds[0].GetValues(DateTime.Today.AddDays(-1), DateTime.Today) if ((mpdv.SampleValue < 200) and (mpdv.SampleValue is not None))])



 



image

Friday, February 11, 2011

Retrieving Performance Data from System Center Operations Manager

System Center Operations Manager’s agent model means it can collect performance metrics from across a diverse technology environment. You use Rules to define the data collection and typically you use the Console to view the data in graphical format.

But, the Console by itself doesn’t allow any numerical analysis. If you want to do that you need to get the data out. You can copy to the clipboard within the Console Action menu but that’s very manual. A better approach is obviously programmatic access and this is  where things get interesting.

For some reason there’s little information on the net on how you get data out of SCOM. What is there usuall refers to querying the SCOM database directly. This doesn’t seem right to me. I’d far prefer an API that stood a chance of staying intact in future versions. So here’s what I’ve done.

Firstly, use the Powershell integration to SCOM to get the data. (This is easiest if you setup the Powershell ISE to connect to SCOM.) You can then use the get-performancecounter and get-performancecountervalue cmdlets to retrieve data, for example:

$starttime = [datetime]::today.adddays(-20)
$endtime = [datetime]::today.adddays(1)

get-performancecounter |
? {$_.ObjectName -eq 'Web Service' -and
$_.CounterName -eq 'Connection Attempts/sec' -and
$_.InstanceName -eq '_Total' -and
$_.MonitoringObjectPath -like 'usuallyAServerNameFilter*' } |
get-performancecountervalue -starttime $starttime -endtime $endtime |
select SampleValue, TimeSampled |
export-csv "c:\temp\ConnectionAttemptsPerSec.csv"


You can also use the .Net framework and languages to get the data – the documentation isn’t great but this example works (I’ve reduced the font size to try and make it fit in this blog template):


/// <summary> 
/// Gather performance data
/// </summary>
using System;
using System.Collections.ObjectModel;
using Microsoft.EnterpriseManagement;
using Microsoft.EnterpriseManagement.Common;
using Microsoft.EnterpriseManagement.Configuration;
using Microsoft.EnterpriseManagement.Monitoring;

namespace MySamples
{
class Program
{
static void Main(string[] args)
{
ManagementGroup mg = new ManagementGroup("someManagementServer");

MonitoringPerformanceDataCriteria mpdc =
new MonitoringPerformanceDataCriteria(
@"ObjectName = 'ASP.NET' and CounterName like 'Request Execution%' and MonitoringObjectPath like 'usuallyAServerNameFilter%'");
ReadOnlyCollection<MonitoringPerformanceData> mpds =
mg.GetMonitoringPerformanceData(mpdc);

if (mpds.Count > 0) {
ReadOnlyCollection<MonitoringPerformanceDataValue> mpdvs = mpds[0].GetValues(DateTime.Today.AddDays(-1), DateTime.Today);

foreach (MonitoringPerformanceDataValue mpdv in mpdvs) {
Console.WriteLine("TimeSampled = " + mpdv.TimeSampled +
", SampleValue = " + mpdv.SampleValue);
}
}
Console.ReadLine();
}
}
}


Now I typically take the data out to F# and/or Excel and graph/model it to help figure my way through a problem. However, Microsoft have just released a nicely packaged analysis tool in Microsoft Sho – basically IronPython wrapped up with some plotting/graphing/math libraries – much like many of the other Python packages out there.


I thought this would be a fun way to interactively deal with performance data collection and analysis so I tried the following as an experiment (put in appropriate management server/monitoring object filters for you – I just used ASP.NET request execution times from a test environment as in the programs above).  The hist command takes a list, or I guess anything it can enumerate over and get numbers from, and returns a histogram dialog and plot.


ShoLoadAssembly("c:\Program Files\System Center Operations Manager 2007\SDK Binaries\Microsoft.EnterpriseManagement.OperationsManager.dll")
ShoLoadAssembly("c:\Program Files\System Center Operations Manager 2007\SDK Binaries\Microsoft.EnterpriseManagement.OperationsManager.dll")

import Microsoft.EnterpriseManagement
import Microsoft.EnterpriseManagement.Configuration
from Microsoft.EnterpriseManagement.Monitoring import *
from System import *

mg = Microsoft.EnterpriseManagement.ManagementGroup("aManagementServer")

mpdc = MonitoringPerformanceDataCriteria("ObjectName = 'ASP.NET' and CounterName like 'Request Execution%' and MonitoringObjectPath like 'someMonitoringObjectFilter%'")

mpds = mg.GetMonitoringPerformanceData(mpdc)
mpds.Count

hist([mpdv.SampleValue for mpdv in mpds[0].GetValues(DateTime.Today.AddDays(-1), DateTime.Today)]
)



And here’s the output:


image


Cool huh!?

Wednesday, February 09, 2011

Engineering tricks with WolframAlpha

I’ve found Google great for simple problems like this:

20 pounds per cubic foot in kilograms per cubic meter= 320.36926 kilograms per cubic meter

But wolframalpha also lets you combine unit conversions with calculations:

(2 * 175 grams) per square meter * (2*3 meters squared) + ((20 pounds per cubic foot in kilograms per cubic meter) * 3 square meters * 5 millimeters)

Or just click on this link: http://www.wolframalpha.com/input/?i=(2+*+175+grams)+per+square+meter+*+(2*3+meters+squared)+%2B+((20+pounds+per+cubic+foot+in+kilograms+per+cubic+meter)+*+3+square+meters+*+5+millimeters)

What’s the above tell me? It’s the theoretical weight of the kayak I’m currently building – about 7kg. In reality when hand laying fibreglass you get a much higher weight of epoxy rather than the one to one ratio that’s possible under vacuum application and which I’m using above.

Hull fibreglassing

It’s taken awhile to get here, but the hull is now fibreglassed.

  • After struggling with the Resene colorwood stain affecting epoxy seal coats I found that a wash with isopropyl alcohol made for a nice clean bonding surface.
  • I rediscovered how to apply epoxy, everyone seems to have their own method but for me it’s plastic cards (old atm cards/loyalty cards etc are ideal) and short foam (usually yellow) rollers.
  • A hot air gun does absolute wonders at getting rid of air bubbles/foam.
  • I used high density 170g/sq m fibreglass that’s often used for model making. That equates to 5 ounce high density fibreglass which should be pretty strong (easy way to do the conversion… just use google – type “175 grams per square meter in ounces per square yard” in the search box).
  • Wet sanding within 48 hours is more effectively at smoothing the epoxy finish than waiting for it to set really hard.

second hull fill coat

Thursday, February 03, 2011

Hull ready to be fibreglassed

Finally completed staining/fairing/sanding the hull. Long process with lots of waiting for epoxy to dry. Here’s a photo of how the hull looks wetted down. The staining in combination with a kind of matching fairing compound does look patchy but I’ll roll on a very thin layer of epoxy to seal the wood before the fibreglass goes on and that will darken it down and make it look a lot more consistent.

WP_000022