Enterprise Management

Oracle OpenWorld 2009

Monday Oct 19, 2009

I'm just back in the office from a 2 week long business trip that ended with a week at Oracle OpenWorld 2009.  This was an odd event for me due to my own misunderstanding of what the conference had grown into over the years.  I had the same experience 2 years ago at VMWorld.  You see these conferences are no longer about the company with their name in the title (although believe me they get something out of it).  These conferences have taken on a life of their own in order to attract the massive number of attendees that they do.  VMWorld has become the virtualization conference while Oracle OpenWorld has become the datacenter conference.  

There I was, in the middle of over 40,000 people, celebrating the datacenter.  Something I have spent a career at Sun trying to figure out.  Now I'm in marketing and before that professional services, sustaining, and support services...so I've pulled booth duty many, many times.  I will say that this is the first time I had a high quality of questions related to features.  In the recent past, I'd normally field the "how does this relate to Solaris feature X" or the "talk to me at the technology level regarding what the application is written in".  So the show was a very refreshing experience for me. 

If any of you stopped by the Sun booth and had a chance to spend some time with the live Ops Center 2.5 product demo, thanks!  If you have not had a chance to poke around, I've created 5 quick and dirty screen recordings without voice to give you a feel.

 1.) Oracle Enterprise Manager Integration:  This is a proof of concept integration that will likely go public within the next 4-8 months.  It requires no extra code delivery from Enterprise Manager or Ops Center and therefore uses both product's public integration APIs.  Basically the Enterprise Manager is calling JMX remoting APIs from Ops Center.  In the demonstration of this POC integration, we show a few powerful basic moves:

  • straight attributes from Ops Center showing up in Enterprise Manager
  • taking attributes from Ops Center and running them through logic or formula and showing the result in Enterprise Manager
  • Events or alarms from Ops Center showing in the Enterprise Manager alarm console
  • Enterprise Manager graphing Ops Center telemetry
  • Objects automatically appearing in Enterprise Manager when they are added to an Ops Center logical group
  • summation dashboards in Enterprise Manager built from a composite of information from Ops Center and Enterprise Manager on the same page

2.) Monitoring: There are some new graph views in Ops Center 2.5 related to virtualization (zones and LDOMs).  This is in addition to what was already there related to agentless hardware and power monitoring plus OS monitoring via a javaME agent.  Some fixes in the ssh and ipmi connections.

3.) Provisioning: OS Provisioning in Ops Center 2.5 cleaned up some service processor interaction, DHCP implementations, routing issues, cleaned up JET, fixed some scaling issues, and added the ability to bare metal provision Oracle Enterprise Linux. 

 4.) Patching: Software Life cycle management picked up a new distribution channel in this release: Oracle Enterprise Linux.  There is also an amazing integration built into the product with Microsoft's System Center Configuration Manager.  This allows you to patch Windows machines from the Ops Center user interface.  We also fixed scalability issues and patch handling bugs.  Ops Center is now patching Sun Cluster properly.  It officially supports Solaris Live Upgrade by allowing you to treat an alternate boot environment as if it was a physical box.  We also ship with default handling policies for Live Upgrade.  If you use Ops Center to migrate a zone, we ask the proper upgrade on attach question.  If you are using Solaris 10u8 and above, we leverage the parallel patching fixes.  We cleaned up some dependency mapping and handling bugs.

 5.) Virtualization Management: Ops Center 2.5 now supports zone and Logical Domain (LDOM) management.  Everything you would expect to be there is there...discovery/mapping, create/destroy, stop/start/suspend, migration, etc.  You are allowed to leverage Fiber Channel connected storage, local storage, and NFS for zone and LDOM guest housing.  Awareness is there too...you can see what resources are left, what are the constraint levels on existing guests, and historical and realtime utilization.  I will say that one amazing feature is virtual pools for LDOMs.  With LDOM virtual pools you can grab 10's of physical boxes in the user interface and create a pool.  You can then place a policy on the pool to either select the least loaded in terms of system load and guest number or a policy based on straight CPU and RAM utilization.  Now when you create an guest in the pool, Ops Center with automatically find the host that mets your policy.  Ops Center will also offer a calender to you to select a time at which it can auto-balance all your guests across the pool (via the use of warm LDOM migration).  That is truly a remarkable feature.

Hopefully you now have a feel for Ops Center 2.5.  Please visit us at sun.com/opscenter for more information and "contact me" forms.  Oracle OpenWorld 2009 definitely lived up to it's promise!

[0] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg

Motivation For Getting Healthy

Monday Oct 19, 2009

One part of the Sun blogging tradition is the "personal" entry.  Not too many companies that allow corporate blogging allow personal blog entries.  Blogs tend to sit on corporate infrastructure and most companies don't like mixing business with non business activities.  So I figure since I don't have too much time before the EU makes a decision, I'd get my one personal blog entry in there for the record. 

There is just something about October that makes me want to get back into my "physical health" as opposed to mental or professional.  It could be the colder air coming in, the leaves changing, the fact my birthday is in October, etc.  I will admit that I haven't done anything remotely related to my health since 1999.  My main issue with physical health is the fact there is no immediate utility from it.  Let me explain because I know that sounded awful.  I was considered "healthy" once in my life about 15 years ago.  Now the only reason I was in shape was due to a sport I was playing.  To be good at the sport, I had to be in shape.  There was an immediate correlation between an activity I performed and working out.  Now for me, sport or athleticism has always just games you played if you were not working on other things and not really a category of life.  No offense to professional athletes.  So to make a long story short, once the athleticism was removed I had a hard time working out for any reason.  I actually felt a bit vain doing so at times. 

I was reading in a yahoo headline story that stated looking at pictures of yourself  over the years can be a  powerful motivation tool.  And showing those pictures to others could help you stay on track (some accountability angle so the article said).  So I looked around and found 15/10/5/current pictures of myself. 

Here I am 15 years ago.  I rowed through high school and into college.  During my second year in college I injured my lower back to the point I had to stop the sport.  It took awhile to live with the pain...that is always having it but at times forgetting it's there.  I still have pain today.

15 years ago

Here I am about 10 years ago.  Early sign of the beer belly starting to show, but at this point in my life there was still some limited muscle showing through from the 5 years previous.  

10 years

Now we are up to 5 years ago.  This would be a decade since I did anything physical if you are still following along.  Things are starting to get ugly for me.  At this point, I'd have to really be trying hard to open a pickle jar (reference for all you Seinfeld fans out there) for you to see any muscle I had built 10 years previous.

5years

Nothing left now but the present.  Here I am recently (within the last year or so)  looking stunning. 

current

So there's my body changing over a 15 year period of time.  It's difficult to see (at least for me) but now maybe I'll be motivated to do something.  I think I'm at the point in my life where if I continue down the path I'm on I will have health issues to deal with in the not too distant future.  So it looks like doing nothing for 15 years helps you find that immediate utility I was missing in the past...day to day survival.

If I sound like I have the same mentality as you and you have had success in this area, please let me know what worked for you.  In the meantime, I'll start stretching ;)

[3] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg

I'm Back! Again....

Monday Sep 14, 2009

You can always tell when I'm totally buried in work and looking for things to aid in my procrastination.  I decided to update my blog (which I haven't touched in two years).  Where does the time go! 

Where have I been?  In those two years I was re-organized three times as Sun tried to stay in sync with the fluid system management industry.   I don't mind "re-orgs" too much as long as there is some reasoning.  I believe these were justified.  The first one was a switch to form a new team that allowed engineering to receive feedback and bug awareness from the field and customers faster regarding our systems management product set.  The second was a move into technical marketing for our virtualization products.  Lastly, the third was a move into datacenter software technical marketing where I gain more awareness of all Sun's datacenter software portfolio, but remain focused on infrastructure management.  I find in this new group there is cross pollination I can achieve with Solaris, OpenSolaris, OpenStorage, HPC, OpenNetwork, and how they relate to infrastructure management.  As an example, check out this how-to-guide I recently wrote for the Automated Installer in OpenSolaris.

Today I also decided to open a Twitter account.  I had opposed Twitter for the longest time because I thought it crossed a line that technology should not have...reading that someone is eating carrots!  But I've noticed people are open to looking to Twitter to find legitimate information  regarding products and technology so I figured "Why Not!"  You can find me here.


[0] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg

Walking the system (hardware) SNMP maze

Friday Sep 21, 2007

One question that comes up about a hundred times per year is "Explain to me what Sun offers in terms hardware mib instrumentation."  So I thought I would take this chance to dig into the topic.

One thing to keep in mind as you navigate this posting is that Sun is offering you a tremedous amount of freedom in the solution you choose below, but with any freedom there is a complexity of choice.  Your local Sun client service rep can help you map your needs to the correct solution.

There are 4 main themes in the SNMP solutions Sun offers for our hardware.  All of them support SNMPv3:

1) boxes with an embedded SNMP agent on their service processor/shelf
2) boxes that require you to install an unbundled agent on the OS
3) hardware alarms that are support by the Solaris 10 Fault Management Architecture (FMA)
4) boxes that support Sun Management Center (private/unsupported SNMP interface)

 


 


1) boxes with an embedded SNMP agent in their service processor/shelf

This category is growing the fastest and is becoming the prefered method at Sun.  The objective is to begin to run many virtualized environments on top of our hardware and thus we are motivated to place the hardware identification/fault layer down in the SP instead of the OS. 

- Sun Blade B1600
    http://www.sun.com/download/products.xml?id=3fcceae1
        ENTITY-MIB
        SUN-PLATFORM-MIB


- Sun Fire v20z and v40z
    http://www.sun.com/download/products.xml?id=45b0166d
    Drivers source Solaris (nsv-os_name)
        SP-EVENT-MIB           
        SP-INVENTORY-MIB       
        SP-MasterAgent-MIB     
        SP-PLATFORM-MIB         
        SP-SP-MIB            
        SP-SST-MIB


- ILOM based x64 u1/u2 rack mounts
All of the ILOM based u1/u2 rack mounts listed below support the SUN-PLATFORM-MIB which is located in the firmware.  Here is an example from the X4600 drive website:
    http://www.sun.com/download/products.xml?id=464d312d
    Tools and Drivers CD ISO/Common.zip
       SUN-PLATFORM-MIB

The problem with the SUN-PLATFORM-MIB on these platforms was that it was traping out IPMI event numeric codes that were in the specification for IPMI but not human usable or friendly.  So these boxes also support the IPMI Platform Event Traps (PETS).  This MIB module enables management platforms that are not IPMI compliant to partly decode standard IPMI Platform Event Traps (PETS) generated by an IPMI Baseboard Management Controller (BMC).
    http://www.sun.com/download/products.xml?id=4602d9ee
        Sun Fire X2100 M2
        Sun Fire X2200 M2
        Sun Fire X4100
        Sun Fire X4100 M2
        Sun Fire X4200
        Sun Fire X4200 M2
        Sun Fire X4500
        Sun Fire X4600
        Sun Fire X4600 M2
        Sun Blade X8400 Server Module
        Sun Blade X8420 Server Module

I should mention that based on this mib, Sun has begun to author integrations with many 3rd party management vendors in this space.  You can find them all listed here.  More will pop-up here over time:

    http://www.sun.com/systemmanagement/tools.jsp 



- Sun Blade 6000 Chassis
        Sun Blade X62x0
            http://www.sun.com/download/products.xml?id=461fbffa
            Tools and Drivers CD
                SUN-PLATFORM-MIB.mib
                SUN-ILOM-CONTROL-MIB


- Netra CT 900
     Netra CP 3010
     Netra CP 3020
        http://docs.sun.com/source/819-1178-10/snmp.html
            PPS-SENTRY-mib
            SAF-HPI-mib


- ALOM CMT firmware 6.4.4 or higher platforms
    Sun Fire T2000
    Sun Fire T1000
    Netra T2000
    Netra CP3060
        http://www.sun.com/download/products.xml?id=4653838a
            ENTITY-MIB
            SUN-PLATFORM-MIB


- Sun SPARC Enterprise M-series (Mx00)
    http://www.sun.com/download/products.xml?id=465de685
        OPL-SP-MIB
        FM-mib


- Sun Blade 8000 Chassis
    http://www.sun.com/download/products.xml?id=46d1c056
        Sun ILOM Control MIB
        Sun Platform MIB
        Sun HW Trap MIB
        Sun ILOM PET Events MIB



 



2) boxes that require you to install an unbundled agent on the OS

Let's move on to the second category of solutions we have here at Sun for our server.  The Management Agent for Sun Fires (MASF).  It uses many of the same library calls as the Sun Management Center agent (which we will discuss later) and therefore many of the packages are the same or at least lay down the same directory structure.  The current version (1.5.1) of the MASF agent can be found here:
    http://www.sun.com/download/products.xml?id=464b336a
        Sun Blade T6300 Server
        Sun Fire V125 Server
        Sun Fire V210 Server
        Sun Fire V215 Server
        Sun Fire V240 Server
        Sun Fire V245 Server
        Sun Fire V250 Server
        Sun Fire V440 Server
        Sun Fire V445 Server
        Sun Fire T1000 Server
        Sun Fire T2000 Server
        Sun Fire V1280 Server
        Sun Fire E2900 Server
        Netra 210 Server
        Netra 240 Server
        Netra 440 Server
        Netra 1280 Server
        Netra 1290 Server
        Netra T2000 Server
            ENTITY-MIB
            SUN-PLATFORM-MIB

 Maybe now is a good time to talk about the Sun Platform and Entity mib since you have seen it called out so many times above.  Some people love the extensibility of the two mibs.  The Sun Platform mib is dependent on the Entity mib.  The relation can best be explain by talking through an example.  Image I took your box and wrote down all the FRU components.  CPU1, CPU2, PS1, FAN2, etc.  Next let's say I started assigning each FRU component a number.  Now for the rest of the time I'm talking about your box's parts I use that number I assigned instead of the component's true name.  As long as you have the mapping of the number to the name (entity mib) you'll be able to understand what it is I'm talking about (Sun Platform mib).





3) hardware alarms that are support by FMA

If you are on Solaris 10, you can take advantage of the FMA mib.  Solaris 10 contains an net-snmp based agent called SMA.  Starting in s10u2 I believe Sun started shipping a FMA mib implimentation for the SMA agent.  FMA is growing with each release of Solaris, but in it's current state it does not offer the same coverage of fault awareness and the other solutions I mention in this blog.  I believe it is currently covering CPU, MEM, I/O, and PCI/X events as of s10u4.  Examples of things it doesn't see yet are power supplies, LED lights, fans, environmentals/tachometers.  For more information able enabling FMA via SNMP, check out this blog:
    http://blogs.sun.com/wesolows/entry/a_louder_voice_for_the




4) boxes that support Sun Management Center (SunMC) (private/unsupported SNMP interface)


Now that was a lot of boxes we just covered.  Time for a quick quiz.  What boxes have I not mentioned yet?  The really old stuff, the desktops, and the current high end workgroup server and higher.  The current high end workgroup server and higher would be:

    Sun Fire 280
    Sun Fire V480/V490
    Sun Fire V880/V890
    Sun Fire 3x00 --> 6x00
    Enterprise 4x00 --> 6900
    Sun Fire 12K --> Enterprise 25K

SunMC covers everything in the SPARC fleet that ever was released by Sun and currently is being released.  It is your onestop place for all Sun SPARC hardware monitoring needs.  For those of you that are unfamilur with SunMC, its most popular feature is the configuration reader which offers a tabled strcuture look at the hardware tree including FRU information along side of fault information. 

 

It supplements that with a picture of the box with realtime, mouse over fault and component redering.  Did I mention that part was free :)

 




SunMC is an on host solution.  So it doesn't stop at hardware monitoring and continues on up the stack to include in-depth Solaris (including pools, projects, and zones) and generic application monitoring.  Having said that, it is the closest thing Sun has to a traditional EMS.  Under this architecture you would normally integrate SunMC's server layer with your existing EMS event console.  There are four main ways to integrate SunMC events into an exsiting enterprise management framework.  Only two are officially supported by Sun Microsystems completely.  The other two are only partialy supported which include the SNMP interface.

Supported Ways:


1.) Our first and most robust integration is accomplished via the use of the developer's interface known as the client API.  The official documentation for this interface can be found on docs.sun.com in the SunMC Developer's Guide:

http://192.18.109.11/819-5203/819-5203.pdf
Chapter 18 ClientAPI
    pages 409 -- 417


If you installed the SUNWessdk package you will also find addition information in /opt/SUNWsymon/sdk.  In here you will find a docs/html directory with java class definations of the API plus actual code examples in /opt/SUNWsymon/sdk/examples/client.

Further, you can check out the SunMc integration whitepaper located here which will discuss the use of the clientAPI:

http://www.sun.com/software/whitepapers/solaris9/sunmc35_integration_wp.pdf

The whitepaper has a great example of using the clientAPI to save SunMc events to a file.

page 32

2.) There are many 3rd party integrations on the market and two bundled integrations within SunMC.  You will find all of these integrations are based on the SunMC clientAPI we discussed above.  these integrations go a step further and add in the files and procedures needed to integrate into the 3rd party.  SunMc ships a CA NSM 3.x integration and Tivoli TEC 3.x integration.  More information can be found here:

Tivoli TEC:
    http://192.18.109.11/817-1100/817-1100.pdf

CA NSM:
    http://192.18.109.11/817-1101/817-1101.pdf

The majority of the other integrations are either from the 3rd party itself or from Halcyon (www.halcyoninc.com).  Halcyon has an excellent solution called EventAction that will add enhanced filtering on top of the event stream coming from SunMC.


Unsupport Ways:


The next two integrations are based on SNMP.  The intefaces we will be discussing are supported.  The mib oid information found in mibs located in /opt/SUNWsymon/util/cfg are supported.  The only thing that is not supported is the OID information coming from private mibs.  Private mibs are those not contained in the /opt/SUNWsymon/util/cfg directory.  The only difference between the following two ways are the touch points.  One is an integration at the agent layer while the other an integration at the server layer.

3.) The agent Layer SNMP trap integration involves the es-trapdest command:

This command will send all trap notifications to a 3rd party via SNMPv1, 2c, or 2uSEC.  SNMPv3 traps are not allowed at this point by the code base.  It is up to the user to choose which traps he prefers to receive.  The most important are the sunmcTraps as they contain the status change alarms produced by the agent when a module threshold is breeched.

# /opt/SUNWsymon/sbin/es-stop -a
# /opt/SUNWsymon/sbin/es-trapdest -a <hostname>:<port> -v SNMPv1 -u public -f "sunmcTraps snmpTraps coldStart warmStart linkDown linkUp authenticationFailure"
Added new entry:
[new] <hostname>:<port>
    SNMP version: SNMPv1
    SNMP user: public
    Trap filter: sunmcTraps snmpTraps coldStart warmStart linkDown linkUp authenticationFailure

To see the multiple trap destinations issue the following comamnds:

# /opt/SUNWsymon/sbin/es-trapdest -l

Now start the agent and begin receiving traps.

# /opt/SUNWsymon/sbin/es-start -a

This will produce a trap in the following format:

The most imporant part of the trap is the faultOID line.  The format of the faultOID is the following:

attribute OID in fault with an DEC tranlation of the table index appended to it:table row instance number:severity:rule in violation:threshold in breech:current value

2006-05-12 12:52:11 ally [129.148.171.122] (via 129.148.171.122) TRAP, SNMP v1, community public
        SNMPv2-SMI::enterprises.42.2.12.2 Enterprise Specific Trap (1) Uptime: 6 days, 16:42:22.85
        SNMPv2-SMI::enterprises.42.2.12.2.1.3.1.0 = OID: SNMPv2-SMI::enterprises.42.2.12.2.2.12.5.1.1.1.3 SNMPv2-SMI::enterprises.42.2.12.2.1.2.2.1.8.0 = STRING: "1.3.6.1.4.1.42.2.12.2.2.12.5.1.1.1.3.2:2:error-gt:rCompare:5.0:94.1" SNMPv2-MIB::snmpTrap.5.0 = OID: SNMPv2-TM::snmpUDPDomain SNMPv2-MIB::snmpTrap.6.0 = Hex-STRING: 81 94 AB 7A 04 89

The trap mib for SunMC is located in /opt/SUNWsymon/util/cfg.  This will explain the varbinds in the trap, but will not translate the oids found within the varbinds.  

The next step is discovering which OIDs exist in the product so you will be able to get on something.  Most 3rd party snmp managers are looking for this information in SNMP ANS mib format.  To see this information, do the following:

First, to get the ANS MIB files, you can start the agent in interactive mode:

# /opt/SUNWsymon/sbin/es-stop -a
# /opt/SUNWsymon/sbin/es-start -ai
>

Wait a few minutes while the agent loads the mibs.  You can tell by cd'ing out on the OID path.

> cd .iso.org.dod.internet.private.enterprises.sun.prod.sunsymon.agent.modules

Or watching the /var/opt/SUNWsymon/log/agent.log.  When they are loaded issue the mibExport command.

> mibExport

That will dump the ANS mib files in the /var/opt/SUNWsymon/cfg directory with a filename.txt.  Next quit out and start the agent up again in daemon mode as normal.

> exit

# /opt/SUNWsymon/sbin/es-start -a

Remember, these are the private mibs.  The public ones are located in /opt/SUNWsymon/util/cfg.


4.) The forth and final integration point is to forward the traps from the server layer.  This is explained in the following location:

http://192.18.109.11/819-5203/819-5203.pdf
Appendix H Trap Subscriptions
page 645


You can also issue the same es-trapdest command mentioned above on the SunMC server layer and then move the resulting /var/opt/SUNWsymon/cfg/agent-trapdests-d.dat to /var/opt/SUNWsymon/cfg/trap-trapdests-d.dat.  Then stop and start the SunMC server layer:

# /opt/SUNWsymon/sbin/es-stop -A
# /opt/SUNWsymon/sbin/es-start -A






[3] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg

I'm Back!

Wednesday Sep 19, 2007

I must say it's pretty embarrassing to have not posted for about 2 years.  It's been a busy 2 years that included a ton of work related travel, multiple project changes, a job change, and a new member to my family.  But now that I've moved out of the field architecture, implementation, and deployment work and can be more focused on getting the word out about Sun's IT management stack to a larger population of users.  I currently hold the title of Technical Marketing under the Connected Systems Management business unit within Sun.  I look forward to filling you in on the little tricks of the trade I have learned over my 9 year career at Sun and welcome the comments in return.  

 You've got to love technology though.  I've neglected this blog long enough that Sun now has wiki sites people can create that are product specific.  Maybe I should spend my energy there?  We'll see how this goes and make that choice later on in 2008.
 

[0] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg

First Posting

Monday Jan 09, 2006

Hello World.

This is my first postings to Sun's external web log (blog) collection. My name is Mike Barrett and I work in the N1 Software Integration Group at Sun Microsystems. My plan is to use this blog as a place to release my thoughts on Sun's exciting N1 manageability software line. For those that do not know which specific products that includes:

  • N1 Service Provisioning System (SPS)
  • N1 System Manager (N1SM)
  • Sun Management Center (SunMC)
  • Sun Control Station (SCS)
  • Solstice Enterprise Manager (SEM)

I will speak on topics related to deployment hints, feature explainations, ITLM, systems or hardware monitoring, provisioning, modeling, etc. I will try my best to keep it interesting and as close to the road or real life as possible.

[3] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg