Monday, February 22, 2010

Nails everywhere I look....

The other day a colleague asked me what tools I use on a daily basis for the troubleshooting I do. My first reaction was 'my brain', but that didn't sound very nice and I got concerned he would take me literally and we would have a 'Silence of the Lambs' thing happening and I would end up in some well strangling some little poodle to save my life. Anywho, the tools I use pretty much on a daily basis are comprised of freeware from several different developers as well as built-in applications.

The first set of tools will not be that much of a surprise to most Windows admins, Sysinternals (Mark Russinovich, you're my hero). These tools are so awesome that Microsoft bought the company and Mark is now a technical fellow there. There are about 65 different utilities, some are great (process explorer) and some aren't (RegJump). The main ones I use are:

Process Explorer
This one is my favorite. Think of it as Task Manager on steroids. It will show everything you could ever want to know about a process including all handles and dlls that are in use.

Process Monitor
Shows real-time FS, Registry, TCP/UDP, and Thread/Process for all processes. This is a must if you ever want to know what a process is actually doing.

AutoRuns
Shows you what programs are configured to run during system bootup or login, and shows you the entries in the order Windows processes them. I have found some many compromised servers using this tool.

Strings
This is used to display all strings contained within a file. This is also handy for compromised systems.

As I said there are over 60 tools and these are just the tip of the iceberg, so check them out.

Another tool set I use are from NirSoft. These guys have some great tools that I can't ever seem to replace.

Currports
This is network monitoring software that displays the list of all currently opened TCP/IP and UDP ports on your local computer and the executable listening.

OpenedFilesView
Displays the list of all opened files on your system. For each opened file, additional information is displayed: handle value, read/write/delete access, file position, the process that opened the file. You can also close one or more opened files, or close the process that opened these files.

RegDllView
This tools shows all the object that are registered on the system. Is a must for anyone responsible for a web environment

NirSoft also has a full line for password recovery utilities that have saved me more than once.

As for built-in commands I use the following everyday (i'll let you Google them):

netstat
tasklist
taskkill
findstr
netsh
sc
net

I hope these tools can help you find the cause of any issues that you are trying to resolve as they have helped me for years.

Jeff

Wednesday, February 3, 2010

Troubleshooting. It ain't just a river in Egypt.

Working as a sysadmin I use my troubleshooting ability on a daily basis. I also see a lot of other techs really struggle with troubleshooting. I have never had any formal training on troubleshooting, I think it is one of those skills I was just born with and I am happy I was. It makes my life easier. But I do realize that not everyone has this skill and I want to share a basic troubleshooting methodology that anyone can follow for any troubleshooting they need to do.

Troubleshooting typically consists of the following steps:

1) Define
2) Analyze
3) Implement

Define

Problem Statement: Create a clear, concise statement of the problem.
Identify the symptoms: What works? What doesn't?
Identify Differences and Changes: What has changed recently? What is unique about this system?

Analyze

Brainstorm: Gather Hypotheses: What might have caused the problem?
Identify Likely Causes: Which hypotheses are most likely?
Test Possible Causes: Schedule the testing for the most likely hypotheses. Perform any non-disruptive testing immediately.

Implement

Implement the Fix: Complete the repair.
  • Verify the Fix: Is the problem really fixed?
  • Document the Resolution: What did we do?

  • Using this basic formula will provide a smooth, timely troubleshooting path.
  • I'll focus on more real world examples in my next post.

  • Jeff
  • Intro...

    Well I decided to follow my colleagues and but a blog online with interesting things as I find them. I should tell you a bit about myself so you can decide on whether or not you want to actually ready this.

    I am a systems engineer with over 15 years IT experience. My current job is focused on Windows but I have touched a lot of things in the industry. AS I run into interesting things or cool sysadmin stuff I'll be putting it here.

    Jeff