Steve Jones has asked for some folks in the community to talk about what it’s like to work in our field across four posts. So how do I talk about my job across four job posts? If you didn’t read part 1, you can click here to read about what the day to day is like.
My four series post is structured as:
- Day to day, what is normal? What are 95% of my experiences?
- Best days ever, these are the highlights.
- Worst days ever, these don’t happen all of the time and this is what nightmares are made out of.
- Weird days, Brent Ozar said it best when he said Database Administration is weird.
The weird days of being a DBA.
You can run into a lot of weird things being a DBA, but I don’t think the topics I’m going to discuss are what you are expecting. Most of the weird things I run into are design structures or processes in place that go against best practice or are not typically practical. Kind of like if the past admin gave out Neapolitan ice cream instead of vanilla. Everything is custom, unique, and different, whereas I try to instantiate things as “vanilla” as possible. If things are predictable and implemented consistently, it’s easier for anyone to come in and troubleshoot or support.
Well, what kind of weird things do I see then? Well, you have self-made issues, you have known and unknown bugs, and sometimes you just have weird behavior due to … computer gremlins.
Here are some examples of impractical implementations I’ve worked with: Primary keys that are clustered indexes based on 32 bytes (guid + datetime2 + datetime2), procedures meant to implement high concurrency DML when interacting with a table without handling isolation levels, and more than five duplicate indexes on the same table (minus the name of the index of course…).
Those scenarios are not best practice and it really goes against practical design. So you need to look at the overall picture and decide what the end goal and intentions are. Think about why and why not some of the features were implemented or missed. Whenever I go against best practice design, explanations go immediately into the documentation and comments go into the code. Sometimes it’s important or necessary to stray from vanilla to get some strawberry ice cream, but make sure your successors and current employers know/understand why you deviated.
Sometimes, things just DON’T make sense. I saw a weird issue where you cannot expand a drop-down box in SSRS when you open it. This took me a few minutes to figure it out. But what was the issue? It was a service pack bug. A quick cumulative update fixed that right up. This goes into what I talked about in my last session, the worst days ever. If you are propagating your changes up through multiple environments, you should catch issues early on and you don’t have to scratch your head wondering why you suddenly had new changes.
Other times it could be a bug that has no fix or report yet, but those are even rarer. And since they are rare, you can imagine how frustrating it can be trying to fix things that aren’t documented or experienced by many people. Microsoft has employees who help on StackOverflow who can help you identify bugs, so that may be a good resource if you get stuck banging your head against the wall.
Most issues you run into are probably user error or bugs. Sometimes though, things just don’t work and there may not be an explanation you can find for it. Not that there isn’t an explanation, just that you need either the proper tools, access, or knowledge prior to the problem occurring. If it’s a one-time event and it doesn’t happen again, good luck figuring that out without having the problem monitoring in place. Likewise, if something is going on outside of your control (network, sysadmin, application, etc), you’re going to have a hard time figuring out why things are behaving why they are. (Noisy neighbor syndrome in a VM is a common example. Where one box on the host eats the resources and the other VM has to shift resources back to the ESX Host to help that neighbor.) I like to call those problems, computer gremlins.
The best way to combat computer gremlins is with mentors and knowledgeable co-workers. It’s hard to know what you don’t know. When I started learning SQL, it was difficult to learn because I didn’t know what kind of questions to ask. Mentors can help guide that path and get you where you need to be so when a gremlin inevitably pops up, you’ve thought about and then implemented monitoring that can assist in that scenario or you know someone who can help.
The three weirdest IT experiences I’ve had:
- A ticket that literally stated: “Desktop background is showing a clown spitting plates and there is a typo in the background. This is affecting all agents and they would like the “scary” picture changed or removed.”
- That time I had to drive to another site to help teach users how to use the CTRL+C and CTRL+V hotkey functions.
- Troubleshooting split brain syndrome in an AG setup.