For instance, developing custom modules. Here’s a great example. The online docs could definitely benefit from advanced examples such as that. Well, anything that discusses advanced topics like that. This is what I was hoping to take away from the developer training, but I don’t think they’ll be covering topics like this.

See, I haven’t had any money goals recently (having met all the previous ones) and I only recently took up a new hobby which I’m saving up to buy something for. So I felt it was prudent to begin slowly picking away at the 176k; every little bit helps.

Concerning some work that I’ve been doing recently, I’ll have available a comparison of different approaches I took to mass name resolution at work and the tools to back it up.

For a couple months now we’ve been doing mass DNS resolving of all our border netflow. My solution is able to resolve on the order of 500,000 addresses a minute. Note that this include duplicates. All told, it’s about 24 gig of netflow a day.

We splunk all that, and have made special field extractions so that searching it is trivial. You may have noticed some of the charts that I posted a while ago which were relevant to these name resolutions.

Well, the plan is to start resolving more than just our border router. There are two problems though.

1. We’re not sure how much more the current solution can do before falling behind
2. We don’t have enough cash to buy the necessary splunk license

#1 is solvable and is what I’m currently working on. I have no idea how we’re going to solve #2 though. That’s ultimately the problem with splunk. You can find so many uses for it that you run out of money before you run out of good ideas.

That needs to be added to their product roadmap.

The work that I’ve been doing for #1 involves 2 approaches.

1. Distributing the work between nodes via TCP and HTTP
2. Distributing the work between a shared NFS volume

#1 is a PITA to make work correctly, however #2 isn’t any walk in the park either. I think I have a solution for both though in the form of events in node. I’ve made a P.O.C. using idea #2 since it was easiest to wrap my head around at the time.

I really like node. It’s the “generalized event engine” that I was dieing to have. It’s evented like bro, but it can be applied to so many more situations than bro can. On top of that, it compiles cleanly on linux and it doesn’t have some special language that you need to write stuff in; it’s all javascript. Thank you Ryan.

One thing led to another though and before I realized it, I was in the middle of the splunk documentation again, reading up on how to embed one of their fancy charts in said application.

I love splunk’s charting ability. It’s fantastic.

I’ve wanted to do this embedded thing for a while so that I can stop fussing with all the crap open-source charting solutions out there. I’ve read the documentation several times in the past but only today did I try to tackle it.

Well, the story ends with “I got it working”. The story itself though is composed of a couple hours of me asking myself why the hell am I getting this weird error.

The error was

App does not support UI access. See its app.conf for more information.

The full details are here at splunk-answers.

One thing I took away from this foray is that in the documentation for doing this, when they say

Refresh your view when you make changes to it by loading this URI:

They literally mean to use that URI. I was replacing the “search” value with the name of my own app. It don’t work that way.

Another thing that I hit a snag on is some screwy thing to do with permissions. I have a user that has two roles. One of those roles has the permissions

srchDiskQuota = 10000
srchJobsQuota = 50
rtSrchJobsQuota = 100

The other role doesn’t have those permissions; they’re not specified. splunk, for whatever reason, was using the values defined in etc/default/authorize.conf

[default]
srchDiskQuota = 100
srchJobsQuota = 3
rtSrchJobsQuota = 6

For the values. Why?

Anyhoo, I did what I originally had set out to do, with the exception that I haven’t actually embedded it in my app. That would be the next step.

1. Sign my name
2. Put my acct number
3. Write the words “Deposit only”

Now, you may be asking yourself “why #2″ and for good reason. I always asked myself the same thing. I guess I could understand #3 to some extent, and really it doesn’t seem to make much difference in what else you may write on the check as long as they’ve got that signature.

My parents justification was “well, if there’s a question about which account it goes into, putting the acct # there will clarify that”. Hmm. Ok, well, that makes sense I guess. But thinking a bit longer about that explanation leads one to questions oneself. “That’s not how ATMs work…is it?”

Last time I was at an ATM to deposit a check, the transaction went something like this

1. slide your ATM card
2. enter your pin

At this point, logic would dictate that all future “stuff” you may do will be recorded as being done by you. The Tim. Not the other Tim, the current Tim.

1. give check to ATM
2. ATM reads check, asks if deposit guess is correct
3. Yes or no
4. Receipt if asked for

So if you were to stop and think about the potential ways to C.Y.A here, you’ve got

1. A transaction log that may have been created when you swiped your card and enter your pin
2. Verification by ATM that deposit amount was correct
3. Receipt from ATM that you made a deposit. Some banks will print your deposit check on the receipt too.
4. Record in bank’s website that there is a pending transaction on your checking account
5. Picture of check on bank’s website for your account after check has made its way through system

Which leads me to ask “whats the point of writing the account number” ?

The simple answer is that there is no point, there never was a point, and you should stop doing it now because it’s not helping things.

Let me point out something that you may not have noticed. Let’s say your bank takes pictures of your checks and posts them online in your transaction register. Usually they take two pictures. A picture of the front and a picture of the back. Well, if you take a moment to look at the picture of the back of the check, guess what you’ll see?

1. A signature
2. An account number
3. The words “Deposit Only”

Whoops, shouldn’t have added those extra items…

So yes. With the advancements in technology and the lack of any form of liability on the consumers part for bank mishaps, I would suggest that it is sufficient to only write your signature on the back of the check. Well, maybe a haiku or something would also be appropriate if you’ve got the extra space.

You wouldn’t know it was Millington though because there is no sign that says it is. It was getting late in the day so I found some more back-roads and eventually got back on 71 and headed home; it was a great time.

I also watched Seven Pounds the other night (feel good movie of the year…not) and have been making my way through the Harry Potter movies because my friend Bernie is itching to see the upcoming two and I’ve never seen any of them, so I need to catch up. Aside from them being 2 and a half hours long, their not as terrible as I thought they would be. And, well, with Netflix, who really cares if they were terrible or not.

• javascript – is ip in subnet
• step, control flow in node.js
• common programmer health problems
• selenium2/webdriver
• guest information packet
• splunk search example
• socket.io for node
• chocomize
• car complaints

Well, my brother Dan and I met up at about 10:30 pm on Monster Hunter where he was doing some arena quests. We putz’d around for a little time not doing anything in particular and then I told him I needed some help getting some parts for weapons.

Now, for about a week now I’ve been stuck at level 50 because there are a ton of assholes on MH3 online who like to talk down to the younger, less experienced, players.

The way it usually works is a newly level 50′d player will go on the Recruiting servers, start up a channel specifying they need help with their 5 star urgent quest and then wait there…forever. No one will usually show up. Or those that do will also be level 50 and they will get into this pissing match about who gets to start the urgent quest. Reason is because only the person who starts the quest will get the win if the whole team wins.

Now, what sometimes happens is a 51+ char will walk into the room and start talking all big like they are god’s gift to the world. Usually they talk down to the younger players. These people are fags, plain and simple. They contribute nothing and they are all looking for respect that they haven’t earned.

In the slim chance that they help you, they usually aren’t very good. They will join the quest, and probably not die due to their high level armor, but when the team loses, these assholes will blame all the younger players…who have no experience fighting Alatreon. How can you blame the younger players?

That was all until tonight. At about 2 am I was getting tired and ready to call it a night, and then this player came into our room; Olina. When they entered, the first thing they said was “Greetings, mortals”

Now, first thing I thought was, great, another talker ready to give us shit. Well, Olina came into the room with an Alatreon hammer. I figured, ok, what’s this player all about. I checked out their hunter rank, only to find out they were level 283! hahaha!

OK! So, this person appears to walk the walk.

Anyways, Olina was really cool. It was a female character, but who knows if it was a dude playing the character or not. They were funny and didn’t give any of us shit about being lower level than them.

Olina asked us “whatcha up to?” and we responded “trying to figure out how to beat our 5 stars” to which she responded “cool! I’m in!” hahaha. I love happy players.

And sure enough, she was good. She went with the hammer and gave us enough advice to be really helpful but not sound like a prick.

So, in the slim chance that Olina ever reads this post, here’s to you!

I was telling my brother Dan, that “see, this is what we gotta do. Being awesome like this is totally the way to go”. So yeah, I can’t wait to be better equipped and have Dan and maybe Will along for the ride so that we can help out others like Olina helped us. Way cool.

I also discovered why everyone who has an Alatreon weapon has either a long sword, hammer, or lance. The reason? Because Alatreon drops the items to make those weapons more than he drops any other item!

So when you see all those players with Ala weapons, you might just want to think again as to whether they are actually good players or not. Most of them are not. Then there are real players who deserve those weapons and armor. Tonight, we had one of those real players on our team.

So I signed off at about 4 am and was going to go to bed, but was still riding the wave of Olina’s help, so now I’m watching Django from Netflix. Tomorrow is Lindsey’s birthday and that means cake; woot.

There was a peculiar bug that was occurring whereby the audit results on a /24 subnet would have all the IP address for that subnet listed even if there was no live node on the network. This turned out to be incredibly frustrating too as I knew it was a problem with my software and not Nessus this time.

So after a little while scratching my head, I tracked it down to nessquik not sending the appropriate values in the POST to Nessus to create the scan. It was resulting in none of the port scanners, or their associated settings, being provided to Nessus. So Nessus did the next logical thing and just decided to scan and report everything it could. The result is that you’d get these reports with lots of irrelevant information in them.

Well so I fixed that and now Nessus finishes scans much quicker. To the point where I almost cannot keep up with scheduling new stuff, which is fantastic.

• freegeekchicago
• quilt
• redshift
• cherrypy tutorial
• pennpoint

It’s especially important in regards to web applications because people have no tolerance for waiting on the web. We want things to be responsive, snappy, fast. Developers can give us that, but it takes more foresight (and experience in failing to do this) than it does to take the easy way out.

The easy way out is to send everything back and forth between the browser. It’s easy because you do not need to keep track of anything. You put it all out there and you get it all back.

The problem arises when it comes time to deal with larger volumes of data. You did plan on your software managing larger volumes of data than you tested with…right?

Most people don’t. I know I didn’t. I figured with nessquik “why do I need to page this target list? I can’t imagine any single person having more than 50 or so targets”. And then I found myself adding site-audit functionality….with exclusion lists of 1500+ targets. In the words of the grail knight from The Last Crusade, “he chose poorly”

I had made the decision to send everything back and forth between the client and server, and my code on the backend operated this way. However, to do this was unacceptable because it caused the user experience to be abysmal. Loading several meg of HTML into the browser makes most browsers kack.

This problem turned me in the direction of paging. I could page the target list, only loading a small subset, 15 or so, and then loading the rest on demand, a page at a time.

This works, but requires that the backend be modified to support paging and the front end be modified to support paging. In addition to that, I also need to modify the backend and frontend to support updating of the audit targets in a totally different way; by using only the changes.

Since the whole list will usually never be on the screen, I can no longer rely on sending the whole list back to the server to be saved. I now needed to track the changes to the list and apply those changes on the backend when the audit is saved.

These changes are not overly complicated, but they require time, and time is a precious commodity.

Unfortunately, you usually do not have these revelations until you’ve experienced doing it wrong. Even for small applications, or applications that are home-grown and will stay in-house, you develop the false belief that your data volume will always ever be small. That’s usually not the case.

Beware that there is quite a large amount of ass-kissing and sucking up to a particular vendor. The majority of the article isn’t worded as I worded it (editors) but it sure makes me come off as if I have my lips firmly attached to the ass of every member of splunk’s board of directors.

So please, for heaven’s sake. If you read the article, do not come to this blog and give me praise for how glowing of an article I wrote for splunk; it isn’t like that. Splunk is cool, no doubt, but they are far from where they were a couple years ago.