Apr 202011

This week has seen some truly amazing spring weather around the MadLab including everything from tornado threats and sustained high winds to flash flooding and dense fog.

April showers, as they saying goes, will bring May flowers – so we don’t mind too much as long as the power stays on and the trees don’t fall on the roof!

In cyberspace things are also picking up it seems. For about the last three weeks we’ve seen declining severity and frequency of spam storms. However this week has been different.

Beginning about 3 days ago we’ve seen a surge in new spam storms and in particular a dramatic increase in the use of hacked web sites and URL shortener abuse.

Previous 30 days of spam storms.

After 3 weeks of declining spam storms, a new surge starts this week...

There is also another notable change in the data. For several years now there has been a pretty solid 24 hour cyclical pattern to spam storms. This week we’re seeing a much more chaotic pattern. This and other anecdotal evidence seems to suggest that the new spams are being generated more automatically and at lower levels across wider bot nets.

We used to see distinct waves of modifications and responses to new filtering patterns. Now we are now seeing a much more chaotic and continuous flow of new spam storms as current campaigns are continuously modified to defeat filtering systems.

Chaotic spam storm arrival rates over the past 48 hoursThere’s no telling if these trends will continue, nor for how long, but they do seem to suggest that new strategies and technologies are coming into use in the blackhatzes camps. No doubt this is part of the response to the recent events int he anti-spam world.

Microsoft takes down Rustock spam botnet

DOJ gets court permission to attack botnet

In response to the blackhatzes changes my anti-spam team and I have developed several new protocols and modified several of our automated friends (rule-bots) to take advantage of new artifacts in the data. The result has been a dramatic increase in the creation rate of new heuristics, reduced response times, and improved preemptive captures.

Rule Activity Display shows higher rule rates and smoother hit densities

With these changes, changes in blackhatz tactics, and new sniffer engine updates coming along I’m going to be very busy watching the blinking lights to keep track of the weather outside the MadLab and in cyberspace.

Apr 072011

Here’s a new term: quepico, pronounced “kay-peek-oh”

Yesterday Message Sniffer had a quepico moment when Brian (The Fall Guy) of the sortmonsters coded rule number 4,000,000 into the brains of ARM’s flagship anti spam software.

You read that right. Since I built the first version out of spare robot parts just over a decade ago more than 4.00e+6 heuristics have been pumped into it and countless trillions (yes trillions with a “t”) of spam and malware have been filtered with it.

I had another quepico moment yesterday when I realized that a task I once did by myself only a couple of hours per day had now expanded into a vast full-time operation not only for the folks in my specific chunk of the world, but also for many other organizations around the globe.

The view from SortMonsters Corner

The view from SortMonsters Corner

Just as that 4 millionth rule represents a single point of consciousness in the mind of Sniffy, these realizations are represented somewhere in my brain as clusters of neurons that fire in concert whenever I recall those quepico moments.

Interestingly some of these same neurons will fire when I think of something similar, and those firings will lead to more, and more until it all comes back to me or I think of something new that’s somehow related. This is why my wetware is so much better than today’s hardware at recognizing pictures, sounds, phrases, ideas, and concepts when the available data is sketchy or even heavily obscured like much of the spam and malware we quarantine to protect us from the blackhatzes.

Blackhatzes: noun, plural. People and organizations that create malware and spam or otherwise engineer ways to harm us and exploit or compromise our systems. These are the bad guys that Sniffy fights in cyberspace.

Sniffy on guard

At the moment, most of sniffy’s synthetic intuition and intelligence is driven by cellular automata and machine learning systems based on statistics, and competitive and cooperative behaviors, adaptive signal conversion schemes and pattern recognition.

All of that makes sniffy pretty good but there is something new on the horizon.

Quepico networks.

For several years now I’ve been experimenting with a new kind of self organizing learning system that appears to be able to identify the underlying patterns and relationships in a stream of data without guidance!

These networks are based on layers of elements called quepicos that learn to recognize associations between messages that they receive. These are organized into layers of networks that identify successively higher abstractions of the data presented to the lower layers.

Noisy Decision

Noisy Decision in a Quepico Network

The interesting thing about the way these work is that unlike conventional processing elements that receive data on one end and send data from the other, quepicos send and receive messages on both sides simultaneously. As a result they are able to query each other about the patterns they think they see in addition to responding to the patterns they are sure they see.

When a quepico network is learning or identifying patterns in a stream of data, signals flow in both directions – both up the network to inform higher abstractions and down the network to query about features in the data.

In very much the same way we believe the brain works, these networks achieve very high efficiencies by filtering their inputs based on clues discovered in the data they recognize. The result is that processing power is focused on the elements that are most likely to successfully recognize important features in the data stream.

Put another way, these systems automatically learn to focus their attention on the information that matters and ignore noise and missing data.

So what’s in a name? Why call these things quepicos? As it turns out – that’s part of my own collection of quepico moments.

One day while I was drawing some diagrams of these networks for an experiment, my youngest son Ian asked me what I was doing. As I started to explain I realized I needed a name for them. I enlisted his help and we came up with the idea of calling them thinktons. We were both very excited – a quepico moment.

While looking around to see if this name would cause confusion I discovered (thanks Google) that there were several uses of the term thinkton and even worse that the domain thinkton.com was already registered (there isn’t a site there, yet). A disappointing, but definite quepico moment.

So, yesterday, while roaming sortmonster’s corner and pondering how far we’ve come and all of the little moments and events along the way (trillions of little = pico, que = questions, whats, etc) I had another quepico moment. The word quepico was born.

Google’s translator, a favorite tool around the mad lab and sortmonster’s corner translates “que pico” as “that peek.” That fits pretty good since quepicos tend to land on statistical peaks in the data. So quepico it is — I’ll have to go tell Ian!