Why SAP HANA needs help to unfold it’s full potential
First of all, I think that IT is still a dynamic environment. I know that if you go to SAP TechEd, or DSAG (here in Germany) or any other IT fair, you can see the slowly increasing average age. Not so many young talented engineers are going into IT like it was 10 or even 20 years ago.
Nevertheless, IT still has loads of possibilities for new and exciting inventions. And right now, there is one unfolding right in front of our all eyes. Well all eyes that have something to do with SAP at least.
A new invention
SAP founder Hasso Plattner has, I think the right terminus would be, co-invented an In-Memory Database which in turn SAP has put into a usable platform. A database? What the heck is anything inventive or cool or even worth writing about a database? you might ask. Nothing. I would answer, if this one was anything even remotely as boring as the next Oracle, or DB2 or even a new try on an XML-DB system.
But HANA is different. First of all, it’s in memory, you probably guessed that. Second, it makes use of a completely different way to store it’s data. And third, and because of one and two, it’s the first database to actually use all that immense computing powers, you can have in todays, affordable server systems.
What is SAP HANA?
I won’t dive into the depths of SAP HANA here. Other people with much more insight then me have already done that countless times (Update: I’ve actually done that in a later post). No, what I’d like to do here is point out the major hurdle HANA has right now.
Ok. I will write 2 or 3 sentences about HANA, might make it easier to understand my point of view.
HANA stands for High performance Analytics Appliance. It is a combination of certified hardware (currently, to my knowledge, provided by Cisco, IBM, HP and Fujitsu-Siemens) with a column oriented database installed on it. It also comes with development tools to write and install applications on the platform. So far nothing special. Yet the trick is, the database and all the applications to run on it, are completely held in memory all times. No bit of data needed for any task you throw at the box, has to be read from hard disk. By the way writing to disk is only needed for archiving and backup and is done asynchronously.
This alone brings an incredible performance boost, as you can imagine. Reading and writing to RAM is much by magnitudes faster compared to the same operation on disk. But this alone wouldn’t make HANA anymore interesting as, let’s say, Oracle Version Blah Blah, or whatsoever. The really inventive fact about it is this: the data in that database, is not organized in the usual relational database scheme of columns with attributes and rows that make up the actual dataset. Instead it is stored column-oriented and that is perfect in sync with the fundamental architecture of computer memory itself. Because of this, the CPU is usually able to simply read through neighbouring RAM locations to get a complete dataset and work on it. And because all data and applications are in memory, the CPU, or to be more specific, the multiple CPUs with multiple cores, won’t waste precious cycles waiting for the data to be transferred from hard disk into RAM, instead you throw a computing task on HANA and all CPU cores can read the needed data and work on it in parallel, nearly instantly.
Update: Of course, HANA can also use Row-storage, but this is slower than column-storage and shal be part of my technical deep dive on HANA.
Keeping everything in RAM and organizing the data in such a way that the CPUs can access mostly neighboring areas, increases the speed and brings HANA miles ahead of any other database technology out there. It also ends that biggest of all performance killers, when data, that is thought to be not needed anymore, is unloaded from RAM (or even from the CPU cache) back to disk, just in that second, when it is again needed. Usually this happens because the garbage collector, or whatever memory optimizing technology is in use, decided the data won’t be needed anymore. A garbage collector is, if viewed from a user centric work load, a mind blowing stupid thing, because it has no knowledge on how a user works.
There is another architectural speciality to SAP HANA – it never updates or deletes a dataset, but simply appends new values with timestamps. This brings you the possibility to not only work on the latest data, but also to compare those to past values and the time span in which data is updated. It might not sound too interesting, but it unlocks a whole new way of working with your data – take a look at the SAP presentations for more information on why this is great.
Of course my explanation is, to say the least, a highly inaccurate oversimplification of HANA and how it works. I suggest you take a look at these two links, to get more information and the real hard facts about SAP HANA.
My two cents
As I said, I won’t write a new blog about HANA and how it works, but instead would like to tell you, what is the biggest problem.
Here’s my first thesis: I say, that HANA is the next big thing in computer technology. Just like the invention of microprocessors, like Client Server back in the 80’s, like affordable and actually usable personal computers and of course the internet. But, without a broad developer base from independent (and that means independent from SAP) companies. Without creative users and architects and designers to think beyond SAP’s enterprise analytics and planning horizon, it can’t unfold it’s full potential.
SAP HANA is fast. SAP HANA is flexible (you can build your own applications on it and let it run in a secured and predictable environment) and, again, SAP HANA is really fast with Big Data.
So what can you do with it? I say, anything. And definitely more than just really, really fast analysis and prediction of customer reactions to an increase of price for a certain product – sorry SAP that is of course important to a lot of customers, but it’s really not the end of the story.
So, what needs to be done?
My second thesis: I say. SAP needs to lower the licensing costs for HANA for startups and small but very inventive companies, so that these people can invent on SAP’s invention. I don’t say they should give it to the world for free. Of course not. But they should rethink the pricing for it. I won’t write how much licensing fee SAP takes. I’m not sure, I’m allowed to do so, but I say, current pricing is way too high to spark a real gravitation center of inventions. I actually think this to be sad, because it is the one most fascinating and promising technology to have emerged in IT in the last 10 or maybe even more years. And also the one, that really has the potential to change the world of computers.
My third thesis is this: With SAP HANA a lot of inventions that we already buried in the mass grave of “not possible, sorry. technology just didn’t hold it’s promises”, could be worth a second look to see how they now work.
Let me give you an example. Some 15 or more years ago, I had the idea of a firewall that did not work on static rulesets like “that IP address is ok, but that not or this protocol is ok, but that not“, but rather analyzed normal network traffic in a company and created a baseline of this. It then would compare all traffic against this baseline and, when it found something strange or unusual, it would react. The firewall was supposed to learn how normal network behavior, for it’s specific environment, was like and based on that knowledge would find suspicious network activity.
Sounds cool, eh? Yeah, I thought so too. But unfortunately it was impossible to realize. Too slow CPUs, too low bandwidth with regard to IO and way too much data volume to analyze. This was because, if you really wanted the network analysis to be trustworthy and accurate, you had to store massive amounts of data. Did I mention, that the baseline idea, didn’t work out? It didn’t, but that’s exactly what killed the idea. Because the aggregation of the network traffic rendered the whole idea of comparing it to current traffic useless. On the other hand, with raw network data for the comparison, no computer system was fast enough to do the analysis.
With the new SAP HANA appliances, this old idea might actually work today. HANA is able to provide the speed and extremely fast reaction time needed (that’s only because of better and faster hardware). And it combines this with the possibility to analyze high volumes of data in near real time, because everything is in RAM. The result is, you could actually have your firewall inspect your network traffic and on the fly compare it with network traffic from the past, to see, if there is something suspicious going on or not.
Don’t get me wrong. I’m not saying that a firewall is the perfect use case for SAP HANA, it’s probably not. It’s just an example of how SAP HANA has the potential to be so much more then just the next BI or even the Real-Time Application platform for ERP (see https://www.sap.com/solutions/rapid-deployment/hana-erp-operational-reporting/index.epx). I can see that these are the interesting business cases for SAP and of course I totally understand why they create exactly these type of applications on HANA.
I just think (and this is my fourth thesis), if SAP would make it easier for creative and new groups of users from completely different areas of business, HANA has the potential to not only change SAP Business Suite and the way we do business on a computer. It could actually change the whole computer awareness and experience in every aspect of our daily life.
It actually also has a bit frightening possible use case. The old ghost of echelon () could become usable for intelligence services, because with Sentiment Intelligence HANA can actually filter and recognize human communication and make it analyzable for an algorithm. I have to say, this is probably still a bit science fiction, but I wanted to point out, that, as with every really inventive technology, you can make use of it in good or bad ways.
Now it’s your turn, go ahead, write a comment or send me an email to [email protected]. Tell me if you think I’m on the completely wrong track, or, of course even better, if you agree and you’d like to engage in a creative discussion on what could be possible with SAP HANA.
Greetings to all of you and especially to the inventors of SAP HANA (Great Kudos),