Want to know The Truth About CPM?

26 September 2013

Essbase futures at Oracle OpenWorld 2013

The Essbase blog post to end all Essbase blog posts

That phrase could mean two different things:
  • I am going to share with you some of the really cool things that Oracle have announced at OpenWorld and so this is just a turn of phrase to show how excited I am by all of this.
  • I am going to share with you some of the really cool things that Oracle have announced at OpenWorld but maybe didn’t want broadcast to world+dog and by doing so I will royally annoy Oracle product management, development, and probably areas of Oracle (hint:  legal) I’ve never heard of, get kicked out of the ACE program, and in despair I give up blogging.

Hopefully this post will fall more into the former than the latter but one never knows. In my possible future defense, I note that there are a bunch of Tweets out there that cover some of this very same stuff and there were people taking pictures at the session, so maybe it’s all okay. I hope.

So why bother?

I didn’t take any pictures (I got in trouble last year but happily my picture taking skills are pretty bad) this year although I saw many doing just that.

What I did take were notes. Somewhat alarmingly, I am actually not half bad (note, I did not say I was good) at typing thanks to my much-missed grandmother who taught high school business in Royal Oak, Michigan. And that meant when we visited her in the summer (Michigan is a great place for a kid) we got to play with her electric typewriter (yes, it was really cool in the 1970s and yes you now have an idea about how old I am) and all of the typing books publishers sent her as samples. And that meant that in those long ago days before video games, I learnt how to touch type. And that means you now understand why my blog posts are so long. Whew.

So with the caveat that I am on the outside looking in, and with the additional HUGE caveat that everything Oracle stated was under safe harbor, i.e., they are thinking about this stuff BUT NO PROMISES and what they present MAY NOT MAKE IT TO THE PRODUCT and the performance MAY NOT COME CLOSE TO WHAT IS PRESENTED I think I can now relate what Gabby Rubin and John Baker showed to the almost completely sold out room.

What’s new in

Yes, this isn’t exactly the future as is here now, but Oracle related some DEVELOPMENT LABS numbers that I thought were pretty exciting. What does Oracle mean by DEVELOPMENT LABS numbers? DEVELOPMENT LABS numbers mean: in development code lines, using incredibly specific test cases, on servers of unknown capacity, such and such results were observed. What I take that to mean is that these numbers are a rough guide to performance but for goodness’ sake, do not take this as gospel – as always with Essbase, your performance may vary quite a bit and that variation may all be on the bad side so test, test, test. And don’t say I didn’t tell you so when it all goes pear shaped.

With that caveat, here are the big things in
  • New BSO functions, especially @CREATEBLOCK and run time Essbase Substitution Variables
  • ASO dimension builds are improved (no percentage given), especially with shared members
  • Better MDX performance
  • XREF performance is hugely improved, sometimes coming very close to intracube performance. NB – These are DEVELOPMENT LABS numbers. Test, test, test.
  • The JAPI has been improved but honestly my highly-touted note taking skills failed me here.

So the big one in is the XREF performance. That means to me that all of those games we play in Planning-land to get XREFs declared as stored (and taking the cost to do the data moves) may no longer be required. And that we can reevaluate data export processes in all kinds of BSO databases to move data across databases – maybe file exports aren’t required any more.

What’s in the future for Essbase, post

A bit of background

Essbase is in kind of a unique position vis-√†-vis other products – it straddles both EPM (my home) and Business Intelligence (my home, one day, maybe). In some respects this is fantastic for those of us who love Essbase because that means it gets funding. Of course there is a flip side that means that functions are sometimes for BI and don’t come over to EPM because it simply doesn’t make sense. At least Oracle thinks it doesn’t make sense where you or I might disagree.

An example of a BI-only function in Essbase is its Essbase Persistent Aggregates – these are totally dynamically generated Essbase databases that are optimized for performance against OBIEE.

And there’s another bit about Essbase that I think most of us aren’t aware of – Essbase is the analytic engine behind Oracle Fusion GL Balances, CRM Territory Management, and Projects Performance. These aren’t Essbase databases we’re supposed to access via Smart View and in fact may make our brains explode as we bring our EPM perspective to how a cube ought to look. Gabby made an analogy that stated that what we might think of as a four dimension model may resolve to 14 or 200 dimensions in Essbase. Yup, that sure isn’t Sample.Basic.

The next 12 months

Again, remember, this is all what Oracle product management would like to do. They don’t know if it will happen. I don’t know if it will happen. Under no circumstances make any business or technology decisions based on what I am going to write. You Have Been Warned.

With that yet again restated absolutely huge, did-you-pay-attention-this-means-you caveat, here’s what I at least think I heard.


Naming restrictions like member name length, reserved keywords, special characters – all of that is a thing of the past. Before you go crazy with joy, remember that when your relational source has crazy ascii characters they now are going to come into Essbase. I am thinking of the Biblical giveth with one hand, taketh away with the other. Maybe a better way to think about this is that flexibility can be a double edged sword.

This will be available first in JAPI and SQL data sources. The whole issue around uniqueness will be addressed at a later date.


Improved performance for queries, calculations, and API performance are all on the roadmap.

Performance (and I guess to some extent features) are going to have a bit of a split between Exalytics and everything else. When a terabyte or two or more of RAM along with 40 or 128 CPU cores are available to Essbase, Oracle know that they can do certain things. When there’s 32 gigabytes of RAM and four cores, Oracle realize that there is much Essbase simply cannot do. That difference in capacity is going to tell, especially with the continuing success of Exalytics. They’re not abandoning the non-Exalytics world but those of us in the “normal” world will just have to look over the wall and sigh in envy.

I mentioned above that Essbase is being driven by BI, EPM, and Fusion. You will see functions and performance come out that may only make sense in one of these environments. This is going to be weird for those of us who have used Essbase since the beginning but again just a fact of life.

Simplified sophistication

Oracle know that we Essbase geeks love to twirl the Essbase dials. This is particularly true in BSO but there is bit-twiddling that is available in ASO as well. That’s all well and good for those of us who enjoy it but their goal is to take the sophisticated tuning we geeks do and make it available to everyone while not losing that option to do the tweaking of the settings. I for one am very curious to see how this works out. Hopefully no more brain-dead Essbase databases will be the result.

Enhanced calc script language

The BSO calc language that we all know and love is viewed as a real product differentiator that makes Essbase the awesome thing it is. Note that this is not MDX. More on this in a bit.

As the BSO calc language is cool, awesome, and generally rocks, it will continue to be enhanced.


If you think about what needs parallelization in Essbase, aggregations are an obvious choice and of course it already exists. But what about all those calcs that use FIXes to “do stuff”? That will be, someday, available as a parallel process as well.

Beyond just member formulas and allocations, DATACOPY, DATAEXPORT, XREFs (this is beyond’s better performance), and XWRITE all will work within FIXPARALLEL.

FIXPARALLEL will work within FIX statements and vice versa. Btw, I wrote this when I heard it: “That is pretty damn cool.” And it is.

Additionally, you will be able to control task dimensions for a division of work that is not dependent on outline order and when specifying the parallelism you will be able to define the number of threads. Within those threads there will be new Essbase variables (a feature I use all the time today) that are initialized within each thread and for each task within a thread.


XRANGE will be, maybe, extended to many more (approximately 30) functions but I couldn’t type them all fast enough in my notes. So we’ll have to wait and see what happens.

Smart View Recorder Services

This is an internal utility that Oracle will provide through OTN so now there will be no support. Essbase Hackers aren’t afraid of that, are we?

Recorder Services is an Application Provider Services add on (I have no idea how it is installed) that can be used as a performance and data benchmarking tool. It records user activity in APS from Smart View and logs it to…disk? Tables? Sorry, I missed that part but I am guessing to disk.

That log can be analyzed or replayed. Imagine what you might do with that. <insert big grin>

MDX improvements

MDX is not the red headed stepchild that is beat on a regular basis. Nope, it’s the key to ASO Essbase and of course is a pretty awesome query language for ASO and BSO.

So what can we expect to see as improvements to MDX? Better optimization for attributes (Does this include varying attributes aka Slowly Changing Dimensions? Dunno.)

Optimized Aggregate command is coming. This is very commonly used in BI queries and there will be significant performance improvements for totals at multi-level hierarchies. Improvements will be based on query and dimension depth.

The big MDX improvement is the concept of Sub Selects. What this means is that in order for Essbase to support the huge Essbase cubes that BI generates, Sub Selects slice out or filter portions of the ASO cube and force operations to only apply to that subset of the overall database. This means that relative performance to the whole cube can be realized. What I take that to mean is that less data equals faster performance. When the required data set for a particular calculation isn’t the whole ginormous ASO database, this could be a very valuable function indeed. Note that relative performance will seem greater the larger the database in question – again this is a case of limiting the scope of the calculation.

Query parity

There is a big difference between data results from SQL versus Essbase. I think of this as the top down nature of SQL versus the bottom up nature of Essbase. By that I mean – for a given data set, if the Essbase outline doesn’t map to all of the fact members, data will get rejected (hello dataload.err). For those of you who read Developing Essbase Applications you will note that I wrote a chapter that forced metadata matching to fact to get round this very issue using Essbase, ODI, and SQL.

Oracle have taken a different approach with what they call the Renegade Member. Instead of forcing the metadata to match the fact, they create default members that get the aggregated results of all of the non-matching fact data. This is huge, to put it mildly as totals shall now always tie.

Of course you are still going to have to figure out what’s missing, but in fact there will be a Renegade Member error log that you can then use to build hierarchy.

You will be able to have one for each dimension if you wish (I can think of some dimensions where it would not be terribly likely) and the suggested good practice is to stick this Renegade Member at the second generation of the dimension.

The big one – Hybrid Essbase

I love BSO Essbase. I love ASO Esbsase. Sometimes BSO is the answer, sometimes ASO is the answer, sometimes it is a real stinker trying to bring the two of them together.

Wouldn’t it be nice to have a database that gave us the flexibility and power of BSO Essbase with the raw power of ASO? That would mean we could continue to do all of the cool block-y things we do in BSO and then not worry so much about what happens when BSO Essbase blows up when the database gets too big.

What the Hybrid Essbase engine does (Slight digression: ASO, BSO, so obviously CSO?) is give you a choice about aggregation. At the base is always the same level zero blocks that BSO Essbase has always supported. Above that, with Hybrid Essbase (HSO?) you have the choice of making some dimensions dynamic (or even all of them if you chose so) or some dimensions dynamic and others stored (so same BSO aggregations with blocks).

This is not going to be like ASO so no Multiple Hierarchies – the dynamic nature is either on or off on a dimension basis.

What is also not like ASO is there is no ASO bitmap but instead (as I sort of understand this) the very best Intelligent Calc (or something akin to it) the world has ever seen. Hybrid Essbase will understand what you are looking at an upper level and will sum the blocks dynamically and do that aggregation very quickly indeed.

You will still have the option of stored upper level members and that will act like a BSO stored dimension today.

Using DEVELOPMENT LABS NUMBERS, Hybrid Essbase pretty much matched ASO Essbase and in some instances beat ASO performance, at least in the case of one development test case. Who knows what the real world will bring but the numbers were very, very, very and did I mention very impressive. How do they do this? Beats me and yeah, when you don’t understand something well it seems like magic. :) I love magic shows.

Just to make this clear (and to reiterate why this is so exciting): Hybrid Essbase has BSO capabilities with ASO dynamic performance. I can hardly wait.

Beyond the next 12 months

Gabby also related what might come out in Essbase beyond the next 12 months but it was so much, so fast, and so tenuous that it simply isn’t worth listing.

Also, that is a teaser to get you to come to Kscope14. :)

And that’s the end

Hopefully just of this post, not my blogging career. The new features are just so exciting I thought I owed the Essbase world at least a peek into what is coming from Oracle, maybe.

I have to say that I wrote this post with some trepidation but I hope my frequent OMG DO NOT BELIEVE ANYTHING I WRITE statements will give you some idea of what’s coming for Essbase while not committing Oracle to much of anything.

Some of this (actually, a lot) was available at Kscope13 but it was super tenuous and Oracle specifically asked us not to write about these future features. The restrictions seem to be (mostly) off and I for one am hopeful that many if not all of these features actually ship, and soon. But I’m not holding my breath and neither should you.

Be seeing you.

25 September 2013

Oracle OpenWorld 2013, day four

Oh noes, we’re halfway done

It is sort of scary how fast conferences go by.  Maybe I shouldn’t be so shocked but wow, attending OpenWorld feels like flying a Super √Čtendard about 50 feet off the deck at 600 knots.  Exciting, in other words.

OpenWorld is a very exciting time, because of all of the glitz that only a firm the size of Oracle could bring and because of the fantastic information sharing and networking opportunities that a conference of this size affords.

So, will Developing Essbase Applications ever have a second edition?

The answer is yes, exact contents to be determined.  I had breakfast with my editor of CRC Press at David’s Delicatessen and it is a go.  Now it’s just up to We Happy Few to figure out what the heck we are going to do with the second edition.  

We’ve done well as far as sales goes, but the Essbase world is changing, and we need to make the book mirror that new landscape.  I have a bunch of ideas (probably too many) and I need to work through what will be new, what will be revised, and what just might get dropped.

I’ll see how many new grey hairs will result.  

Oracle Enterprise Performance Management in the Cloud

Ananth Avva, LiveOps Inc.; David Pooley - Sr. Director - EPM Global Market Development, Oracle;
Jennifer Toomey - Senior Director, Business Analytics Product Marketing, Oracle;
I’m sitting in on the cloud presentation given by Oracle and my buddies at Qubix.  As readers of this blog know, I’ve been a huge fan of EPM in the cloud for development purposes.  

On my last project the client suffered a number of infrastructure issues and we were able to go to the Amazon cloud to run EPM without issue and then migrate back down to their servers when they were available.

But of course I couldn’t guarantee security (I am an Essbase hacker, not an infrastructure hacker) so we very purposely did not bring any real data up to the cloud, did not use any employee metadata (which was a bit difficult as there was a workforce application), and in general tied our hands.  It was better than nothing, in fact a lot better than nothing, but we were constrained in what we could do.

I’d love to be able to go to the client and say, “Hey, Oracle’s got a product, it’s the full deal, and it’s secure.  Let’s get that going for the first phase and the project timeline pressure that hinges on your infrastructure simply goes away.”  Yes, that sure would be nice.  And now it’s possible.
Roger Cressy of Qubix is now speaking.  His comments:
  • Not just Planning in the cloud but big rewrite of the tool.
  • Biggest change to the tool since it came out.
  • Customer benefit is that the dependence on infrastructure at project startup is simply gone, no install, opex vs. capex, and this is full Planning, not a crippled version of the tool.
  • Consulting doesn’t go away (Cameron the consultant is glad to hear this) so functional requirements and design is still there.  Data and metadata extractions, optimizations, calcs, etc. are all still part of the Planning implementation game.
  • But from both a customer and consultant perspective, the big dependencies on infrastructure are gone.

Real-World Performance: What Is Big Data, and What Makes It Big?
Andrew Holdsworth - Senior Director of Real World Performance, Oracle; Kaiyao Huang - Principal
Performance Engineer, Oracle; Michael Hallas - Architect, Oracle;
Trillion row tables?  Hmm, a bit bigger than your typical Essbase data source.  An interesting bit of math is:  if you process a row a second, how long does a trillion rows take?  Answer:  32 years.  Hmm, I have seen slow queries but that would most definitely take the cake.

If I were going to try to summarize this session (which apparently is part three of three), it is how to use set-based SQL along with a bunch of other good SQL practices (which are mostly over my head, sadly) to tackle really big data sets.

Product Development Panel Q&A: Oracle Hyperion EPM Applications

Matthew Bradley - VP, EPM Development, Oracle; Prasad Kulkarni - Senior Director, Financial Planning
Applications, Oracle; Kash Mohammadi - VP of Development, Oracle; Douglas Cosby - VP of Software Development, Oracle; Rajesh Bhatia - VP of Product Development, Oracle;
I am looking forward to this one.  A lot.

And I wasn’t disappointed.  Oracle took all questions with quite a bit of grace – I wouldn’t want to be the St. Andrew of product management.  Nothing was off the table and I for one am glad that Oracle are as open as they are.
I asked the question I seem to always ask at meetings like this – when oh when oh when will we get METAREAD filters in Planning.  The answer is the 32nd day of Never, unfortunately.  The Way Forward is to use Planning data sources and I suppose that makes sense when you want to capture all of the relational goodness and Smart Lists that Planning supports.
So for those of you who persist (like me, for instance) in preferring Essbase as a data source, may I point you to this post here on rolling your own METAREADS from Planning.

There’ll be more

I’ll be (hopefully) doing a better job today with the live blogging.  

24 September 2013

Oracle OpenWorld 2013, day 3

Continuing the coverage, with a break in the beginning

No, I did not sleep in, nor did I over imbibe last night (for the record a glass of soy milk and a cup of tea – I am so boring), but instead I went to the Oracle Publishers Seminar, courtesy of the Oracle Publishers Program.  And I got into that because of OTN’s Oracle ACE program.  Whew.  

Did I actually do anything to deserve being there?  I did, so long as one remembers that I had 12 coauthors and editors that all helped make Developing Essbase Applications possible.  I guess since during the writing of that book I whined the most I get to attend sessions like this.  All kidding aside, because the ACE program got me access to Oracle’s Publishers Program, I was able to be the editor in chief of DEA and that’s really the reason I was there.  

I’d love to tell you more about it, but all of the cool product futures are under NDA.  I did get some very pointed suggestions re books about Business Analytics and I hope to incorporate at least some of them within the second edition of DEA.

What’s New and What’s Coming: Oracle Hyperion Enterprise Planning Suite – InterContinental - InterContinental Ballroom C

Shankar Viswanathan - Director, Product Management, Oracle
Lots of cool stuff coming in Planning.  The highlights are, in outline format and please note that while this is not NDA’d always, always, always KNOW THAT FUTURE FEATURES MIGHT HAPPEN AND THEN AGAIN, MIGHT NOT.  In other words, everything Oracle stated was under safe harbor, i.e., they are thinking about this stuff BUT NO PROMISES and what they present MAY NOT MAKE IT TO THE PRODUCT and the performance MAY NOT COME CLOSE TO WHAT IS PRESENTED.  Are you getting this?  Don’t make buying decisions on future items unless Oracle tell you otherwise.  This blog would not be the place I would risk the price of a cup of coffee.  And I wrote it.  With those hopefully strong enough caveats and qualifications here’s what’s coming in Planning :
What’s new/roadmap
  • PBCS
  • Planning mobile
  • EPM mobile approvals
  • UI & usability improvements
Post 12 months
  • Cloud module
    • Continued ease of use
    • PFP and PSPB in PBCS
  • Analytics from Planning
    • OBIEE Reporting integration
    • Tighter FR integration
  • Missing
    • Full data copy for ASO to BSO and vice versa including relational functionality
    • Trickle feed
    • Sandboxing
  • Out of box contents
    • WFP will be more global
    • Functional starter kit to help kick start projects
  • Enterprise planning
    • EBS Projects integration with PFP
    • Primavera P6 with PFP (not sure about that last bit)
  • Cloud
    • Planning on Oracle public cloud
      • Pure SAAS deployment
    • No need to maintain it
    • Much faster implementation cycles
  • Application diagnostics
    • Diagnose design before rollout
    • Show warnings and errors to make performance efficient
    • Runs on entire app or selected bits
    • Good design practices at design time but in an automated fashion
    • Bad things blocked at run time
    • What’s monitored/blocked/governed
      • Dimensions
      • Business rules, in forms and standalone
      • Forms
      • Planning Units
      • Map Reporting
  • Planning Mobile
    • Mobile browser
    • Optimized for mobile
    • Key end user functionality
  • Smart View for Win 8 Pro
    • Office 2007-2013
    • HSF and SV for Planning

Don’t forget:  NONE of the above may happen.  Or all of it might.  Then again, maybe not.  There’s no way to tell and even Oracle don’t know what might or might not really get delivered.  So if you make a single strategic decision based on what I wrote above…well, you’re even dumber than I am.  Which is saying something.  So take the above with a huge grain of salt and a very general guide to future direction.  And enjoy.

Oracle Business Intelligence Product and Technology Roadmap

Paul Rodwick - VP, Product Management, Oracle;
It’s interesting, or maybe I should write it isn’t so surprising (which, I grant you, is sort of the opposite of interesting) that so many of these slides are the same.  Not a criticism of Oracle but more like presentation proof that the EPM and BI world are merging.  

I think I love it.

Developing Essbase Applications on sale at OpenWorld

OpenWorld runs a bookshop with a focus, not terribly surprisingly, on all things Oracle.

My (and 12 other Happy Few) baby was here.  Glenn MMIC Schwartzberg was kind enough to take a picture of me holding it in the bookstore.  Yes, I was lucky enough to be in the photo, but the responsible parties are:
  • Dave Anderson
  • Joe Aultman
  • John Booth
  • Gary Crisci
  • Natalie Delemar
  • Dave Farnsworth
  • Mike Nader
  • Dan Pressman
  • Robb Salzmann
  • Tim Tow
  • Jake Turrell
  • Angie Wilcox
  • Yr. Obdnt. Srvnt.

With no false humility I can say that this was a group project.  Each and every one of the above list are responsible for the success of our book.

I will blog post OpenWorld something about the sales of our book and maybe give some insight (and solicit your input) on what the second edition might include.

Essbase/EPM Meetup

7:00 to 10 pm
Here’s all you need to know:  

It was a blast, I got to bore Essbase product management to death and who knows, one or two of my whines/ideas/rants might get into the product and oh yeah, you should have been there.  It was a lot of fun meeting old friends and making new ones.  

Here’s some idea of what happened through the miracle of my profoundly poor picture taking skills.

Be seeing you.  Maybe at the 3rd Annual Tim and Cameron’s EPM Meetup?