I will be available on 1 December, 2014. Special projects forever! Or something like that. Contact me on LinkedIn if you're interested.

07 April 2014

EPM 11.1.2.3.500 Patching Woes, Trials, and Tribulations

A warning
Don't take infrastructure advice from me.  Ever.  That begs the question, “Why are you reading a blog post on infrastructure by yr. obdnt. srvnt.”?  Simply because tales of suffering, woe, and misadventure are always instructive and sometimes entertaining.  Read the below to know that when I write this I state the truth. 

The following workaround fixed my issue, but I make no warranty that it will apply to anything other than my environment.  With that huge caveat, off we go...
Why I am here
As most of you know, I am a serially challenged (I do it again and again and I fail again and again) infrastructure idiot.  However, needs must when the devil drives, and no one else is going to install the 11.1.2.3.500 patch for me, so....

I spent most of the last weekend in March struggling with that EPM 11.1.2.3.500 17529887 patch.  Much of the issue was that Planning simply didn't work.  My compact deployment started up all right, but when I tried to enter a Planning app, nothing.  And by nothing I mean I could not edit dimensions, forms, etc., etc., etc.  Bummer.

Help is on the way
In desperation, I reached out to John Booth (if a customer needs infrastructure work, I am not sure why he would use anyone else – for sure you shouldn't hire me) and described my system symptoms.  John asked if the ADF patches 16964825 and 18362693 had been applied. 

I think they are supposed to be applied automatically as part of that big (2 gigabyte) patchset but it didn't (apparently) happen. 

I tried applying those patches from the normal C:\Oracle\Middleware\EPMSystem11R1\OPatch location.  They failed.  I reached out to John again (not many would take calls on a Sunday -- thanks, John) and he explained to me that there are two opatch locations.  The ADF patches, because they are at a base level of Fusion (I think I have this right), are applied at C:\Oracle\Middleware\oracle_common\OPatch.  I applied the patches and now Planning mostly worked.

Here are the screenshots (the second one is important because there is a twist) of the patches being applied.

16964825


18362693


Did you see the special bit of the patch here?  Don't just patch 18362693, patch 18362693\oui.  From John I know that "oui" in this context is not "Yes" in French, but instead Oracle Universal Installer.

Did it work?  Yes and no.

The issue

I could now get into Planning via IE.  Terrific.  But Planning via Smart View...it was interesting.  What's wrong with the below screenshot?

Take a look at it in IE:

Yet Another Infrastructure Moment of Pain or YAIMP.

The fix

What I did was roll everything back (thank you VMWare and thank you Cameron for being the cautious geek you usually are) via VMWare’s snapshot functionality to a prepatch environment.

I then applied those two ADF patches before I did anything else in that C:\Oracle\Middleware\oracle_common\OPatch location.

I then applied the rest of the .500 patches in the normal C:\Oracle\Middleware\EPMSystem11R1\OPatch location.  Opatch tried to apply the two ADF patches (it is built into the 17529887 patch as far as I can tell) and aborted their reapplication, but continued on with the rest of the patching process.

I then crossed my fingers.  Did it work? 

Why yes it did. Whew.

The clew I should have knew

Were I more eagle eyed, I would have seen this in my initial failed install:

That would be the two required ADF patches going KABOOM.  Did I see it?  Nope.  Dumb, dumb, dumb on my part.  As my teachers used to say back in school, “A smart boy, but does not pay attention to detail.  Must do better.”   It is nice (?) that my fundamental personality traits are constant through time.

Even dumber is that this is explicitly noted in Oracle Support KB article Issues Using Planning after Applying EPM Patch 11.1.2.3.500 (Doc ID 1640411.1).  Although I swear that when I looked for some clew (or clue) to my issue I couldn’t find it.  Sigh.

The cause

So why didn’t it work?  Most likely because I didn’t have my 11.1.2.3.000 environment set up quite right.  Careful reading (oh sure, now I do this) of the Great and Good John Goodwin’s post on 11.1.2.3.500, rewards the faithful reader with this important note:
One thing is nice is that applying the patch looks to automatically install the required ADF patches in to oracle_common home. (if you have opatch added to the path variable that is).

Looking at my PATH environment variable I see:

No reference to C:\Oracle\Middleware\oracle_common there and I think that is why the whole thing didn’t work.

The end

I think I have ten new grey hairs from all of this.  And if you wonder why I suffered through this
process, I have two Kscope14 sessions riding on a .500 environment.  It was either do or die.  Luckily this time I lived.  Again, I would never have gotten this far without John Booth's help.  Thanks, John!

Be seeing you.

Cameron

16 comments:

Anonymous said...

Thank you for the post Cameron. After installing the patch successfully(as per the log) I still see the essbase version to be the same as my 11.1.2.3 setup
Should I be seeing a different version number like 11.1.2.3.500 or something. All other apps like Planning, WS etc show .500. But essbase still shows the same version(both in EAS and also when I launch maxl on the server where I have .500 installed). let me know

Cameron Lackpour said...

This patch is just the EPM application servers -- not Essbase, not EAS, not APS, etc.

See the last post for the Essbase patches:
http://camerons-blog-for-essbase-hackers.blogspot.com/2014/04/getting-started-with-hybrid-essbase.html

Regards,

Cameron Lackpour

Anonymous said...

I am confused. The installation log says that a patch was applied to essbase as well
Even in your screenshot I see "patching essbase server"
Just to be sure can you please let me know what version number do you see when you Edit properties -> License tab in EAS.

Dinesh said...

Hi Cameron;

Regular visitor of your blog. I patched my system this Sunday. It went well without any issue. Started exploring Hybrid Essbase :-) . Thank You for sharing your wealth of knowledge and experience with others!

Regards,
Dinesh

Dinesh said...

Hi Cameron;

Regular visitor of your blog. I patched my system this Sunday. It went well without any issue. Started exploring Hybrid Essbase :-) . Thank You for sharing your wealth of knowledge and experience with others!

Regards,
Dinesh

Henri Vilminko said...

Hi,

I couple of comments about the patch... The statement "Remove previous unzipped content for 16964825 - The system cannot find the file specified." is perfectly normal and is produced because a script just tries to clean up the OPatch directory in case the ADF patches were previously extracted to ...\oracle_common\OPatch.

But then you should see OPatch extracting the ADF patches:

"[unzip] Expanding: D:\Oracle\Middleware\EPMSystem11R1\OPatch\p16964825_111170_Generic.z
ip into D:\Oracle\Middleware\oracle_common\OPatch"

And finally applying both of them:

"Applying ADF 11.1.1.7.1 OPatch 16964825"

The process completed without issues for me. I checked that I don't have either of the OPatch directories in my %PATH% so that should not be required. But looking at the last lines in your big screenshot I noticed something odd - the Middleware home directory is shown using the shortname notation "MIDDLE~1". As mentioned above the same message for me looked like:

"... into D:\Oracle\Middleware\oracle_common\OPatch"

I wonder if this could have been caused by the way you ran opatch (using the %epm_oracle_home% variable)?

Cameron Lackpour said...

Henri,

When it comes to me + infrastructure, anything is unfortunately possible.

I just barely know enough to get the stuff working (although I continue to suffer weird Smart View moments where Excel just freezes up).

It is better than it used to be -- I can remember trying to install System 9.01 and finally just giving up.

Regards,

Cameron Lackpour

Addison Augustin said...

I found all the above interesting. As i ran into the same problem. In one instance all i did was rollback the patch and reapply it and it all worked. In another environment I'm setting up I ran into the same problem but the rollback and reapply did not work. When i try installing the ADF patches i get the following message.
"Patch "16964825" is not needed since it has no fixes for this Oracle Home."
So for some reason, i cannot get this patch to apply no matter what i try. I even tried putting the OracleCommon directory in the path. The patch set gives me an error while trying to apply the ADF section.

Vijay said...

Hi Cameron, When we rolled back everything and tried to install the ADF patches manually, we got an error saying the patches are not needed. Did you get this error ? SOunds like it is not allowing us to install the adf patches manually.

Cameron Lackpour said...

Vijay,

I wonder if your issue is that you already have ADF patches applied to your machine. Oracle Support KB Doc ID 1640411.1 states that if existing ADF patches exist, they must be uninstalled *before* the second set of ADF patches happen.

And don't forget the \oui folder in the patch for 18362693.

Regards,

Cameron Lackpour

Vijay said...

Cameron,

The patchset .500 worked. We had to delete the tmp directory from all servers in Domains folder and started the apps again. This fixed the planning issue. No manual steps to install/uninstall the ADF patches required.

Sarah said...

Hi Cameron,

Came across a .500 bug that I thought I would share. In Workspace, if you have a dimension in an application shared to the Shared Library and you have custom Top Members and/or exclusions, Workspace could act funny.

I have an Accounts dimension that has custom top members that differ from the Shared Library. Once I choose to add a new top member, exclude a new member, etc, the full hierarchy will show in Workspace and not the filtered view. Once you right-click on the dimension name and choose "Select Top Members", only the ones you had originally chosen show...not the full list as is shown in Workspace. However, once you click cancel in the "Select Top Members" popup and click the Refresh button at the top of the Workspace screen, it goes back to normal.

Weird.

They need a patch to fix the patch.

SarahZ

Nathan said...

Hi Cameron,

Many thanks for this blog entry. I have just applied the .500 patch. I did not apply the ADF patches beforehand. Planning worked but I experienced the same behaviour in Smart View as you described. I restored the cloned VM and applied the ADF patches before the .500 patch but this still did not resolve the problem as it did for you. Do you have or are you aware of any further pointers?
Thanks, Nathan

Cameron Lackpour said...

Nathan,

Sorry, no I am not aware of any other way round this.

I *think* there are some Support KB articles on this but I'm afraid I don't know them off the top of my head.

They should however be pretty easy to find.

Regards,

Cameron Lackpour

Brian said...

I'm with Nathan. I just went through this about 5 times with no luck. I've tried applying the patches before and after and not at all. No luck. I feel like I must be missing something simple.

Nathan said...

Hi Brian,

I just revisted the blog entry as I did not patch Essbase to .500. The issue discussed I resolved. The Smart View client needs to be the latest one, namely 11.1.2.5.210.

Thanks,

Nathan