I hit the error below during startup of my 10.2.0.1 RAC installation running on Solaris X86-64 on VMWare

ORA-00600: internal error code, arguments: [kcratr1_lastbwr], [], [], [], [],[], [], []

Basically none of the instances start-up with srvctl command and log this error to alert log.

Tried to run manual startup of instances but no chance same error.

Quick search on MOS (yes it was working ) take me to the note 393984.1 (no bug number)

It says this is a bug on Enterprise Edition – Version: 10.2.0.1 to 11.1.0.6.0

My Oracle RAC had the problem, so I investigated... I found:

Top 5 Timed Events Avg %Total

~~~~~~~~~~~~~~~~~~ wait Call

Event Waits Time (s) (ms) Time Wait Class

Preparing a Linux box for Oracle Clusterware and RAC installation is made fast and simple with the help of the Oracle Validated RPM. Downloading and installing the RPM that matches you Linux Kernel automates the install of numerous RPMs required to setup OCW and RAC on Linux.

The RPM is available to the public. Note 728346.1 contain general information about it.

This document will run through the installation of the Oracle Validated RPM on Oracle Enterprise Linux 5.

I’ve taken my first tentative steps into 11gR2 RAC and it was a big surprise.

•  
• Oracle 11g Release 2 RAC On Linux Using VMware Server 2

11gR2 RAC feels very different to 11gR1 RAC. I can imagine quite a few people wanting to upgrade from 11gR1 thinking it will be trivial and getting a rude awakening…

I just wanted to share few metalink notes on Diagnosing RAC Issues across various platforms. Oracle Support has recently few interesting metalink notes on how to troubleshoot the various layers in a RAC Install i.e Storage, Network, OS, Database, Listener, CRS and ASM Components.

The document also lists various whitepapers/metalink articles on troubleshooting IPMP, RAC Node evictions, Performance Tuning CRS on various platforms

I have been given with a challenging task to convert one of our critical production databases, which is of 1 TB (Terabyte) in size, to Oracle 10g RAC with ASM storage option. Even though, there are many methods and tools available to perform this activity, I have preferred to use the RCONFIG tool.

We prepared an input XML file required for RCONFIG tool, and run the RCONFIG utility as follows:
 

Welcome back to our videocasts — this is the last video in the series about Oracle RAC Virtual IP’s. I strongly suggest to watch the two previous videos:

To read the entire article at its source, please refer to Pythian Video: Oracle RAC VIP’s — Troubleshooting Connectivity Issues

We just conducted an extremely successful Oracle RAC with Enterprise Manager Grid Control workshop at my company S&I Systems Singapore, jointly with Oracle. For details of this and future workshops, please see:

http://www.si-asia.com/marketing/oracleworkshops2009/

Hi Babu,

Thank you for the tip. Looks like I created the clone of the guest OS a little too soon!

I will try this approach tomorrow and let you know.

Cheers!
KV

To read the entire article at its source, please refer to 10g RAC configuration on Solaris 10

Many times I tried to kill session in RAC  and then get message back saying that session cannot be killed because my session is connected to different instance. Then I would login to instance where blocking sessions runs  and kill it.

Starting with 11g , Oracle included instance name in alter system kill command (3thd parameter or  @instance_id ) .So to kill blocking session on any node from any node I would run:

SQL> set serveroutput on
SQL> exec kill_blocker;

At the moment for a customer I am investigating whether it's a good idea to use "Jumbo Frames" within a RAC environment. In the next couple of days I will post some results here.

Until that time, I like to share some thoughts.

First: why do you possibly want "Jumbo Frames" ?

To read the entire article at its source, please refer to Jumbo Frames for RAC Interconnect

The Oracle 11g DRCP connection pooling "connection broker" process handles incoming connection requests. It also retains a socket for idle connections, thus enabling faster reconnection. Large web applications with multiple mid-tier Apache servers typically need to increase the database server OS limit on file descriptors so the broker can receive all the connections and work to full capacity.

HugePages and Oracle RAC

• HugePages is a feature integrated into the Linux kernel with release 2.6. This feature basically provides the alternative to the 4K page size providing bigger pages.
• HugePage sizes vary from 2MB to 256MB based on kernel version and H/W architecture.
•  

 Why HugePages Should Be Used on RAC with 64-bit Linux?

• Very Large Memory: The 64-bit systems do have VLM. So the general VLM issues apply.
• Not swappable: HugePages are not swappable.
  

I always forget how to resolve those shortcuts and which one does what, especially CSS versus CRS

11/04/2009:
1) Finally got some time for cleaning up Grid Control (dropping ora2 and ora3). Secured all agents (on VMs: gc, prac1, prac2). I’ve also cleaned up XEN dom0 (from quadvm). These VMs are not needed anymore. db3.lab (RAC on prac1, prac2) is in GC. Installed 10.2.0.5 32-bit agent on srac1 (single node standby).
2) Testing application of single-node RAC standby for differences in Standby Redo Logs processing (verifcation performed by using read-only mode).
3) LNS (ASYNC=buffers_number in LOG_ARCHIVE_DEST_2 parameter) performance fun.

The RAC SIG, Oracle and IOUG are thrilled to present the hands-on event dubbed “RAC Attack!” at

• RAC
• Events

Sequence values are cached in instance memory and by default 20 values are cached. As instance cache is transient, loss of an instance can result in loss of cached sequence values. Permanent record of highest possible value from any instance is kept track in SEQ$ table.

SQLs accessing this sequence within an instance will access instance SGA. As each instance caches its own sequence values it is highly likely that SQLs accessing this sequence from different instance will create gaps in sequence values.

Those of us that have dealt with RAC environments for a while are familiar with the behavior of Oracle Services in an Oracle Cluster. Services are an essential component for managing workload in a RAC environment. If you’re not defining any non-default services in your RAC database, you’re making a mistake. To learn more about services, I strongly recommend reading the definitive whitepaper by Jeremy Schneider on the topic.

There’s a recent thread in Oracle-L about deadlocks and a recommendation to dump various instance information when the deadlock happens. A deadlock trace dumps some useful things automatically, but sometimes you want more, especially in RAC environment.

So is it possible to make Oracle dump additional things when the deadlock event happens? Yes it is and it’s doable with Oracle diagnostic event handling infrastructure.

I was able to proceed with VIPCA and NETCA configuration successfully, but stuck with the below error while running DBCA

ATLANTA, Feb 09, 2009 /PRNewswire via COMTEX/ -- Reliance Trust Company is one of the leading investment firms in the country providing personalized investment, wealth management, Investment Services and financial services to individual, corporate and institutional clients throughout the country for over 30 years.