Local_listener value getting set automatically to a wrong one...

It was a perfectly fine evening in support, till we got a mail saying client connection to datawarehouse production database is not working..first instinct we thought issue at client end, even a test connection i did from my client went fine to the database. but later other users started reporting, there ETl reports were failing..now we realised we have a problem at our hand.on examination of the alert log we found that local_listener parameter value got set to a wrong value autmatically....puzzling...

we set it back to its original value and issue got resolved..

SR was raised, Oracle confirmed it is a internal bug

bug 11772838 .please refer the note note ID 1384707.1: ORA Agent modify or overwrite LOCAL_LISTENER

How to avoid this issue in further..the note gives a workaround.

set local_listener value in the below format

ALTER SYSTEM SET LOCAL_LISTENER='(ADDRESS=(PORT=<port#>)(PROTOCOL=TCP)(HOST=<host>))'  SID='<instance>';


One thing that puzzled me more is the RAC concept of high availability,obviously we had issue with local_listener value in only 1 node, being a 6 node RAC how was this affecting the client directly. clients are in fact using the scan listener to connect to db.

Database Machine..

Exadata is an engineered system, broadly you have 3 important layers(Database,network and Storage)

I am writing this as and when i am studying the student guide provided in the training class

datawarehouse environment.

I have a live example in my project where we use exadata for dataware house, 

How does exadata help for datawarehouse environment ?

Support large and complex queries, offloads most of query processing to storage layer, which otherwise usually happens in the DB layer buffer cache.

Can manage massive database. utilize resource effectively. Improved compression techniques

OLTP environment.

I just learned in my training class that my company has a telecom client, where they use exadata.

How does exadata hellp for OLTP environment ?

Can support large transaction volume, large i/o  per second.

Caching frequently accessed data.

Fast response times by efficient commit processing.

Consolidating Environment.

Support mixed workloads, and can prioritize workloads. Can dynamically allocate resources based on workloads or system configuration. 

This is interesting...Imagine a bank environment...During business hours they have OLTP load and at EOD they have Batch runs. The system can be set in such a way that OLTP gets maximum resource during the day and during the night BATCH load gets the maximum resource.

Exadata prouds itself in different components and its specification

Exadat storage server - 64 bit intel based Sun Fire server, oracle provides the exadata storage software, this software imparts the database intelligence to the storage and integrates the storage to oracle database and its features.

Exada storage server is also called "Cell", There are 14 Cell in a Exadata full rack machine. Each cell is a computer with memory,cpu,network adapaters and other components normally found in the server. There would be 2 user logins in this cell server, 'celladmin' and 'cellmonitor'. 

The database and the Cell communicate with each other via a high performance infiniband network.

Storage servers are available in 2 configurations

High performance Disk

High Capacity Disk

Official Oracle doucment link here for configuration details 

Guided tour on exadata box  here (interesting).

Skill Set for Exadata Administration

Arup Nanda explains in his article skillset that are required for exadata administration, interestingly he puts storage administration skills 0%, that explains how oracle has revolutionized the storage layer, and bought in a concept of cell administration.


Oracle RAC DBA  -- 60%
Cell administration --20%
System Administrator --15%
Network Administrator --5%
Storage administrator --0%

Oracle RAC DBA -- More importantly expereience in 11gr2 database with ASM(grid infra)
a full rack will come with 8 database server.Cluster can be configured depending on your requirement. you can have 1 cluster with 8 nodes,(for example- 5 production and 3 qa) or you can have multiple clusters.

Cell Administration -- done through "celladmin" user, using utilities such as "cellcli" or "dcli"
a full rack will come with 14 storage servers. there are 12 physical disk in each storage server or cell server.
The concepts here to learn are Cell, physical disks, cell disks, grid disks, and obviously ASM.

Network Administration --Infiniband swtich, full rack has 3 infiniband switch. You can imagine the network layer in between the database servers and storage servers. basic commands for monitoring like ping, rdpping. Can also be monitored through grid control, though at a very basic level i would imagine.

Exadata Training

Though we have a Exadata machine in our project, our role as a administrator is still a normal core database administrator. As a first step in the transition from Database administrator to a database machine administrator, i am attending a 4 day exadata workshop from Oracle, I am on the 3rd day and feel really excited about exadata.

STORAGE is the key to exadata, so oracle managed to build some intelligence into the storage layer, which otherwise was dumb.

 I will point to few expert links here, this link will explain a great deal on exadata for newbee's. Man behind the link, famous name in oracle communities Arup Nanda.