Daemons and Processes OPROCd Process Monitor provides basic cluster integrity services EVMd Event Management spawns a child process event logger and generates callouts OCSSd Cluster Synchronization Services basic node membership, group services, basic locking CRSd Cluster Ready Services resource monitoring, failover and node recovery LMSn Lock Manager Server process - GCS this is the cache fusion part, it handles the consistent copies of blocks that are tranferred between instances. It receives requests from LMD to perform lock requests. I rools back any uncommitted transactions. There can be upto ten LMS processes running and can be started dynamically if demand requires it. they manage lock manager service requests for GCS resources and send them to a service queue to be handled by the LMSn process. It also handles global deadlock detection and monitors for lock conversion timeouts. LMON Lock Monitor Process - GES this process manages the GES, it maintains consistency of GCS memory in case of process death. It is also responsible for cluster reconfiguration and locks reconfiguration (node joining or leaving), it checks for instance deaths and listens for local messaging. A detailed log file is created that tracks any reconfigurations that have happened. LMD Lock Manager Daemon - GES this manages the enqueue manager service requests for the GCS. It also handles deadlock detention and remote resource requests from other instances. LCK0 Lock Process - GES manages instance resource requests and cross-instance call operations for shared resources. It builds a list of invalid lock elements and validates lock elements during recovery. DIAG Diagnostic Daemon This is a lightweight process, it uses the DIAG framework to monitor the healt of the cluster. It captures information for later diagnosis in the event of failures. It will perform any neccessary recovery if an operational hang is detected. General Administration Managing the Cluster starting /etc/init.d/init.crs start crsctl start crs stopping /etc/init.d/init.crs stop crsctl stop crs enable/disable at boot time /etc/init.d/init.crs enable /etc/init.d/init.crs disable crsctl enable crs crsctl disable crs Managing the database configuration with SRVCTL start all instances srvctl start database -d -o Note: starts listeners if not already running, you can use the -o option to specify startup/shutdown options force open mount nomount stop all instances srvctl stop database -d -o Note: the listeners are not stopped, you can use the -o option to specify startup/shutdown options immediate abort normal transactional start/stop particular instance srvctl [start|stop] database -d -i , display the registered databases srvctl config database status srvctl status database -d srvctl status instance -d -i , srvctl status service -d srvctl status nodeapps -n srvctl status asm -n stopping/starting srvctl stop database -d srvctl stop instance -d -i , srvctl stop service -d -s , -i , srvctl stop nodeapps -n srvctl stop asm -n srvctl start database -d srvctl start instance -d -i , srvctl start service -d -s , -i , srvctl start nodeapps -n srvctl start asm -n adding/removing srvctl add database -d -o srvctl add instance -d -i -n srvctl add service -d -s -r srvctl add nodeapps -n -o -A /network srvctl add asm -n -i -o srvctl remove database -d -o srvctl remove instance -d -i -n srvctl remove service -d -s -r srvctl remove nodeapps -n -o -A /network srvctl asm remove -n OCR utilities log file $ORA_HOME/log//client/ocrconfig_.log checking ocrcheck Note: will return the OCR version, total space allocated, space used, free space, location of each device and the result of the integrity check dump contents ocrdump -backupfile Note: by default it dumps the contents into a file named OCRDUMP in the current directory export/import ocrconfig -export ocrconfig -restore backup/restore # show backups ocrconfig -showbackup # to change the location of the backup, you can even specify a ASM disk ocrconfig -backuploc # perform a backup, will use the location specified by the -backuploc location ocrconfig -manualbackup # perform a restore ocrconfig -restore # delete a backup orcconfig -delete Note: there are many more option so see the ocrconfig man page add/remove/replace ## add/relocate the ocrmirror file to the specified location ocrconfig -replace ocrmirror '/ocfs2/ocr2.dbf' ## relocate an existing OCR file ocrconfig -replace ocr '/ocfs1/ocr_new.dbf' ## remove the OCR or OCRMirror file ocrconfig -replace ocr ocrconfig -replace ocrmirror CRS Administration CRS Administration starting ## Starting CRS using Oracle 10g R1 not possible ## Starting CRS using Oracle 10g R2 $ORA_CRS_HOME/bin/crsctl start crs stopping ## Stopping CRS using Oracle 10g R1 srvctl stop database -d srvctl stop asm -n srvctl stop nodeapps -n /etc/init.d/init.crs stop ## Stopping CRS using Oracle 10g R2 $ORA_CRS_HOME/bin/crsctl stop crs disabling/enabling ## use to stop CRS restarting after a reboot ## Oracle 10g R1 /etc/init.d/init.crs [disable|enable] ## Oracle 10g R2 $ORA_CRS_HOME/bin/crsctl [disable|enable] crs checking $ORA_CRS_HOME/bin/crsctl check crs $ORA_CRS_HOME/bin/crsctl check evmd $ORA_CRS_HOME/bin/crsctl check cssd $ORA_CRS_HOME/bin/crsctl check crsd $ORA_CRS_HOME/bin/crsctl check install -wait 600 Resource Applications (CRS Utilities) status $ORA_CRS_HOME/bin/crs_stat create profile $ORA_CRS_HOME/bin/crs_profile register/unregister application $ORA_CRS_HOME/bin/crs_register $ORA_CRS_HOME/bin/crs_unregister Start/Stop an application $ORA_CRS_HOME/bin/crs_start $ORA_CRS_HOME/bin/crs_stop Resource permissions $ORA_CRS_HOME/bin/crs_getparam $ORA_CRS_HOME/bin/crs_setparam Relocate a resource $ORA_CRS_HOME/bin/crs_relocate Nodes member number/name olsnodes -n local node name olsnodes -l activates logging olsnodes -g Oracle Interfaces display oifcfg getif delete oicfg delig -global set oicfg setif -global /:public oicfg setif -global /:cluster_interconnect Global Services Daemon Control starting gsdctl start stopping gsdctl stop status gsdctl status Cluster Configuration (clscfg is used during installation) create a new configuration clscfg -install upgrade or downgrade and existing configuration clscfg -upgrade clscfg -downgrade add or delete a node from the configuration clscfg -add clscfg -delete create a special single-node configuration for ASM clscfg -local brief listing of terminology used in the other nodes clscfg -concepts used for tracing clscfg -trace help clscfg -h Cluster Name Check print cluster name cemulto -n Note: in Oracle 9i the ulity was called "cemutls" print the clusterware version cemulto -w Note: in Oracle 9i the ulity was called "cemutls" Node Scripts Add Node addnode.sh Note: see adding and deleting nodes Delete Node deletenode.sh Note: see adding and deleting nodes Enqueues displaying statistics SQL> column current_utilization heading current SQL> column max_utilization heading max_usage SQL> column initial_allocation heading initial SQL> column resource_limit format a23; SQL> select * from v$resource_limit; Messaging (tickets) ticket usage select local_nid local, remote_nid remote, tckt_avail avail, tckt_limit limit, snd_q_len send_queue, tckt_wait waiting from v$ges_traffic_controller; dump ticket information SQL> oradebug setmypid SQL> oradebug unlimit SQL> oradebug lkdebug -t Lighwork Rule and Fairness Threshold downconvert select cr_requests, light_works, data_requests, fairness_down_converts from v$cr_block_server; Note: lower the _fairness_threshold if the ratio goes above 40%, set to 0 if the instance is a query only instance. Remastering force dynamic remastering (DRM) ## Obtain the OBJECT_ID form the below table SQL> select * from v$gcspfmaster_info; ## Determine who masters it SQL> oradebug setmypid SQL> oradebug lkdebug -a ## Now remaster the resource SQL> oradebug setmypid SQL> oradebug lkdebug -m pkey GRD, SRVCTL, GSD and SRVCONFIG Tracing Enable tracing  $ export SRVM_TRACE=true Disable tracing  $ export SRVM_TRACE="" Voting Disk adding crsctl add css votedisk deleting crsctl delete css votedisk querying crsctl query css votedisk Pasted from