...
- Usual configuration - Points to primary instance.
- Primary Instance - http://prod-opsdb01.geant.net/,
- Secondary Instance - http://prod-opsdb02.geant.net/
OpsDB runs on two VMs on each environment (Prod, UAT and Test)
...
- Check if the primary instance available by going to: http://prod-opsdb01.geant.net/
- Something wrong with DNS entry (troubleshooting steps)?Suggests DNS issue with opsdb.dante.net - OC should be able to deal with it.
- If Primary instance of OPSDB is not available then check if the secondary instance of the OPSDB available by going at http://prod-opsdb02.geant.net/
- If yes, switch the DNS entry for OPSDB from the ‘Primary(01)’ instance to the ‘Secondary(02)’ instance. This will allow the general user to continue working on OpsDB whist we continue with our investigations as to why it initially went down.
...
If out of hours, log into VCentre (please use win/adm-xxxx account) and check if the VMs are running. If the server can't be pinged :
...
This should start or restart MySQL on the VM – please perform this on both VMs separately.
Check Disk Usage
Is the VM disk full? Is the allocated OpsDB disk space full.Follow the steps here: clean up big files
This should already be being monitored and reported upon if it is becoming full , so this scenario should never occur.
Final Step
Please raise ticket with Software Development Support and include the details of the steps taken out of business hours so that detailed analyses of the failure can be carried out.