My MariaDB server was killed by the OOM killer due to lack of RAM. I had this issue in the past and never got this problem when trying to restart MariaDB. For some reasons my database was corrupted.
Note: I have a backup, and you should always have one too. But mistakes happen. I will share how you can recover your databases when you don’t have a backup to restore from. I am using MariaDB, but I will be writing it as MySQL for the sake of tutorial.
I had a couple of logs when trying to start MariaDB server
[ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption
[Note] InnoDB: Starting shutdown…
[ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
[Note] Plugin 'FEEDBACK' is disabled.
[Note] Plugin 'wsrep-provider' is disabled.
[ERROR] Unknown/unsupported storage engine: InnoDB
[ERROR] Aborting
[ERROR] InnoDB: Unable to apply log to corrupted page 3 in file ./blog/wp_relevanssi_log.ibd
[Note] InnoDB: Set innodb_force_recovery=1 to ignore corrupted pages.
[ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
[Note] Plugin 'FEEDBACK' is disabled.
[Note] Plugin 'wsrep-provider' is disabled.
[ERROR] Unknown/unsupported storage engine: InnoDB
[ERROR] Aborting
Step 1 – Start MySQL:
You can’t start MySQL server normally, so you need to activate recovery mode by adding innodb_force_recovery=1
to your MySQL config.
Your databases will be in read-only mode. So you cannot use it in production.
In my case,
$ sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf
Restart MySQL
$ sudo systemctl restart mysql
If you still cannot start MySQL, increase the value of innodb_force_recovery
to 2, up to 6. Please increase it one by one because higher number can further corrupting your database. (reference)
Step 2 – Dump your database:
MySQL is running, so now you can backup your database. I have only 1 database to recover, so I just do
$ mysql -uroot -p my_database > my_database.sql
You should dump all database that you want to recover
Step 3 – Reinstall MySQL
Now that you have all your databases dumped, it is time to reinstall MySQL.
To start MySQL normally, you need to drop all corrupted databases. But to do so, you need a running MySQL server to run `DROP` command. If you found a way to remove them without reinstalling MySQL, go for it. Because reinstalling is faster for me than figuring out how to drop databases manually, I am going to reinstall.
Stop and remove MySQL. In my case,
$ sudo systemctl stop mysql
$ sudo apt remove --purge mysql-*
Select YES when being asked to remove all databases.
Then install MySQL back
$ sudo apt install mysql-server
Step 4 – Import dumped databases:
You need to setup your MySQL as usual.
Import all your dumped databases that you made, and you are done.