Recovering MariaDB for Nextcloud (Failed)

以下のようなエラーが出た。

code:log

db_1 | 2020-10-14 13:08:27+00:00 Note Entrypoint: Entrypoint script for MySQL Server 1:10.5.5+maria~focal started.

db_1 | 2020-10-14 13:08:27+00:00 Note Entrypoint: Switching to dedicated user 'mysql'

db_1 | 2020-10-14 13:08:27+00:00 Note Entrypoint: Entrypoint script for MySQL Server 1:10.5.5+maria~focal started.

db_1 | 2020-10-14 13:08:27 0 Note mysqld (mysqld 10.5.5-MariaDB-1:10.5.5+maria~focal) starting as process 1 ...

db_1 | 2020-10-14 13:08:27 0 Note InnoDB: Using Linux native AIO

db_1 | 2020-10-14 13:08:27 0 Note InnoDB: Uses event mutexes

db_1 | 2020-10-14 13:08:27 0 Note InnoDB: Compressed tables use zlib 1.2.11

db_1 | 2020-10-14 13:08:27 0 Note InnoDB: Number of pools: 1

db_1 | 2020-10-14 13:08:27 0 Note InnoDB: Using SSE4.2 crc32 instructions

db_1 | 2020-10-14 13:08:27 0 Note mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)

db_1 | 2020-10-14 13:08:27 0 Note InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728

db_1 | 2020-10-14 13:08:28 0 Note InnoDB: Completed initialization of buffer pool

db_1 | 2020-10-14 13:08:28 0 Note InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().

db_1 | 2020-10-14 13:08:28 0 Note InnoDB: Starting crash recovery from checkpoint LSN=127792882773

app_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.3. Set the 'ServerName' directive globally to suppress this message

app_1 | Wed Oct 14 13:08:28.344866 2020 mpm_prefork:notice pid 1 AH00163: Apache/2.4.38 (Debian) PHP/7.3.18 configured -- resuming normal operations

app_1 | Wed Oct 14 13:08:28.344924 2020 core:notice pid 1 AH00094: Command line: 'apache2 -D FOREGROUND'

db_1 | 2020-10-14 13:08:28 0 ERROR FATAL InnoDB: Trying to read page id: space=0, page number=4294967295 which is outside the bounds of tablespace innodb_system. Byte offset 0, len 16384

db_1 | 201014 13:08:28 ERROR mysqld got signal 6 ;

db_1 | This could be because you hit a bug. It is also possible that this binary

db_1 | or one of the libraries it was linked against is corrupt, improperly built,

db_1 | or misconfigured. This error can also be caused by malfunctioning hardware.

db_1 |

db_1 | To report this bug, see https://mariadb.com/kb/en/reporting-bugs

db_1 |

db_1 | We will try our best to scrape up some info that will hopefully help

db_1 | diagnose the problem, but since we have already crashed,

db_1 | something is definitely wrong and this may fail.

db_1 |

db_1 | Server version: 10.5.5-MariaDB-1:10.5.5+maria~focal

db_1 | key_buffer_size=134217728

db_1 | read_buffer_size=131072

db_1 | max_used_connections=0

db_1 | max_threads=153

db_1 | thread_count=0

db_1 | It is possible that mysqld could use up to

db_1 | Hope that's ok; if not, decrease some variables in the equation.

db_1 |

db_1 | Thread pointer: 0x0

db_1 | Attempting backtrace. You can use the following information to find out

db_1 | where mysqld died. If you see no messages after this, something went

db_1 | terribly wrong...

db_1 | stack_bottom = 0x0 thread_stack 0x49000

db_1 | mysqld(my_print_stacktrace+0x32)0x560333f87152

db_1 | Printing to addr2line failed

db_1 | mysqld(handle_fatal_signal+0x485)0x5603339d3935

db_1 | /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)0x7f268cc5d3c0

db_1 | /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)0x7f268c76418b

db_1 | /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)0x7f268c743859

db_1 | mysqld(+0x62f61b)0x56033369b61b

db_1 | mysqld(+0x6423ba)0x5603336ae3ba

db_1 | mysqld(+0x6434f3)0x5603336af4f3

db_1 | mysqld(+0xdf85b8)0x560333e645b8

db_1 | mysqld(+0xddfc55)0x560333e4bc55

db_1 | mysqld(+0xde198a)0x560333e4d98a

db_1 | mysqld(+0xd873ed)0x560333df33ed

db_1 | mysqld(+0xd92a5a)0x560333dfea5a

db_1 | mysqld(+0x627ef0)0x560333693ef0

db_1 | mysqld(+0xc5f269)0x560333ccb269

db_1 | mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x82)0x5603339d6792

db_1 | mysqld(+0x76d80a)0x5603337d980a

db_1 | mysqld(_Z11plugin_initPiPPci+0x965)0x5603337daae5

db_1 | mysqld(+0x68e60b)0x5603336fa60b

db_1 | mysqld(_Z11mysqld_mainiPPc+0x435)0x5603337001b5

db_1 | /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)0x7f268c7450b3

db_1 | mysqld(_start+0x2e)0x5603336f4c1e

db_1 | The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains

db_1 | information that should help you find out what is causing the crash.

db_1 | Writing a core file...

db_1 | Working directory at /var/lib/mysql

db_1 | Resource Limits:

db_1 | Limit Soft Limit Hard Limit Units

db_1 | Max cpu time unlimited unlimited seconds

db_1 | Max file size unlimited unlimited bytes

db_1 | Max data size unlimited unlimited bytes

db_1 | Max stack size 8388608 unlimited bytes

db_1 | Max core file size unlimited unlimited bytes

db_1 | Max resident set unlimited unlimited bytes

db_1 | Max processes unlimited unlimited processes

db_1 | Max open files 1048576 1048576 files

db_1 | Max locked memory 65536 65536 bytes

db_1 | Max address space unlimited unlimited bytes

db_1 | Max file locks unlimited unlimited locks

db_1 | Max pending signals 31401 31401 signals

db_1 | Max msgqueue size 819200 819200 bytes

db_1 | Max nice priority 0 0

db_1 | Max realtime priority 0 0

db_1 | Max realtime timeout unlimited unlimited us

db_1 | Core pattern: core

db_1 |

db_1 | Fatal signal 11 while backtracing

Help me please!, MariaDB from Nextcloud crashes and I just don't know why - support - Nextcloud community からリンクされている、InnoDB Recovery Modes - MariaDB Knowledge Base を見てやってみる。

とりあえず recovery mode 1 から始めると良いと書いてあるので、

MySQL :: MySQL 5.6 リファレンスマニュアル :: 14.19.2 InnoDB のリカバリの強制的な実行

のように、 cnf ファイルに

code: my.cnf

mysqld

innodb_force_recovery = 1

のように書くというのがたくさん出てくるが、どこに書くのかわからない

以下の場所にあった。 ~/.my.cnf に書けば良さそう (db が docker-compose.yml で設定している service 名)。

code:bash

$ docker-compose run db cat /etc/mysql/my.cnf

# The MariaDB configuration file

# The MariaDB/MySQL tools read configuration files in the following order:

# 0. "/etc/mysql/my.cnf" symlinks to this file, reason why all the rest is read.

# 1. "/etc/mysql/mariadb.cnf" (this file) to set global defaults,

# 2. "/etc/mysql/conf.d/*.cnf" to set global options.

# 3. "/etc/mysql/mariadb.conf.d/*.cnf" to set MariaDB-only options.

# 4. "~/.my.cnf" to set user-specific options.

# If the same option is defined multiple times, the last one will apply.

# One can use all long options that the program supports.

# Run program with --help to get a list of available options and with

# --print-defaults to see which it would actually understand and use.

# If you are new to MariaDB, check out https://mariadb.com/kb/en/basic-mariadb-articles/

# This group is read both by the client and the server

# use it for options that affect everything

client-server

# Port or socket location where to connect

# port = 3306

socket = /run/mysqld/mysqld.sock

# Import all .cnf files from configuration directory

!includedir /etc/mysql/conf.d/

!includedir /etc/mysql/mariadb.conf.d/

root user で実行していて home directory は /root なので、そこに書こう。

以下のように、 mariadb の image の entry point は mysqld を実行するだけみたい。

code: bash

$ docker image inspect mariadb

...

"Cmd": [

"/bin/sh",

"-c",

"#(nop) ",

"CMD \"mysqld\""

...

"Entrypoint": [

"docker-entrypoint.sh"

...

結局以下のようにしたけど起動しなかった。

code: docker-compose.yml

db:

image: mariadb

restart: always

volumes:

- db:/var/lib/mysql

environment:

- MYSQL_ROOT_PASSWORD=xxxxxx

- MYSQL_PASSWORD=xxxxx

- MYSQL_DATABASE=nextcloud

- MYSQL_USER=nextcloud

command: bash -c 'printf "mysqld\ninnodb_force_recovery = 3\ninnodb_purge_threads =0\nuser = root\n" > /root/.my.cnf && mysqld'

Running mysqld as root - MariaDB Knowledge Base

innodb_force_recovery = 4 まではエラーメッセージが変わらなかったが、5 にしたら以下のようになり起動した (この error は以前から出ていたもののような気がする)。

code:log

Attaching to nextcloud_db_1, nextcloud_app_1

db_1 | 2020-10-25 7:23:58 0 Note mysqld (mysqld 10.5.5-MariaDB-1:10.5.5+maria~focal) starting as process 1 ...

db_1 | 2020-10-25 7:23:58 0 Warning option 'innodb-purge-threads': unsigned value 0 adjusted to 1

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Using Linux native AIO

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Uses event mutexes

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Compressed tables use zlib 1.2.11

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Number of pools: 1

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Using SSE4.2 crc32 instructions

db_1 | 2020-10-25 7:23:58 0 Note mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Completed initialization of buffer pool

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Starting crash recovery from checkpoint LSN=127792882773

db_1 | 2020-10-25 7:23:58 0 Note InnoDB: Starting final batch to recover 364 pages from redo log.

db_1 | 2020-10-25 7:23:59 0 Note InnoDB: Removed temporary tablespace data file: "ibtmp1"

db_1 | 2020-10-25 7:23:59 0 Note InnoDB: Creating shared tablespace for temporary tables

db_1 | 2020-10-25 7:23:59 0 Note InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...

db_1 | 2020-10-25 7:23:59 0 Note InnoDB: File './ibtmp1' size is now 12 MB.

db_1 | 2020-10-25 7:23:59 0 Note InnoDB: 10.5.5 started; log sequence number 127793058837; transaction id 0

db_1 | 2020-10-25 7:23:59 0 Note InnoDB: !!! innodb_force_recovery is set to 5 !!!

db_1 | 2020-10-25 7:23:59 0 Note InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool

db_1 | 2020-10-25 7:23:59 0 Note Plugin 'FEEDBACK' is disabled.

db_1 | 2020-10-25 7:23:59 0 Note Server socket created on IP: '::'.

db_1 | 2020-10-25 7:23:59 0 Warning 'proxies_priv' entry '@% root@0ad86da7bf7a' ignored in --skip-name-resolve mode.

db_1 | 2020-10-25 7:23:59 0 Note InnoDB: Buffer pool(s) load completed at 201025 7:23:59

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._db'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._transaction_registry'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._time_zone_transition'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._general_log'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._plugin'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._help_keyword'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._columns_priv'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._help_topic'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._gtid_slave_pos'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._time_zone'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._time_zone_transition_type'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._host'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._servers'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._help_relation'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._event'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._index_stats'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._user'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._table_stats'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._func'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._innodb_table_stats'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._tables_priv'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._help_category'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._column_stats'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._proxies_priv'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._procs_priv'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._roles_mapping'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._slow_log'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._time_zone_leap_second'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._proc'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._time_zone_name'

db_1 | 2020-10-25 7:23:59 1 ERROR Invalid (old?) table or database name '._innodb_index_stats'

db_1 | 2020-10-25 7:23:59 1 ERROR InnoDB: Failed to find tablespace for table mysql.gtid_slave_pos in the cache. Attempting to load the tablespace with space id 4

db_1 | 2020-10-25 7:23:59 0 Note Reading of all Master_info entries succeeded

db_1 | 2020-10-25 7:23:59 0 Note Added new Master_info '' to hash table

db_1 | 2020-10-25 7:23:59 0 Note mysqld: ready for connections.

db_1 | Version: '10.5.5-MariaDB-1:10.5.5+maria~focal' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution

この状態で Nextcloud の Web client から access すると、

code:log

db_1 | 2020-10-25 7:27:01 3 ERROR InnoDB: Failed to find tablespace for table nextcloud.oc_appconfig in the cache. Attempting to load the tablespace with space id 8

db_1 | 2020-10-25 7:27:03 4 ERROR InnoDB: Failed to find tablespace for table nextcloud.oc_storages in the cache. Attempting to load the tablespace with space id 9

db_1 | 2020-10-25 7:27:03 4 ERROR InnoDB: Failed to find tablespace for table nextcloud.oc_mimetypes in the cache. Attempting to load the tablespace with space id 11

db_1 | 2020-10-25 7:27:05 11 ERROR InnoDB: Failed to find tablespace for table nextcloud.oc_jobs in the cache. Attempting to load the tablespace with space id 40

db_1 | 2020-10-25 7:27:09 12 ERROR InnoDB: Failed to find tablespace for table nextcloud.oc_users in the cache. Attempting to load the tablespace with space id 50

db_1 | 2020-10-25 7:27:09 12 ERROR InnoDB: Failed to find tablespace for table nextcloud.oc_preferences in the cache. Attempting to load the tablespace with space id 16

のように error が出ている。よくわからない。data が消えたわけではないのでこの辺であきらめて DB 作り直すかな。

Move Nextcloud files のように backup した file を新たな Nextcloud server に追加するのは簡単だった。

#Nextcloud #MySQL