MySQL 探索記 ~ Unit Test がビルドされる時に必要なライブラリとリンク~

はじめに
前回との差分
- 余談
本題
おわりに

はじめに

どうも、ユニットテストをかけるようになったぜと前回の記事で喜んでいたらバイナリの書き込みで盛大に 1byte ずれていることに気がついてしまったけんつです。
rabbitfoot141.hatenablog.com

最近、MySQL ごとビルドする場合に unittest/ 以外のディレクトリでユニットテストをサポートする方法について書いたが、いざ自分が作っているプラグインをビルドしてみると link 周りでコケることが分かったのでその原因についてまとめる。

前回との差分

前回の記事で gunit_large, server_unittest_library の２つがユニットテストをサポートする上で重要なライブラリであるという話を書いたが、それは間違いではなくそのまま。
問題は MYSQL_ADD_PLUGIN で指定する plugin_args にあった。ここに特定のキーワードが含まれる場合とそうでない場合で上記のライブラリにリンクされるかどうか結果が変わってくることが分かったというのが今回ここでする話である。

余談

gunit_large, server_unittest_library という２つの必要なライブラリが存在するが、 gunit_large は gtest を動かす上で必要なラッパーライブラリ的な側面を持っており、server_unittest_library というのは mysql-server に関連するユニットテストの実行に必要なライブラリ群であるというの間違いないと思われる。

本題

前回の記事で紹介した手順にしたがってビルドをしていくと、MYSQL_ADD_PLUGIN の引数次第では gtest を含むユニットテストをビルドした場合に「undefined reference to」という見慣れたエラーが出てくる場合がある。
というわけでやや適当に書いてしまった前回記事から更に少し調べる必要に迫られたというわけである。

一度冷静になる

gunit_large の役割については前述の通りであるというので間違いないと思われるので、問題は servier_unittest_library をリンクするあたりにあるということがわかる。

というわけで今一度 server_unittest_library のビルドについて調べる。

 MERGE_LIBRARIES_SHARED(server_unittest_library SKIP_INSTALL LINK_PUBLIC
      sql_main
      ${MYSQLD_STATIC_PLUGIN_LIBS}
      minchassis
      ext::icu
      # Import some core symbols. Other symbols needed by the unit test
      # executables are pulled in transitively by symbol dependencies.
      #
      # Since everything has visibility("default") the library will
      # export every symbol pulled in from the source libraries.
      #
      # If some symbols are still missing, they will be picked up from
      # dependent libraries, since we LINK_PUBLIC.
      # To see what symbols we need to import, remove LINK_PUBLIC above.
      #
      # The strings library uses visibility=hidden for all symbols,
      # except those explicitly tagged with MYSQL_STRINGS_EXPORT.
      # If we get ODR violations for executables using server_unittest_library,
      # it means the symbol has been found in strings and
      # server_unittest_library, which means the unit test is using
      # some non-exported symbol from strings.
      EXPORTS
      builtin_perfschema_plugin            # Pulls in the whole server.
      mysql_service_mysql_rwlock_v1        # Pulls in minchassis
      )

https://github.com/mysql/mysql-server/blob/ea1efa9822d81044b726aab20c857d5e1b7e046a/CMakeLists.txt#L2249-L2273

sql_main などは置いておいて、今ここで一番怪しそうなのは MYSQL_STATIC_PLUGIN_LIBS である。というかどうみてもそれぐらいしか可変であると思われるものはない。

頑張って実装を追う

ここで更に冷静になって、MYSQL_ADD_PLUGIN を読み直す。

...
# Update mysqld dependencies
SET (MYSQLD_STATIC_PLUGIN_LIBS ${MYSQLD_STATIC_PLUGIN_LIBS} 
${target} ${ARG_LINK_LIBRARIES} CACHE INTERNAL "" FORCE)
...

https://github.com/mysql/mysql-server/blob/ea1efa9822d81044b726aab20c857d5e1b7e046a/cmake/plugin.cmake#L158-L160

どうやらここに到達させる必要があるというので間違いないと思われる。

というわけで、ここに MESSAGE をつけて cmake を実行してみることにする。

DEFAULT

まずは前回と同じ様に STORAGE_ENGINE DEFAULT な状態。

debug: ARCHIVE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib
debug: BLACKHOLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib
debug: CSV, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson
debug: EXAMPLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib
debug: FEDERATED, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson
debug: HEAP, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson
debug: INNOBASE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson
debug: MYISAM, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library
debug: MYISAMMRG, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson
debug: NDBCLUSTER, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson
debug: PERFSCHEMA, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib
debug: TEMPTABLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson
debug: NGRAM_PARSER, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson;ngram_parser
debug: MYSQLX, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson;ngram_parser;mysqlx;ext::libevent;ext::icu;mysqlxmessages_lite;libprotobuf-lite;extra::rapidjson;ext::lz4;ext::zstd;ext::zlib

これを見るに server_unittest_library にリンクされるものが MYSQL_ADD_PLUGIN の該当箇所を通過するたびに追加されていくという理解で合っていることがわかる。

実際に DEFAULT がついている場合に通過する部分を見るに WITH_${plugin} = 1 にしているので実装とも合っている。

  IF(ARG_DEFAULT)
    IF(NOT DEFINED WITH_${plugin} AND
       NOT DEFINED WITHOUT_${plugin} AND
       NOT DEFINED WITH_${plugin}_STORAGE_ENGINE)
      SET(WITH_${plugin} 1)
    ENDIF()
  ENDIF()

https://github.com/mysql/mysql-server/blob/ea1efa9822d81044b726aab20c857d5e1b7e046a/cmake/plugin.cmake#L72-L78

というわけで次にこの部分に到達するために必要な IF を見る。

  # Build either static library or module
  IF (WITH_${plugin} AND NOT ARG_MODULE_ONLY)

https://github.com/mysql/mysql-server/blob/ea1efa9822d81044b726aab20c857d5e1b7e046a/cmake/plugin.cmake#L146C1-L147C46

WITH_${plugin} が true で MODULE_ONLY でない場合に到達するらしい。
MODULE_ONLY は引数で渡す場合にその shared library のみを作成してくれるもので、これを指定していると LINK_LIBRARIES の段階でエラーとなるので今回は関係ないといえば関係ないが、ユニットテストをサポートするときには必要ないだろう。
問題はその他である。WITH_${plugin} が true 相当にならないといけないので、その周辺を見ていく。

WITH_"${plugin}"_STORAGE_ENGINE

まずは -DWITH_EXAMPLE_STORAGE_ENGINE=1 にして DEFAULT を外した場合。

debug: ARCHIVE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib
debug: BLACKHOLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib
debug: CSV, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson
debug: EXAMPLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib
debug: FEDERATED, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson
debug: HEAP, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson
debug: INNOBASE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson
debug: MYISAM, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library
debug: MYISAMMRG, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson
debug: NDBCLUSTER, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson
debug: PERFSCHEMA, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib
debug: TEMPTABLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson
debug: NGRAM_PARSER, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson;ngram_parser
debug: MYSQLX, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson;ngram_parser;mysqlx;ext::libevent;ext::icu;mysqlxmessages_lite;libprotobuf-lite;extra::rapidjson;ext::lz4;ext::zstd;ext::zlib

このパターンは MYSQL_ADD_PLUGIN の実装を見ても WITH_${plugin} = 1 を設定しているので、実装と合っている。

  IF(WITH_${plugin}_STORAGE_ENGINE 
    OR WITH_{$plugin}
    AND NOT WITHOUT_${plugin}_STORAGE_ENGINE
    AND NOT WITHOUT_${plugin}
    AND NOT ARG_MODULE_ONLY)
     
    SET(WITH_${plugin} 1)

https://github.com/mysql/mysql-server/blob/mysql-8.0.33/cmake/plugin.cmake#L96-L102

MANDATORY

次に MANDATORY を指定した場合。これは必須ストレージエンジン or Plugin という意味合いで、見た範囲では InnoDB にこれがついている。

debug: ARCHIVE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib
debug: BLACKHOLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib
debug: CSV, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson
debug: EXAMPLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib
debug: FEDERATED, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson
debug: HEAP, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson
debug: INNOBASE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson
debug: MYISAM, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library
debug: MYISAMMRG, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson
debug: NDBCLUSTER, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson
debug: PERFSCHEMA, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib
debug: TEMPTABLE, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson
debug: NGRAM_PARSER, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson;ngram_parser
debug: MYSQLX, MYSQLD_STATIC_PLUGIN_LIBS: archive;extra::rapidjson;ext::zlib;blackhole;extra::rapidjson;ext::zlib;csv;extra::rapidjson;example;ext::zlib;federated;extra::rapidjson;heap;heap_library;extra::rapidjson;innobase;sql_dd;sql_gis;ext::zlib;ext::lz4;extra::rapidjson;myisam;myisam_library;myisammrg;extra::rapidjson;ndbcluster;ndbclient_static;extra::rapidjson;perfschema;extra::rapidjson;ext::zlib;temptable;extra::rapidjson;ngram_parser;mysqlx;ext::libevent;ext::icu;mysqlxmessages_lite;libprotobuf-lite;extra::rapidjson;ext::lz4;ext::zstd;ext::zlib

これは実装を見ても WITH_${plugin} = 1 を設定しているので実装からしても合っている。

  IF(ARG_MANDATORY)
    SET(WITH_${plugin} 1)
    SET(WITHOUT_${plugin} 0)
  ENDIF()

https://github.com/mysql/mysql-server/blob/mysql-8.0.33/cmake/plugin.cmake#L131C1-L134C10

おわりに

というわけで 3 パターンのビルド方法で unittest をサポートすることが出来ると分かったが、直前の記事でややミスってしまったので自信がない。

それが僕には楽しかったんです。

僕と MySQL と時々 MariaDB