MDEV-39092 Copy Aria data and logs as part of backup

[mariadb-andrzejjarzabek]

This is an initial simple implementation which copies all the Aria files in the "end" phase of the backup. Nothing protects the copy from concurrent DDL or DML. Copying only works on MacOS (intended for refactoring to use common file copy method across engines and SQL layer).

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[dr-m]

This is fundamentally incompatible with the handlerton::backup_step API. Even if you are for now copying everything in handlerton::backup_end, the internal API should be kept as compatible with handlerton::backup_step as possible: copying one file at a time. At least the copy_databases() step must be refactored so that the higher-level API is invoking something that copies one file at a time. Possibly, the copying of log files should be interleaved with that, like innodb_backup_step is doing.

[mariadb-andrzejjarzabek]

This is a stop-gap to get something working. Eventual final implementation will require re-working the hadlerton API and Sql_cmd_backup to take into account that different subsets of files may be copied under diffefent levels of metadata lock, and also "start" and potentially even "end" may need to be split into different phases with different levels of metadata lock. Given that the stage API is written that way to support multi-threaded copy, which isn't implemented at this time, I propose to merge the change as proposed (I will fix some of the other problems you mentioned) and iteratively improve from there.

[dr-m]

This is adding unnecessary object code duplication. Macros would allow zero-overhead abstraction:

diff --git a/sql/sql_backup_interface.h b/sql/sql_backup_interface.h
index d781acff3d9..b7d04705985 100644
--- a/sql/sql_backup_interface.h
+++ b/sql/sql_backup_interface.h
@@ -21,6 +21,7 @@
 # include <copyfile.h>
 # define copy_file(src, dst, off) \
   fcopyfile(src, dst, nullptr, COPYFILE_ALL | COPYFILE_CLONE)
+# define copy_entire_file(src, dst) copy_file(src, dst,)
 #else
 # ifdef __cplusplus
 extern "C"
@@ -32,4 +33,15 @@ extern "C"
 @return error code (negative)
 @retval 0   on success */
 int copy_file(int src, int dst, off_t size);
+
+# ifdef __cplusplus
+extern "C"
+# endif
+/** Copy an entire file.
+@param src  source file descriptor
+@param dst  target to append src to
+@param size amount of data to be copied
+@return error code (negative)
+@retval 0   on success */
+int copy_entire_file(int src, int dst);
 #endif

Outside Windows and Apple, it does make sense to introduce a non-inline function for copy_entire_file() even though it rather small (48 bytes for AMD64). I will include this in the next update of #4817.

[mariadb-andrzejjarzabek]

Would defining it as an inline function not be better then? Macros have the problem of replacing tokens lexically, so if there's any identifier in code called copy_file and this header is included, it will get messed up. The fact that it would only happen on a Mac makes this problem even worse.

[dr-m]

We have a mandatory macOS builder https://buildbot.mariadb.org/#/builders/708 that must pass in order for anything to be merged to a main branch. Hence, compilation failures should only be possible in development branches.

The benefit of defining a macro copy_file() is that the third argument will be guaranteed to be omitted from the generated code. If we had an inline function that ignores the third parameter, some unnecessary code could be emitted related to the evaluation of that parameter.