The current import-sql is implemented using shell script + xargs to parallelize file import.
It currently has a number of limitations, mostly dealing with error handling -- if one of the SQL files fail, it is not trivial to find which file failed and where, plus in some cases the script (xargs) can exit without waiting for files that have not yet completed - thus confusing the output even further.
One simple solution would be to rewrite it in Python, allowing more control and helping readability.
See also #373
The current import-sql is implemented using shell script +
xargsto parallelize file import.It currently has a number of limitations, mostly dealing with error handling -- if one of the SQL files fail, it is not trivial to find which file failed and where, plus in some cases the script (xargs) can exit without waiting for files that have not yet completed - thus confusing the output even further.
One simple solution would be to rewrite it in Python, allowing more control and helping readability.
See also #373