[bootstrap-vcpkg.sh] Do not download tool on Android

[podsvirov]

Reopen #45497

Answer to @BillyONeal comment:

To be clear, I was suggesting docker because I know that's a system where I know one should be able to start from "nothing", post command lines, and we should be able to show that it works by following those exact lines. End to end instructions on an Android device would be OK too. But the aforementioned problem where bootstrap needs to explain needs to get fixed before we would merge this.

Try it via Dockerfile (edited):

FROM termux/termux-docker:x86_64

SHELL ["/entrypoint.sh", "bash", "-c"]

RUN pkg update && pkg install -yq git zip clang cmake ninja

RUN git clone https://github.com/microsoft/vcpkg.git

WORKDIR ${HOME}/vcpkg

RUN git fetch origin pull/51073/head && git checkout FETCH_HEAD

# To fix error: vcpkg was unable to find a libcurl.so.4
RUN cp $PREFIX/lib/libcurl.so $PREFIX/lib/libcurl.so.4

RUN ./bootstrap-vcpkg.sh

# For example, now installing flecs, that missing in termux
RUN pkg install -yq patchelf && ./vcpkg install --triplet x64-linux --host-triplet x64-linux flecs

[podsvirov]

Ok, vcpkg-tool can be build on Android, but why now it print error:

error: vcpkg was unable to find a libcurl.so.4, libcurl-gnutls.so.4, or libcurl-nss.so.4 to use on this system. Please install libcurl from your system package manager and retry vcpkg.

The shared library libcurl.so available:

$ dpkg -L libcurl | grep '\.so'
/data/data/com.termux/files/usr/lib/libcurl.so

[podsvirov]

@robertkirkman, can we add symlink/copy of shared library (with name libcurl.so.4) in libcurl termux's package?

[dg0yt]

vcpkg tries to dynamically load a suitable shared object (dlopen). This has a version suffix on most systems. But Android doesn't use more than the .so suffix for shared objects.

https://github.com/microsoft/vcpkg-tool/blob/e0612b42ce44e55a0e630f2ee9d3c533a63d8bc1/src/vcpkg/base/curl.cpp#L43

You could try using a linked libcurl, setting VCPKG_LIBCURL_DLSYM to OFF.
https://github.com/microsoft/vcpkg-tool/blob/main/cmake/FindLibCURL.cmake

[podsvirov]

You could try using a linked libcurl, setting VCPKG_LIBCURL_DLSYM to OFF

But haw to set it via bootstrap-vcpkg.sh script options?

[BillyONeal]

@robertkirkman, can we add symlink/copy of shared library (with name libcurl.so.4) in libcurl termux's package?

You could also just add it to here: https://github.com/microsoft/vcpkg-tool/blob/e0612b42ce44e55a0e630f2ee9d3c533a63d8bc1/src/vcpkg/base/curl.cpp#L41-L53

[podsvirov]

You could also just add it to here: https://github.com/microsoft/vcpkg-tool/blob/e0612b42ce44e55a0e630f2ee9d3c533a63d8bc1/src/vcpkg/base/curl.cpp#L41-L53

Done in microsoft/vcpkg-tool#1984, please review.

[BillyONeal]

I drafted this because it doesn't seem to work until we do another tool release to pick up your change. However, the out of the box bootstrap instructions need to help the user install the right bits.

Thanks for your submission!

[BillyONeal]

The dependency detection bits still need to be updated for the system you're trying to target.

[robertkirkman]

@podsvirov If I'm understanding correctly, what is meant is that, when the bootstrap-vcpkg.sh is run without having the necessary zip and other dependencies installed in Termux previously, this message that prints when that happens:

tacokoneko@CORSAIR ~/code/termux/podsvirov $ docker build .
[+] Building 29.0s (10/11)                                                                               docker:default
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 480B                                                                               0.0s
 => [internal] load metadata for docker.io/termux/termux-docker:x86_64                                             0.3s
 => [internal] load .dockerignore                                                                                  0.0s
 => => transferring context: 2B                                                                                    0.0s
 => CACHED [1/8] FROM docker.io/termux/termux-docker:x86_64@sha256:d25e324f88e44519c821db0df8117b0b6719d7bfd3a59b  0.0s
 => [2/8] RUN apt update                                                                                           2.8s
 => [3/8] RUN apt install -y git                                                                                  12.0s
 => [4/8] RUN git clone https://github.com/microsoft/vcpkg.git                                                     9.7s 
 => [5/8] WORKDIR /data/data/com.termux/files/home/vcpkg                                                           0.3s 
 => [6/8] RUN git fetch origin pull/51073/head && git checkout FETCH_HEAD                                          2.3s 
 => ERROR [7/8] RUN ./bootstrap-vcpkg.sh                                                                           1.3s 
------                                                                                                                  
 > [7/8] RUN ./bootstrap-vcpkg.sh:                                                                                      
1.307 Could not find zip. Please install it (and other dependencies) with:                                              
1.307 On Debian and Ubuntu derivatives:                                                                                 
1.307   sudo apt-get install curl zip unzip tar                                                                         
1.307 On recent Red Hat and Fedora derivatives:                                                                         
1.307   sudo dnf install curl zip unzip tar
1.307 On older Red Hat and Fedora derivatives:
1.307   sudo yum install curl zip unzip tar
1.307 On SUSE Linux and derivatives:
1.307   sudo zypper install curl zip unzip tar
1.307 On Arch Linux and derivatives:
1.307   sudo pacman -Syu base-devel git curl zip unzip tar cmake ninja
1.307 On Alpine:
1.307   apk add build-base cmake ninja zip unzip curl git
1.307   (and export VCPKG_FORCE_SYSTEM_BINARIES=1)
1.307 On Solaris and illumos distributions:
1.307   pkg install web/curl compress/zip compress/unzip
------
Dockerfile:15
--------------------
  13 |     RUN git fetch origin pull/51073/head && git checkout FETCH_HEAD
  14 |     
  15 | >>> RUN ./bootstrap-vcpkg.sh
  16 |     
  17 |     # For example, now installing flecs, that missing in termux
--------------------
ERROR: failed to solve: process "/entrypoint.sh bash -c ./bootstrap-vcpkg.sh" did not complete successfully: exit code: 1

should not only include directions for Desktop Linux distros - it should also include directions for Termux and pkg install followed by the Termux package names for the needed dependencies.

[podsvirov]

You understand everything correctly.

[robertkirkman]

@robertkirkman, can we add symlink/copy of shared library (with name libcurl.so.4) in libcurl termux's package?

You could also just add it to here: https://github.com/microsoft/vcpkg-tool/blob/e0612b42ce44e55a0e630f2ee9d3c533a63d8bc1/src/vcpkg/base/curl.cpp#L41-L53

I agree that ideally, PRs like this to add Termux support for things should include handling of the current names of .so files provided by Termux, even if they are different from those of GNU/Linux, with exceptions only for cases where Termux developers have decided to intervene and force the versioned .so filename to be provided.

In Termux, whether .so files have numbers at the end of their filenames is mostly determined by what the upstream code of the library in question does when its build system detects it is being built targeting Android.

Before Android API level 23 (Android 6.0), Android did not support versioned SONAMEs. However, Termux uses API level 24, has not supported older API levels for a long time, and all later API levels support versioned SONAMEs.

Some libraries from the GNU/Linux ecosystem are completely unaware of Android and will build the same .so filenames that they do for GNU/Linux, but some others that are aware of Android, especially those which include maintained or vestigial compatibility with Android 5 or older, will detect Termux as Android at build-time (but not necessarily as "API level 24" Android) and change the names of the .so files they provide to be different from their names on GNU/Linux.

Generally, Termux developers avoid changing the SONAMEs that are set by default by each library package built for Termux, without an important reason.

@podsvirov thank you for making this update, I will test it soon as well

[BillyONeal]

Termux uses API level 24

Since you seem to be familiar with this, can you comment on #51081 ?

[robertkirkman]

Termux uses API level 24

Since you seem to be familiar with this, can you comment on #51081 ?

If Vcpkg is primarily for distributing native libraries and executables for developers to put together as part of separate build scripts, rather than for use as a build system that generates completely standalone Android APKs that someone might want to upload to Google Play, in my opinion it should continue to target the oldest available minimum API level that is convenient to until an important reason or user appears that needs it to be increased.

This is because if native libraries or executables are built targeting an Android API level that is recent, they will completely stop working on old Android devices, and could cause crashes in Gradle projects that are trying to support an older minimum API level than the Vcpkg package provided, when the APKs they build are attempted to be run on Android devices that have the minimum API level of that Gradle project.

There is a technical change that occurs when crossing the point of changing the target API level of the APK's Gradle project to 29 or newer, which is that the Android API begins to limit and discourage basic, uncomplicated Gradle projects from running executable native ELF files rather than shared libraries (but does not completely ban it in an uncircumventable way for apps that are willing to implement complex workarounds)

There is a very deep rabbit hole of information here and here that could be explained about what happens if you try to use a Terminal Emulator APK on Android that was compiled targeting API level 29 or higher, but I think the information I should say that is for the most part relevant to this situation, is this:

I believe that running native libraries and executables built for minimum API levels 24 through 28 does continue to work for the most part within the environments of apps whose APK Gradle projects target API level 29 or newer, and in the case of executables, continues to mostly work, to the degree that the Gradle project that built the APK implements the necessary complex workarounds for running native executables (of any API level, whether older or newer than 29) within an APK that targets API level 29 or newer.

Termux has a variant APK which targets API level 36 and is currently available on Google Play, but that variant of Termux is experimental and is intended as a test of the viability of a Terminal Emulator App APK targeting API level 36. It is currently less stable than the recommended stable Termux version, v0.118.3, which has a minimum API level of 24, but it (the API level 36 Termux on Google Play) does work and it does still have the capability of running most native executables and libraries that were compiled with minimum API levels of 24 through 28, though I believe it might have fewer bugs with native executables and libraries that were compiled with minimum API levels (and patches to adapt to the minimum API levels) of 29 and newer.