Add in newer Ubuntu in supported platform, change python bootstrappin…

README.md

@@ -29,156 +29,25 @@ Other firmware development links
 * [Unstructured firmware development notes](https://github.com/makerbot/Toolchain-Release/wiki/Unstructured-firmware-development-notes)
 
 ### Ubuntu setup
-* Install some basic utilities
 
-```bash
-sudo apt-get install build-essential cmake-curses-gui
-sudo apt-get install git python python-setuptools python-software-properties python-pip
-sudo apt-get install openssh-server lsb-release devscripts cdbs debhelper libudev-dev libjpeg-dev genext2fs libusb-1.0-0-dev
-```
-
-* If you are using Ubuntu Precise (12.04), get a new computer
-
-* If you are using Ubuntu Xenial (16.04):
-
-Manually install Python3.4:
-```bash
-mkdir python-build && cd python-build
-wget https://www.python.org/ftp/python/3.4.5/Python-3.4.5.tgz
-tar xf Python-3.4.5.tgz
-cd Python-3.4.5
-./configure --prefix=/usr --enable-shared  # this is important for firmware builds especially
-make
-sudo make altinstall  # this will install Python3.4 alongside 3.5 instead of overwriting it
-```
-Alternatively, you can install Python3.4 from the deadsnakes repo:
-```bash
-sudo add-apt-repository ppa:deadsnakes/ppa
-sudo apt-get update
-sudo apt-get install python3.4 python3.4-dev python3-pip
-```
-
-If you are going to build firmware:
-
-* Install 32-bit libc:
-
-```bash
-    sudo apt-get install libc6-i386 libssl-dev
-```
-* Install Qt:
-
-    Recommended that you do not use apt-get to install Qt. Goto https://www.qt.io/download-open-source/ to download the Qt Open Source Installer. Select Qt version 5.10.1 when performing install. You will need to make symbolic links for the qmake, lrelease, and lupdate binaries.
-
-```bash
-    sudo ln -s /path/to/Qt/5.10.1/gcc_64/bin/qmake /usr/local/bin/qmake
-    sudo ln -s /path/to/Qt/5.10.1/gcc_64/bin/lrelease /usr/local/bin/lrelease
-    sudo ln -s /path/to/Qt/5.10.1/gcc_64/bin/lupdate /usr/local/bin/lupdate
-```
-
-Install yajl and boost:
-```bash
-sudo apt-get install libyajl-dev
-sudo apt-get install libboost1.58-all-dev
-sudo apt-get install mercurial  # only necessary if you want to do full build-root builds
-```
-
-Pip install some additional dependencies:
-
-```bash
-    sudo pip install protobuf
-```
-
-* Use git to download our main repository
+#### Toolpath Quick Setup
 
+If only building toolpath:
 ```bash
-    git clone git@github.com:makerbot/Toolchain-Release.git
-    cd Toolchain-Release/
+sudo apt update
+sudo apt install -y build-essential cmake git ninja-build libboost-dev-all libyajl-dev python2.7 npm
 ```
-
-* Add our internal master repository
-* Use double quotes ("), not single quotes(')
-* This command will automatically add the correct distribution
-
+The update script still requires python2:
 ```bash
-sudo apt-add-repository "http://sid.soft.makerbot.net/apt/develop"
-sudo apt-get update
+python3 -m pip install -U pip
+python3 -m pip install jsonschema dohq-artifactory wheel
 ```
-
-* In the root of the Toolchain-Release repository there is a file called ./ubuntusetup.sh
-If you run it it will install all of the build dependencies for you
-(if it doesn't go bug Ryan)
-
-* Either make sure you do not have qt4 installed, or add the following to your environment:
-`export QT_SELECT=qt5`
-
-* You also need python3.4 with pip and some extra python packages:
-
+The build can be invoked with
 ```bash
-pip3 install wheel jsonschema aiohttp netifaces appdirs lockfile watchdog
-sudo pip3 install artifactory
+python3 update-repos.py -t=toolpather
+python3 build-repos.py
 ```
 
-* From the `Toolchain-Release` directory invoke the following command:
-`./update-repos.py --toolchain=desktop`
-This will pull down either the source or artifacts of each of our other repositories depending default settings
-* After downloading all of the sources run
-`./build-repos.py`
-
-If there are any errors related to missing packages please bother Ryan to update the ubuntusetup script(or do it yourself)
-
-This command will take a long time to finish. It will download the source
-package tarballs for all of our projects, extract them, then for each one
-it will install the build dependencies and compile the project.
-
-
-### Mac-specific Tools Setup
-
-[Setup instructions for 10.10](https://github.com/makerbot/Toolchain-Release/blob/develop/Bundle-Mac/OSX_10_10_setup.md).
-These instructions are up to date as of August 2016, and are probably still more correct
-than anything else even for different OSX versions.
-
-Once setup is done you should be able to run any of the following,
-depending on what you want to build:
-
-```bash
-./update-repos.py --toolchain=penguin  # dependencies for makerbot print
-./update-repos.py --toolchain=sliceconfig
-./update-repos.py --toolchain=toolpathviz2
-```
-
-and then run `./build-repos.py` to actually build.  You may even be able to build the `desktop`
-toolchain (i.e. makerbot desktop 3.x), although in all likelihood there is still something
-missing from the setup instructions.
-
-### Windows-specific Tools Setup.
-* Install Microsoft Visual Studio 2015 (Professional with Update 3 should work fine): https://my.visualstudio.com/Downloads?q=visual%20studio%202015
-* Install CMake v3.4 from http://www.cmake.org/download/ (add it to the path for all users)
- * Note that CMake v3.5 and up will cause boost library link errors, since the FindBoost module has changed and our current adapter no longer works
-* Install Git from https://git-scm.com/download/win (check the 'use Git and optional Unix tools from the Windows command prompt' when the option comes up and 'checkout as-is, commit unix-style line endings')
-
-The following instructions assume you're doing a 64 bit build, substitute the 32 bit versions of any of the following:
-
-* Install latest version of Python 2. At the time of writing, 2.7.18 was the latest version.
-  * preferably the 64-bit version at https://www.python.org/ftp/python/2.7.18/python-2.7.18.amd64.msi
-* Install Boost(again ask for an updated version, 1.61 at time of writing)
-    64bit: https://sourceforge.net/projects/boost/files/boost-binaries/1.61.0/boost_1_61_0-msvc-14.0-64.exe/download
-* Now we need to set up the environment and install some python modules from pip
-  * Run `pip install requests urllib3`
-  * Run `pip install http://y.soft.makerbot.net/files/artifactory-0.1.11-py2-none-any.whl`. It installs our custom artifactory python 2 module to your system. Functions like `get_dependencies.py` use it to communicate with our artifactory server
-* Install nvm: https://github.com/coreybutler/nvm-windows
-* Install node (right now we're using 14.15.3)
-  * Run `nvm install 14.15.3`
-
-
-* Open a command prompt, create a directory called `Toolchain` and cd into it. Run `git clone git@y.soft.makerbot.net:third-party-win.git`. Enter the password when prompted. If you don't know the password, ask your team lead or authorized personnel.
-
-* While in Toolchain directory run `git clone git@github.com:makerbot/Toolchain-Release.git` to download Toolchain-Release repo
-
-* `cd` into `Toolchain/third-party-win` and run `./setup_devtools.bat`
-then close and re-open command prompt to make the changes take
-
-* (Optional) Install NSIS so you can build the installer http://nsis.sourceforge.net/Download
-
 ### Getting our sources & building
 
 * At the root of the Toolchain-Release repo run `update-repos.py`. This will run a bunch of git commands to pull down additional repositories and their submodules into the "Toolchain" subdirectory. It also installs a set of git-hooks that do style checking and other things. You can run this again at any time to pull down the latest updates for everything.

build_tools/platforms.py

@@ -31,6 +31,8 @@
 UBU_1610_32 = 'Ubuntu_1610_32'
 UBU_1610_64 = 'Ubuntu_1610_64'
 UBU_1804_64 = 'Ubuntu_1804_64'
+UBU_2204_64 = 'Ubuntu_2204_64'
+UBU_2304_64 = 'Ubuntu_2304_64'
 
 FED_20_32 = 'Fedora_20_32'
 FED_20_64 = 'Fedora_20_64'
@@ -56,6 +58,8 @@
     UBU_1510_64,
     UBU_1604_64,
     UBU_1804_64,
+    UBU_2204_64,
+    UBU_2304_64,
     FED_20_32,
     FED_20_64,
     FED_21_32,
@@ -106,8 +110,12 @@ def find_platform(Win32=False):
         # TODO(ted): why msvc 12?
         platform_string = 'Windows_MSVC14_' + bitness
     elif utils.is_linux:
-        (distro, version, codename) = platform.linux_distribution()
-        platform_string = distro + "_"
+        try:
+            (distro_str, version, codename) = platform.linux_distribution()
+        except AttributeError:
+            from distro import linux_distribution
+            (distro_str, version, codename) = linux_distribution()
+        platform_string = distro_str + "_"
         platform_string += str.replace(version, '.', '') + "_"
         platform_string += bitness
     else:

cmake-tools/MBCMakeToolsConfig.cmake

@@ -9,18 +9,27 @@ cmake_minimum_required(VERSION 3.1)
 # we manually set them so that the build system devs can use them.
 # I'm not setting all of them, because some don't seem to have any
 # bearing on us.
-if(${CMAKE_MAJOR_VERSION} EQUAL 3 AND ${CMAKE_MINOR_VERSION} GREATER 1)
-    # strict break()
-    cmake_policy(SET CMP0055 NEW)
-elseif(${CMAKE_MAJOR_VERSION} EQUAL 3 AND ${CMAKE_MINOR_VERSION} GREATER 2)
-    # Use COMPILE_DEFINITIONS not DEFINITIONS directory property
-    cmake_policy(SET CMP0059 NEW)
-    # CTest does not by default tell make to ignore errors
-    cmake_policy(SET CMP0061 NEW)
-    # Disallow install() of export() result.
-    cmake_policy(SET CMP0062 NEW)
+if (${CMAKE_MAJOR_VERSION} EQUAL 3)
+
+    if(${CMAKE_MINOR_VERSION} GREATER 1)
+        # strict break()
+        cmake_policy(SET CMP0055 NEW)
+    endif()
+
+    if(${CMAKE_MINOR_VERSION} GREATER 2)
+        # Allow use of ALL over lists in IF
+        cmake_policy(SET CMP0057 NEW)
+        # Use COMPILE_DEFINITIONS not DEFINITIONS directory property
+        cmake_policy(SET CMP0059 NEW)
+        # CTest does not by default tell make to ignore errors
+        cmake_policy(SET CMP0061 NEW)
+        # Disallow install() of export() result.
+        cmake_policy(SET CMP0062 NEW)
+    endif()
+
 endif()
 
+
 # Make makefiles show each command line as it is used.
 set(CMAKE_VERBOSE_MAKEFILE TRUE)
 

cmake-tools/python34.cmake

@@ -91,7 +91,7 @@ macro(add_python34)
             set(PYTHON_INCLUDE_DIR
                 "${CMAKE_INSTALL_PREFIX}/${HEADER_INSTALL_DIR}/python3.4m"
                 CACHE INTERNAL "")
-            find_package(PythonLibs 3.4 EXACT REQUIRED)
+            find_package(PythonLibs   REQUIRED)
             set(PYTHON_DEV_HOME
                 "${CMAKE_INSTALL_PREFIX}/${RESOURCE_INSTALL_DIR}/python34")
         elseif(${CMAKE_SYSTEM_NAME} MATCHES "Darwin")
@@ -106,19 +106,19 @@ macro(add_python34)
             set(PYTHON_INCLUDE_DIR
                 "${CMAKE_INSTALL_PREFIX}/${HEADER_INSTALL_DIR}/python3.4m"
                 CACHE INTERNAL "")
-            find_package(PythonLibs 3.4 EXACT REQUIRED)
+            find_package(PythonLibs   REQUIRED)
             # This only works if LIB_INSTALL_DIR ends with /lib
             set(PYTHON_DEV_HOME
                 "${CMAKE_INSTALL_PREFIX}/${LIB_INSTALL_DIR}/..")
         else()
-            find_program(PYTHON34_EXECUTABLE python3.4)
+            find_program(PYTHON34_EXECUTABLE python)
             # This fails on linux when we are not building a CXX project
             if(CMAKE_CXX_COMPILER_LOADED)
-                find_package(PythonLibs 3.4 EXACT REQUIRED)
+                find_package(PythonLibs   REQUIRED)
             endif()
         endif()
         if(NOT PYTHON34_EXECUTABLE)
-            message(FATAL_ERROR "Python 3.4 not found")
+            message(STATUS "Python 3.4 not found")
         endif()
 
         # Store the basic command for calling python 34 in PYTHON34_COMMAND
@@ -323,7 +323,7 @@ function(build_wheel target_name)
         RESULT_VARIABLE error)
 
     if(${error})
-        message(FATAL_ERROR "Could not determine wheel name")
+        message(STATUS "Could not determine wheel name")
     endif()
 
     # So, we've been trying pretty hard to maintain that under cmake

dev-tools/artifactory_clean_repo_diagnostic.py

@@ -68,9 +68,9 @@
 for repo in repos:
     space_made = 0;
     for plat in plats:
-        all = list(artifactory_utils.all(repo, platform=plat))
-        all = sorted(all, key=lambda art: art.stat().mtime)
-        old = [i for i in all if
+        all_artifacts_unsorted = list(artifactory_utils.all(repo, platform=plat))
+        all_artifacts = sorted(all_artifacts_unsorted, key=lambda art: art.stat().mtime)
+        old = [i for i in all_artifacts if
             (datetime.today()-i.stat().mtime.replace(tzinfo=None)) > age_cutoff]
 
         #for i in all: