SystemDS Install from source - SystemDS 3.0.0
Apache SystemDS™
3.0.0
Toggle navigation
Overview
Home:
Docs Home
Running SystemDS:
Standalone Guide
GPU Guide
Native Backend (BLAS)
Run with Docker
Language Guides:
DML Language Reference
Built-in Functions Reference
DML vs R guide
Algorithms:
ML Algorithms Reference
Other:
Contributing to SystemDS 🡕
GitHub 🡕
API
Java
Python
Issues
SystemDS Install from source
This guide helps in the install and setup of SystemDS from source code.
Windows Guide
Ubuntu/Linux Guide
Mac Guide
Windows
Developer Guide
Ubuntu 20.04
Java and Maven
First setup java and maven to compile the system note that the java version is 1.8.
sudo apt install openjdk-8-jdk-headless
sudo apt install maven
Note: To update the java command to openjdk-8 run:
update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
Verify the install with:
java -version
mvn -version
This should return something like:
openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1ubuntu1-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)
Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 1.8.0_252, vendor: Private Build, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.4.0-37-generic", arch: "amd64", family: "unix"
Testing
R is required to be install to run the test suite, since many tests are constructed to comprare output with common R packages.
One option to install this is to follow the guide on the following link: https://linuxize.com/post/how-to-install-r-on-ubuntu-20-04/
At the time of writing the commands to install R 4.0.2 are:
sudo apt install dirmngr gnupg apt-transport-https ca-certificates software-properties-common
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/'
sudo apt install r-base
Optionally, you need to install the R depedencies for integration tests, like this:
(use sudo mode if the script couldn’t write to local R library)
Rscript ./src/test/scripts/installDependencies.R
See Build the project to compile the code from here.
MAC
Prerequisite install homebrew on the device.
# To allow relative paths:
brew install coreutils
# To install open jdk 8.
brew tap adoptopenjdk/openjdk
brew cask install adoptopenjdk8
# Install maven to enable compilation of SystemDS.
brew install maven
Then afterwards verify the install:
java --version
mvn --version
This should print something like:
Java version: 1.8.0_242, vendor: AdoptOpenJDK, runtime: /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/jre
Default locale: ru_UA, platform encoding: UTF-8
OS name: "mac os x", version: "10.15.5", arch: "x86_64", family: "mac"
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/Cellar/maven/3.6.3_1/libexec
Note that if you have multiple java versions installed then you have to change the used version to 8, on both java and javadoc. This is done by setting the environment variable JAVA_HOME to the install path of open JDK 8 :
export JAVA_HOME=`/usr/libexec/java_home -v 1.8`
For running all tests r-base has to be installed as well since this is used as a secondary system to verify the correctness of our code, but it is not a requirement to enable building the project.
Optionally, you need to install the R depedencies for integration tests, like this:
(use sudo mode if the script couldn’t write to local R library)
Rscript ./src/test/scripts/installDependencies.R
See Build the project to compile the code from here.
Build the project
To compile the project use:
mvn package -P distribution
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  31.730 s
[INFO] Finished at: 2020-06-18T11:00:29+02:00
[INFO] ------------------------------------------------------------------------
The first time you package the system it will take longer since maven will download the dependencies.
But successive compiles should become faster.
Now everything is setup and ready to go!
To execute dml scripts look at Execute SystemDS