Setup Wizard Deployment Guide¶
Follow the steps below to install Kylo using the installation wizard script. This is convenient for local sandboxes (HDP/Cloudera) and single node development boxes. The WGET command is used to download binaries so internet access is required.
The setup wizard is designed for easy installation of all components on one node.
Installing Kylo installs the following software at these Linux file system locations:
- Kylo Applications - /opt/kylo
- Java 8 - /opt/java/current
- NiFi - /opt/nifi/current
- ActiveMQ - /opt/activemq
- Elasticsearch - RPM installation default location
The steps below require root access.
Step 1: Download the RPM¶
Download the RPM and place it on the host Linux machine that you want to install Kylo services on.
To use wget instead, right-click the download link and copy the url.
Download the Latest RPM
Step 2: Create the Linux Users/Groups¶
Creation of users and groups is done manually because many organizations have their own user and group management system. Therefore, it cannot be scripted as part of the RPM install. Here is an example of how to create the users and groups:
$ useradd -r -m -s /bin/bash nifi $ useradd -r -m -s /bin/bash kylo $ useradd -r -m -s /bin/bash activemq
Validate that the above commands created a group by looking at /etc/group. Some operating systems may not create them by default.
$ cat /etc/group
If the groups are missing then run the following:
$ groupadd kylo $ groupadd nifi $ groupadd activemq
Step 3: Run the Kylo RPM Install¶
$ rpm -ivh kylo-<version>.noarch.rpm
The RPM is hard coded at this time to install to /opt/kylo.
Step 4: Optional - Generate TAR file for Offline Mode¶
To run the wizard on an edge node with no internet access, generate a TAR file that contains everything in the /opt/kylo/setup folder including the downloaded application binaries.
- Install the Kylo RPM on a node that has internet access.
- Run the offline install:
|Note||If installing the Debian packages make sure to change the Elasticsearch download from RPM to DEB|
- Copy the /opt/kylo/setup/kylo-install.tar file to the node you install the RPM on. This can be copied to a temp directory. It doesn’t have to be put in the /opt/kylo/setup folder
- Run the Kylo TAR install:
tar -xvf kylo-install.tar
The script downloads all application binaries and puts them in their respective directory in the setup folder. Last it will TAR up the setup folder.
Step 5: Run the Setup Wizard¶
If installing in an HDP or Cloudera sandbox, choose option #2 on the Java step to download and install Java in the /opt/java/current directory.
- From the /opt/kylo/setup directory
- Offline mode from another directory (using TAR file)
$ <PathToSetupFolder>/setup/setup-wizard.sh -o
Both -o and -O work.
Follow the directions to install the following:
- MySQL or Postgres scripts into the local database
- Java 8 (If the system Java is 7 or below)
- NiFi and the Kylo dependencies
The Elasticsearch, NiFi, and ActiveMQ services start when the wizard is finished.
Step 6: Add “nifi” and “kylo” Users¶
In this step, add “nifi” and “kylo” users to the HDFS supergroup, or to the group defined in hdfs-site.xml. For example:
$ usermod -a -G hdfs nifi $ usermod -a -G hdfs kylo
$ groupadd supergroup # Add nifi and hdfs to that group: $ usermod -a -G supergroup nifi $ usermod -a -G supergroup hdfs
Optional: If you want to perform actions as a root user in a development environment run the below command:
$ usermod -a -G supergroup root
Step 7: Additional Cluster Configuration¶
In addition to adding the nifi/kylo user to the supergroup on the edge node, add the users/groups to the name nodes on a cluster.
$ useradd kylo $ useradd nifi $ usermod -G hdfs nifi $ usermod -G hdfs kylo
TBD (need to test this out)
Step 8: Create a Dropzone Folder¶
$ mkdir -p /var/dropzone $ chown nifi /var/dropzone
Files should be copied into the dropzone such that user nifi can read and remove.
Step 9: Cloudera Configuration (Cloudera Only)¶
See the appendix section below “Cloudera Configuration File Changes”.
Step 10: Edit the Properties Files¶
Step 11: Start the Three Kylo Services and NiFi¶
$ /opt/kylo/start-kylo-apps.sh $ service nifi start
At this point, all services should be running. Note that services are started automatically on boot.
Appendix: Cloudera Configuration File Changes¶
The configuration is setup to work out of the box with the Hortonworks sandbox. There are a few differences that require configuration changes for Cloudera.
- Update the 3 MySQL password values to “cloudera”:
spring.datasource.password=cloudera metadata.datasource.password=cloudera hive.metastore.datasource.password=cloudera modeshape.datasource.password=cloudera
- Update the Hive username:
- Update the Hive Metastore URL:
- Update the following parameters: