Release 0.9.0 (February 19, 2018)¶
Highlights¶
- UI improvements with large data feeds
- Improved Datasource Browsing
- Wrangler enhancements with data type conversions and domain type detection
- Exporting Visual Query to file or Hive table
- Kylo Localization support
- NiFi 1.5.0 support
- Connect Reusable templates together
- Improved Provenance handling for streaming feeds
- Security enhancements
- Entity Access Control for Elasticsearch
- Ability to reindex historical feed data for Elasticsearch and Solr
- New high-water mark policy to automatically cancel unreleased water marks
- Feed Versioning
- Save Point processor
- Numerous issues fixed.
Upgrade Instructions from v0.8.4.1¶
- Backup any Kylo plugins
When Kylo is uninstalled it will backup configuration files, but not the /plugin jar files. If you have any custom plugins in either kylo-services/plugin or kylo-ui/plugin then you will want to manually back them up to a different location.
- Uninstall Kylo:
/opt/kylo/remove-kylo.sh
- Install the new RPM:
rpm –ivh <RPM_FILE>
Restore previous application.properties files. If you have customized the the application.properties, copy the backup from the 0.8.3 install.
4.1 Find the /bkup-config/TIMESTAMP/kylo-services/application.properties file
- Kylo will backup the application.properties file to the following location, /opt/kylo/bkup-config/YYYY_MM_DD_HH_MM_millis/kylo-services/application.properties, replacing the “YYYY_MM_DD_HH_MM_millis” with a valid time:
4.2 Copy the backup file over to the /opt/kylo/kylo-services/conf folder
### move the application.properties shipped with the .rpm to a backup file mv /opt/kylo/kylo-services/conf/application.properties /opt/kylo/kylo-services/conf/application.properties.0_8_3_template ### copy the backup properties (Replace the YYYY_MM_DD_HH_MM_millis with the valid timestamp) cp /opt/kylo/bkup-config/YYYY_MM_DD_HH_MM_millis/kylo-services/application.properties /opt/kylo/kylo-services/conf
4.3 Copy the /bkup-config/TIMESTAMP/kylo-ui/application.properties file to /opt/kylo/kylo-ui/conf
4.4 Ensure the property
security.jwt.key
in both kylo-services and kylo-ui application.properties file match. The property below needs to match in both of these files:/opt/kylo/kylo-ui/conf/application.properties
/opt/kylo/kylo-services/conf/application.properties
security.jwt.key=
NOTE: Kylo no longer ships with the default dladmin user. You will need to re-add this user only if you’re using the default authentication configuration:
- Uncomment the following line in
/opt/kylo/kylo-services/conf/application.properties
and/opt/kylo/kylo-ui/conf/application.properties
:security.auth.file.users=file:///opt/kylo/users.properties security.auth.file.groups=file:///opt/kylo/groups.properties
- Create a file called
users.properties
file that is owned by kylo and replace dladmin with a new username and thinkbig with a new password:echo "dladmin=thinkbig" > /opt/kylo/users.properties chown kylo:users /opt/kylo/users.properties chmod 600 /opt/kylo/users.properties
- Create a file called
groups.properties
file that is owned by kylo and set the default groups:vi /opt/kylo/groups.propertiesdladmin=admin,user analyst=analyst,user designer=designer,user operator=operations,userchown kylo:users /opt/kylo/groups.properties chmod 600 /opt/kylo/groups.properties
To enable reindexing of a feed’s historical data:
Verify option in
/opt/kylo/kylo-services/conf/application.properties
for Kylo services. This is true by default.search.history.data.reindexing.enabled=true
If using Solr instead of Elasticsearch as the search engine, add one property to
/opt/kylo/kylo-services/conf/solrsearch.properties
file.config.http.solr.url=http://${search.host}:${search.port}
Start Kylo
/opt/kylo/start-kylo-apps.sh
Once Kylo is up, to enable reindexing of a feed’s historical data, complete these remaining steps:
Import the updated Index Text Service feed via these steps:
- Feed Manager -> Feeds -> + orange button -> Import from file -> Choose file
- Pick the
index_text_service_hs_v<version_number>.feed.zip
file available at/opt/kylo/setup/data/feeds/nifi-1.3/history-reindexing/
- Leave Change the Category field blank (It defaults to System)
- Click Yes for these three options (1) Overwrite Feed (2) Replace Feed Template (3) Replace Reusable Template
- Click Import Feed.
- Verify that the feed imports successfully.
Import the History Reindex Text Service feed via these steps:
- Feed Manager -> Feeds -> + orange button -> Import from file -> Choose file
- Pick the
history_reindex_text_service_hs_v<version_number>.feed.zip
file available at/opt/kylo/setup/data/feeds/nifi-1.3/history-reindexing/
- Leave Change the Category field blank (It defaults to System)
- Click Yes for these three options (1) Overwrite Feed (2) Replace Feed Template (3) Replace Reusable Template
- Click Import Feed.
- Verify that the feed imports successfully.
Highlight Details¶
Kylo Localization support
- Kylo’s UI is capable of supporting localization. Contact Think Big Analytics for translation options available under Kylo’s Enterprise subscription support.
NiFi 1.5.0 support
- Kylo now works with NiFi 1.5.0. If you have NiFi 1.5.0, You should still use the spring profile
nifi-v1.2
in thekylo-services/conf/application.properties
file.
Improved Provenance handling for streaming feeds
- Streaming feed evetns will process much faster in JMS. Streaming feeds are now derived within NiFi and the events will no longer be sent to the thinkbig.feed-manager batch JMS queue.
- Security enhancements
- Entity Access Control for Elasticsearch
- Ability to reindex historical feed data for Elasticsearch and Solr
- Improved Release Highwatermark so flows wont get stuck
- Save Point processor
- A newly added Savepoint NiFi processor and ControllerService allows you to configure flows and retry failures within Kylo Operations Manager.